With this deluge of information, because of the gap between the large volume of data produced and people’s ability to process the information, operators may even be less informed than before. For the information to be processed correctly, it needs to be integrated and interpreted correctly. In addition, the system must provide the operator with the information in a way that is usable cognitively and physically. The system should be designed in such a way so as to support the operator under dynamic operational constraints. This is what Situational Awareness is about — about knowing what important things are going around you.

At the basic level, Situational Awareness (SA) is about determining what information is relevant and how to collect that information. The next level extends SA’s capabilities by understanding that information. This includes how people combine, interpret, store, and retain information. This also includes how information is integrated from multiple sources. In addition to integrating and classifying the information, it is also important to determine how important that information is.

How much of situational awareness in enough? While it is important to have as much information as possible, it is imperative that more information does not come at the cost of processing (comprehending) information that is classified as being more important. The next level –“level 3” — deals with acting on the information. This also includes forecasting future events, and providing recommendations on how to react to such events. The way in which the operator directs her attention in acquiring and processing information fundamentally impacts situational awareness.

Time is a critical ingredient in achieving situational awareness – how much time is available until a specific event occurs or some action must be taken, using time to correlate multiple events of interest. Space is another aspect – correlating events of interest within a space. Lastly, real-world situations are dynamic – they keep changing. So it is important to incorporate the rate at which the information is being updated in the design of a SA system.

Ultimately, the role of building an SA system is for enabling operators to make the correct decisions in a timely manner. It is entirely possible for the operator to have a perfect SA system and still make an incorrect decision. This could be due to poor strategies, tactics or training, among other reasons. So it imperative to have, where possible, a linking between recognizing a situation and taking a decision based on that recognition. Of particular importance is the technique of pattern-matching to recognize information belonging to a known class of situations. With experience, the pattern-recognition/action-selection sequence can become automated and reduce demands on the operator.

To summarize, SA is about creating a model where the system state is captured, including creating an understanding how that state is affected by projected events. A good SA model integrates relevant information from multiple sources, determines the relative importance of different events, and projects the state of the system based on events. This also implies that, to build a system that is situation aware, the model must be accurate and must have the ability to be updated to reflect the current events.

With the technologies available from Real-Time Innovations (RTI), application developers can build an open, standards-based platform that can be used to collect, integrate, and analyze the information required for building a situational aware model for monitoring the enterprise health of a network. This framework will integrate with third-party sensors and probes (using algorithms for sensor fusion) for the accumulation of information and will leverage the  latest advances in groupware applications for providing situational awareness to the operator and enabling them to make distributed decisions in a cooperative manner.

With RTI Data Distribution Service, we can address the following needs of situation-aware applications:

• Enable integration of heterogeneous sensors, across domains and networks
• Provide dynamic, evolvable and type-safe data representation & encapsulation
• Provide minimally-intrusive, efficient, scalable, and real-times-aware collected-data distribution
• Evaluate advances in Service Oriented Architecture (SOA) for enabling integration of information from multiple sources
• Conduct post-attack analysis to determine new patterns for future threat detection

In addition, with complimentary RTI technologies like  Complex Event Processing (CEP), RTI can address the following needs:

• Provide event correlation, through time and space, from multiple sensors
• Apply algorithms from sensor fusion when multiple sensors are observing the same situation
• Determine what data to collect to fight through a cyber conflict, how to protect the security of the network, and how to provide autonomic response to attacks including reconfiguration, recovery, and reconstitution while allowing mission-critical systems to continue to function

For more information on building a situation-aware model using RTI, contact info@rti.com.

In real-time applications, speed and performance matter, and often define the success of the project.

The designer of a sensor-based distributed application (with real-time needs) needs to factor in the topology of the network. The equivalent of this is that a CRM architect needs to account for the speed to disk rotation for designing her SQL query (which she does not need to do since the disk problem is localized and the applications do not care about microseconds).

While getting a faster middleware such as RTI Data Distribution Service is part of the solution, …

(Reference: RTI Data Distribution Services performance numbers)

… the comprehensive way to address this issue is by distributing the intelligence in the network. In many cases, the individual sensor read is not as important on its own – only when it crosses its threshold (example: Ganglia sensor reporting > 25% disk usage) or when the event happens in context of another event (example: Ganglia sensor reporting > 25% disk usage AND new port opened within 10 milliseconds on the same node) does the data become meaningful for transmission to (say) an intrusion-detection engine.

So, besides designing an efficient information model, architects of distributed applications with real-time needs also need to move as much information processing as possible close to the source of the data. This is useful both to protect the bandwidth of the network link and to ensure that the subscriber of the data is not overwhelmed with the rate of data that it need to consume. That is where an intelligent middleware comes in… RTI Data Distribution Service provides many rich features which can be enabled through parameters. One interesting feature is Content-Filtered Topics, which letssubscribers use SQL-like expressions to determine which data are they interested in, saving network resources and CPU.

Another technology that is really useful in adding intelligence to the network is Complex Event Processing, which provides users RDBMS-like operations on streaming data, enabling lower processing latency (since the data does not need to be persisted on the disk before running the queries).

(Reference: Read how Complex Event Processing adds intelligence to the distributed system)

With Complex Event Processing (CEP) you can build an application that is more context aware (since it lets you correlate different data streams based on time or samples) with only the meaningful data published on the wire for consumption.

So, this blog ends with the same note as the last post. Distributed systems architects need to think more beyond just tuning the network link. By using an intelligent middleware, by making intelligent choices about their information model, they can circumvent many challenges posed by physics.

So, while getting started supporting customers developing distributed edge applications with real-time needs at RTI, I found some expected, and some unexpected, differences in how developers built applications for the “edge” compared to the enterprise. While the (embedded) distributed application architects paid closer attention to the “physics” as devices were low powered, networks more complex and sometimes ad-hoc, and microseconds & memory mattered, the relative lack of attention that they paid to the information model was very interesting.

(Reference: Read a paper as a result of this insight: “How Does Your Real-Time Data Look?”)

The technologists in edge environments spend significant time tuning the network links, but they often miss significant opportunities to make optimal use of available bandwidth by not focusing (enough) on data modeling. I recently saw a “mission and performance critical” data model that is put on the wire with over six levels of nesting in its data structure…

This relative lack of attention to the data model, while regrettable, can be better understood if we take into account that until recently, edge devices were weak (could not collect or process enough information), few (not choking the network, though bandwidth is always an issue), or not (richly) context-aware (taking advantage of other information available on the network). Since only a (relatively) few bits were published on a “functionally light” middleware,  the information model was not very consequential.

However, as edge applications become more complex — from Command and Control to Monitoring — the science of tuning the information model for a distributed application can benefit from the advances in building information models for enterprise applications. With the devices and networks becoming more powerful, and with middleware such as RTI Data Distribution Service putting more intelligence on the network, the bottleneck more and more is not the hardware, or the capabilities of a high-performance or a functionally-rich middleware such as RTI Data Distribution Service, but the inefficiencies introduced by a poorly designed data model.

(Reference: Read this paper on what you can expect a modern high-performance middleware to do: “Is DDS for You?”)

What is required is that while designing the information model for the distributed application, architects of high-performance distributed applications sit with network middleware experts to ensure that the information model fully leverages the middleware’s capabilities, such as local processing (Content-Filtered Topics), message aggregation, time-based filtering, and using sparse types to only send the updated fields on the wire…