Data distribution is about observing a changing world. A system whose communication is based on this paradigm tends to become data-centric: it becomes more concerned with modeling the first-class concepts of its business domain and less concerned with managing second-class “who-told-whom-to-do-what” middleware concepts like queues and messages. Along the way, it enjoys the benefits of decreased coupling and improved reliability, scalability, and performance.

Data Distribution and Its Kin

Classically, messaging is an evolution of the remote method invocation (RMI) paradigm — an attempt to make that paradigm less coupled and more scalable by making it asynchronous. A message says “I tell you to do this.” When compared with RMI, “I” and “you” are more abstract, both in identity and multiplicity, and the request can be queued for processing at a later time or by another party without making the sender wait. These are improvements, but the interaction remains coupled, because the roles of “I” and “you” (often in the guises of “client” and “server” or the trendier “service consumer” and “service provider”), as well as the intention of what action should be performed, are still very much in play.

Eventing, like data distribution, is preoccupied with changes to the world. An event says “I changed in this way.” It reduces coupling by entirely removing both the recipient of that information and any notion of intention from you business logic and your mental model; who might receive an event, and what they might choose to do as a result, are not the business of the event source. But state management remains a problem, because in order to understand the change that occurred, all recipients must have an up-to-date understanding of the state of the world prior to the latest event — “the price went up by a dollar” doesn’t do me any good if I don’t know what the price was before. This temporal coupling means that every recipient must process every event in order, whether those events are interesting or not, just in case the interpretation of a subsequent interesting event should happen to require the state established by a previous otherwise-uninteresting one.

The resulting processing and state management are complex and expensive. As a mitigation, they are frequently factored out of the applications that need the data and into state-management “servers” that “clients” must query using a message-centric or even RMI-based approach — a huge regression in engineering practice! The system becomes complicated by the presence of multiple interacting communication paradigms, and the servers (which serve no business role) introduce performance and fault-tolerance choke points.

A data-centric architecture eliminate these problems by simplifying the interactions. A data sample says simply “the world is like this.” It thereby eliminates coupling not only in terms of source, recipients, and their intentions, but also in terms of time. There’s no longer any need for recipients to process or store information they don’t care about, because samples don’t implicitly encompass previous samples. Therefore it becomes perfectly reasonable for one observer to examine the state of the world every second, or every minute, or every hour — and for another to observe every single intermediate state, even if those states change from one to the other many times a second.

Modeling the World with DDS

A set of DDS entities, and the data they distribute and manage, define a view into this changing “world.”

• A “domain” defines the boundaries of the world, the set of information that a collaborating group of applications might find interesting. A “domain participant” defines the presence of some application in that world; it is the data-centric analogue to what is frequently known as a “connection” in the messaging middleware.
• A “type” is a structural description of some part of the world — for example, an Antelope is brown in color and has four legs and two horns; a Ferrari is red in color and has four wheels and two seats. A type has a formal definition, usually (though not always) in a declarative language like XSD or OMG IDL, and it implies a corresponding definition in the target programming language.
• A “quality-of-service” (QoS) definition defines the fidelity with which some party/parties is/are able to describe the world. For example, will the description contain every state the world passes through or only a subset? Will observers have access to new states of the world only, or will they be able to see previous states as well? If the latter, how far back will those previous states go?
• A “topic” defines some aspect or subset of the world consisting of similar objects. As such, it combines a type, which defines the structure of those objects, with a QoS definition, which defines how they can be observed to change.
• An “instance” defines a single object in the group defined by a topic. For example, a topic may be used to distribute the positions of airplanes as detected by a radar. Each plane would be an instance. All radar tracks have the same structure (type) and are updated in the same way (QoS). But they are also distinct from one another: it matters whether the plane at a given location happens to be American Airlines flight 123 or Delta flight 456.
• A “data writer” defines a source of information about a particular subset of the world (topic). As such, it may override the QoS of its topic — multiple parties may provide information about the same part of the world but with different degrees of fidelity.
• A “data reader” defines an observer of a particular subset of the world (topic). As such, it may also override the QoS of its topic. Furthermore, it may only be able and/or interested to observe certain states of the world. For example, it may only be interested in airplanes flying over a particular geographic area or in stocks trading at over $20/share.

By creating a data reader with a certain QoS definition, an application makes an affirmative statement that it wishes to observe a certain portion of the world under a certain set of circumstances. For example, it may state that it is interested in observing the most recent five states (samples) to the objects (instances) in its part of the world (topic), but it doesn’t need to process changes more frequently than once every second.

This statement is one of interest only; it in no way requires the observer to actually observe a certain set of samples in a certain way or within a certain period of time. On the one hand, the observer may choose to be notified asynchronously of every new sample and to respond to it immediately. On the other, it may “go away” to other business and return hours later; when it does, it will find the most recent five samples of each instance, occurring no more frequently than once every second, waiting for it. In the mean time, DDS will have taken care of all of the necessary data reception, filtering, and replacement in order to make that happen.

DDS’s ability to combine notification and lightweight caching — in effect, to maintain an application’s observed state of the world on its behalf — is something no other standards-based technology provides. Developers of data-centric systems reap the benefits: higher performance and scalability, greater tolerance to dynamic network conditions, and ultimately improved ROI and time-to-market.

The answers to these questions are becoming increasingly important as real-time systems scale up and are integrated into Systems of Systems. Without the right underlying architecture and technology, applications will fail to satisfy their performance objectives. For mission-critical systems, the consequences of this could be catastrophic. Integration costs could also explode if proprietary and stovepipe workarounds are required to compensate for limitations in the underlying integration infrastructure.

You can download the white paper here.

Check it out and let us know what you think. What does “Real-Time SOA” mean to you?

• Perhaps the most important milestone for DDS followers was the recommendation for adoption of the new specification Extensible and Dynamic Topic Types for DDS (sometimes abbreviated “DDS-XTypes”). I wrote about DDS-XTypes late last year; it will make it much easier for DDS users to upgrade and evolve their systems over time. It will also enable whole new categories of plug-and-play tools and integration components — such as database connectors, recorders, and bridging/routing services — to be developed in a way that’s portable and interoperable across DDS implementations, increasing the size and diversity of the DDS marketplace and increasing competition. RTI was a lead author of this specification, and a number of the capabilities defined in it are available in pre-standard form in RTI Data Distribution Service today.
• The meeting also saw the presentation of three initial submissions, representing six different companies, to the Web-Enabled DDS RFP that was released last year. (The previous link is only accessible to OMG members; sorry. When it’s finished, the specification will be publicly available.) This very strategic initiative will produce a specification for the integration of systems based on DDS with those based on web technologies. For the first time, it will be possible for SOAP- or REST-based client applications to consume data flowing along a DDS data plane, and for them to provide services back to applications in that plane, in a standards-based way with COTS technology. RTI is currently developing an implementation of this technology in parallel with the standard; this component, called RTI Web Integration Service, will be included in the Professional and Elite Editions of RTI Data Distribution Service. You can see a video demo of DDS-web integration, based on a pre-release version of this component, on YouTube.
• Work continues on further improving the portability and usability of DDS implementations with updated C++ and Java API bindings. The improved C++ APIs are already in the revised submission stage, and we expect them to be adopted within the next small number of OMG meetings. The improved Java APIs will be presented as initial submissions at the next OMG meeting, to be held in Minneapolis in June. RTI’s initial proposal will be based on the DataDistrib4J APIs, which we have made available in open source.
• Finally, the group had a lively discussion of DDS security, especially with respect to data tagging and labeling. We expect to take these discussions to a formal RFP in the coming months.

I think it’s a testament to the strength of the DDS ecosystem that while its core standards remain stable and support significant real-world deployments, at the same time the depth and breadth of that ecosystem continues to grow. It’s an exciting time to be involved with this technology.

We’re dropping that time again. Recently, we rolled out an additional new capability on our web site: the RTI Product Tour. (At least for now, the Product Tour is also linked from the www.rti.com home page.) The Product Tour covers many of the components of RTI Data Distribution Service in a streamlined way with embedded screenshots and demonstration videos. Instead of downloading the software, installing it, and reading through the documentation, you can just click on a couple of five-minute videos to get some basic information with a lot less effort. (We hope that afterward you’ll still want to download the software to try it for yourself!)

So if you’re interested in RTI’s decoupled system architecture, industry-leading performance and scalability, built-in value-added features like recording and analysis, or powerful connectors to enterprise and legacy systems, check out the RTI Product Tour. It’ll just take a few minutes.

Like the U.S. Interstate Highway System, air traffic control systems are based on 50-year old technologies, in the case of air traffic control, ground-based radars and voice communications over radio.

Ground-based radar antennas rotate 5 or 12 times per minute “pinging” aircraft every 12 seconds (or 5 seconds near airports). It is thus possible for aircraft to be miles away from where the controllers think they are. As a consequence, aircraft must be spaced 3-5 miles apart (horizontally).

Air traffic communications have not kept up with digital communications technology. Controllers still issue mundane commands like, “Descend to five thousand (feet),” “Fly heading 270 (fly due west),” and “Reduce speed to 180 (knots).” Furthermore, pilots rely on air traffic controllers to advise them of nearby aircraft.

Meanwhile, higher fuel expenses are motivating the aerospace industry and airlines to experiment with new technologies and procedures because more efficient flying means lower fuel burn, and fuel is the biggest or second largest expense for airlines.

In the United States, a large air cargo carrier uses Automatic Dependent Surveillance-Broadcast (ADS-B) to exchange GPS data with other aircraft in their fleet. Pilots can track the position and direction of similarly equipped aircraft, just like a marine radar on any passenger ship. This is especially useful every night when about one hundred aircraft converge on their hub airport in a narrow time window. Without the positional awareness made possible by ADS-B, disruptions, such as storms, cause traffic jams in the air, forcing them to fly in circles while waiting for their turn to land. With ADS-B units in the cockpit, pilots can space themselves prior to arrival, thus reducing congestion at the hub airport and saving fuel.

Over the Pacific Ocean, the ASPIRE (Asia Pacific Initiative to Reduce Emissions) is testing operational optimizations to burn less fuel on transpacific flights. ASPIRE flights are allowed to take advantage of every optimization during their oceanic flights, most importantly greater freedom to change their altitude and heading based on actually encountered atmospheric conditions, as opposed to conditions forecast hours before flight. This is enabled by precision navigation (GPS) and better air-ground communications. On a recent ASPIRE flight, a 747 flying from LAX to Singapore, via Narita, Japan burned 10,868 kg less fuel, while emitting 33,769 kg less CO2 (carbon dioxide).

Unmanned aerial vehicles (UAV) present new challenges to air traffic management (ATM). In Europe, the ATLANTIDA initiative (Application of Leading Technologies to Unmanned Aerial Vehicles for Research and Development in ATM) is tackling the challenge of applying Trajectory-Based Operations (TBO) to air traffic management (ATM) for Unmanned Aerial Vehicles (UAV). Instead of human air traffic controllers proactively controlling aircraft movements, which is especially challenging with unmanned vehicles, TBO takes aircraft trajectory data and computes optimal solutions and presents decisions to the air traffic controller. ATLANTIDA uses a net-centric service-oriented architecture along with a novel trajectory definition technology called Aircraft Intent Description Language (AIDL) to capture and distribute trajectory data, and has selected RTI Data Distribution Service for its implementation.

The benefits of better air traffic control and management are more efficient flying and lower fuel burn. Consequently emissions are lower. RTI is playing a role in the future of air traffic management and helping the world be a little greener.

Click here for more articles on DDS. Click here for whitepapers on DDS, including a link to the Embedded Market Forecasters study that finds DDS middleware from RTI improves ROI when compared with other alternatives.

This broad-based research provides the first independent quantification of the reduction in integration time, cost and risk you can expect from RTI Data Distribution Service. Findings include:

• Up to 45% Lower Total Cost of Development: The average cost of application development was substantial for projects using internally developed “Roll-Your-Own” (RYO) middleware ($1.61M) and most commercial solutions ($1.34M); however, projects using RTI middleware enjoyed much lower costs ($0.89M).
• Up to 47% Lower Cost Overrun: The average cost overrun was similar for projects using RYO (11.3%) and most commercial middleware (10.1%). Projects using RTI finished closest to expected cost (6.0%).
• Lower Testing Costs: In projects where the cost of testing was less than 30 percent of the total development cost, RYO (72.5%) showed an advantage over commercial (65.5%) middleware. Projects using RTI’s commercial middleware, however, had testing costs less than 30 percent of the total development cost 84.6% of the time.
• Greater Probability of Meeting Design Requirements: Final design outcomes using commercial middleware in general, and RTI in particular, were much closer to pre-design expectations than RYO developments for performance, functionality, features and schedule.

Visit EMF’s web site to download the full report for free:  Choosing between Commercial and ‘Roll Your Own’ Embedded Communication Integration Middleware.

NASA was RTI’s first customer.  In fact, NASA funded the research at the Stanford Aerospace Robotics Laboratory that spawned the technology that became RTI and the DDS standard.

The progress in the NASA program during that time is stunning.  In the 1990’s, robot controllers were clunky boxes with primitive sensors and no real connection off board.  It was a huge accomplishment just to wander around dragging a huge umbilical cord for power and control.  Today’s program connects impressive vision systems, planners, and controllers.  They can be controlled live or run nearly autonomously.  The computing system networks dozens of processors in ground stations and vehicles.  The stovepipe systems (and rivalries!) at the different research centers years ago gave way to common system architectures that allow efficient sharing of code, data, and research progress.  The robots can work independently for long periods in realistic environments.  The researchers can work together in shorter periods on realistic progress.  That’s the Way Things Should Be…

Of course, the networking technology has also made great progress.  The middleware grew from a specialized data server to a general-purpose international standard.  From our research beginnings, RTI now claims hundreds of designs in a dozen industries, including many real-world mission-critical, 24×7 applications.  We are not alone; the DDS standard is backed by multiple vendors in a growing, competitive market.  That’s also the Way Things Should Be.

Anyway, I’m glad we could be some small part of this program.  I want to congratulate NASA on its decades of progress towards the vision of enabling capable, cost effective exploration of our universe.  The sky may be the limit.  It’s refreshing to see and work with those who know it’s the lower one.

With the maturity and broad adoption of DDS, we are seeing a couple of trends.

• DDS is being used in larger and more geographically disperse systems
• Customers are moving to second-generation DDS based systems
• Users are integrating multiple systems that already deploy DDS as their underlying integration bus

To support these efforts, RTI recently introduced RTI Routing Service. RTI Routing Service provides a flexible solution for scaling DDS systems and for integrating disparate DDS applications. This includes:

• Applications that cannot directly communicate because they run on different networks (LAN and WAN), use different transport protocols (e.g., shared memory, IPv4 and IPv6), or are members of different security domains
• Applications that natively use different DDS data types, such as new and legacy applications, individual systems within a System of Systems, and applications that support different Communities of Interest (COI)

To learn how RTI Routing Service significantly reduces the costs of real-time system integration, upgrades and of implementing Cross-Domain Solutions (CDS), visit RTI’s web site or watch this video demonstration.

Many developers suffer from confusion with respect to the differences between mutexes and semaphores. Michael Barr of Netrino provides solid information in his article Mutexes and Semaphores Demystified. My summary: mutexes protect shared resources by enforcing mutual exclusion; semaphores should be used for signaling across tasks.

Niall Cooling of Feabhas provides a longer discussion in two parts:

1. Mutex vs. Semaphores – Part 1: Semaphores
2. Mutex vs. Semaphores – Part 2: The Mutex

Cooling focuses especially on the problems that arise when developers use semaphores where they should rather use mutexes. In particular, he discusses accidental lock release, various causes of deadlock, and priority inversion. (The latter is of special interest to developers of real-time software.)

Although semaphores and mutexes are critical concurrency-control structures, the APIs to use them unfortunately differ from platform to platform with respect to both form and function. At the end of his Part 2, Cooling briefly summarizes some of the differences between the semaphore and mutex APIs of several operating systems. This section will be of interest to anyone developing systems that will be ported across very different platforms. Chris Simmonds goes into more depth, with respect to Linux in particular, in his article Mutex mutandis: understanding mutex types and attributes posted on The Inner Penguin.