• Perhaps the most important milestone for DDS followers was the recommendation for adoption of the new specification Extensible and Dynamic Topic Types for DDS (sometimes abbreviated “DDS-XTypes”). I wrote about DDS-XTypes late last year; it will make it much easier for DDS users to upgrade and evolve their systems over time. It will also enable whole new categories of plug-and-play tools and integration components — such as database connectors, recorders, and bridging/routing services — to be developed in a way that’s portable and interoperable across DDS implementations, increasing the size and diversity of the DDS marketplace and increasing competition. RTI was a lead author of this specification, and a number of the capabilities defined in it are available in pre-standard form in RTI Data Distribution Service today.
• The meeting also saw the presentation of three initial submissions, representing six different companies, to the Web-Enabled DDS RFP that was released last year. (The previous link is only accessible to OMG members; sorry. When it’s finished, the specification will be publicly available.) This very strategic initiative will produce a specification for the integration of systems based on DDS with those based on web technologies. For the first time, it will be possible for SOAP- or REST-based client applications to consume data flowing along a DDS data plane, and for them to provide services back to applications in that plane, in a standards-based way with COTS technology. RTI is currently developing an implementation of this technology in parallel with the standard; this component, called RTI Web Integration Service, will be included in the Professional and Elite Editions of RTI Data Distribution Service. You can see a video demo of DDS-web integration, based on a pre-release version of this component, on YouTube.
• Work continues on further improving the portability and usability of DDS implementations with updated C++ and Java API bindings. The improved C++ APIs are already in the revised submission stage, and we expect them to be adopted within the next small number of OMG meetings. The improved Java APIs will be presented as initial submissions at the next OMG meeting, to be held in Minneapolis in June. RTI’s initial proposal will be based on the DataDistrib4J APIs, which we have made available in open source.
• Finally, the group had a lively discussion of DDS security, especially with respect to data tagging and labeling. We expect to take these discussions to a formal RFP in the coming months.

I think it’s a testament to the strength of the DDS ecosystem that while its core standards remain stable and support significant real-world deployments, at the same time the depth and breadth of that ecosystem continues to grow. It’s an exciting time to be involved with this technology.

One of the really powerful things about DDS is that it brings to distributed systems the same kind of type safety that you’ll find in local applications. In addition to reducing errors, this deep knowledge of data types can improve performance and resource usage by reducing the number of data copies in the system and easing integration with other field- and type-aware technologies, including relational databases and even Microsoft Excel.

But as systems evolve over time, type definitions can evolve too, and it’s important that applications that are already deployed don’t break as the types used by new applications change. It’s also desirable to ease the development of infrastructure or cross-cutting components — like tools, recorders, generic data routing and transformation facilities, and others — that shouldn’t be tied to specific data types. DDS users have been solving these problems in a variety of ways for some time, and some implementations address them already, but it’s time for a standardized solution.

To that end, the OMG is working on a new specification, Extensible and Dynamic Topic Types for DDS, that will provide additional capabilities for the following:

• A clarified and extended type system that incorporates keys and extensibility as first-class concepts
• An API for the definition of new data types at run-time without code generation
• A reflective API for the construction, inspection, and manipulation of data samples based on dynamic type definitions
• The ability to define data types declaratively using not only OMG IDL but XML and XML Schema (XSD) as well for easier integration with enterprise systems

The proposed specification will be discussed at the OMG Technical Meeting next month and some outstanding open issues addressed. I expect the proposal to be voted on and approved at a subsequent meeting not far in the future.

If your organization is an OMG member, you can access the in-progress specification proposal documents yourself.

So, while getting started supporting customers developing distributed edge applications with real-time needs at RTI, I found some expected, and some unexpected, differences in how developers built applications for the “edge” compared to the enterprise. While the (embedded) distributed application architects paid closer attention to the “physics” as devices were low powered, networks more complex and sometimes ad-hoc, and microseconds & memory mattered, the relative lack of attention that they paid to the information model was very interesting.

(Reference: Read a paper as a result of this insight: “How Does Your Real-Time Data Look?”)

The technologists in edge environments spend significant time tuning the network links, but they often miss significant opportunities to make optimal use of available bandwidth by not focusing (enough) on data modeling. I recently saw a “mission and performance critical” data model that is put on the wire with over six levels of nesting in its data structure…

This relative lack of attention to the data model, while regrettable, can be better understood if we take into account that until recently, edge devices were weak (could not collect or process enough information), few (not choking the network, though bandwidth is always an issue), or not (richly) context-aware (taking advantage of other information available on the network). Since only a (relatively) few bits were published on a “functionally light” middleware,  the information model was not very consequential.

However, as edge applications become more complex — from Command and Control to Monitoring — the science of tuning the information model for a distributed application can benefit from the advances in building information models for enterprise applications. With the devices and networks becoming more powerful, and with middleware such as RTI Data Distribution Service putting more intelligence on the network, the bottleneck more and more is not the hardware, or the capabilities of a high-performance or a functionally-rich middleware such as RTI Data Distribution Service, but the inefficiencies introduced by a poorly designed data model.

(Reference: Read this paper on what you can expect a modern high-performance middleware to do: “Is DDS for You?”)

What is required is that while designing the information model for the distributed application, architects of high-performance distributed applications sit with network middleware experts to ensure that the information model fully leverages the middleware’s capabilities, such as local processing (Content-Filtered Topics), message aggregation, time-based filtering, and using sparse types to only send the updated fields on the wire…