Are You Considering Migrating from PrismTech OpenSplice to RTI Connext DDS? 1

migrateToConnextDDSBlog.jpg

With the recent acquisition of PrismTech by the Taiwanese company ADLINK, we are seeing increased demand for porting from OpenSplice to RTI Connext DDS. One of the benefits of going with a standard like Data Distribution Service for Real-Time Systems (DDS) is that you have options if your middleware supplier becomes unreliable for any reason. That’s why the DDS community went to so much trouble to develop more than just a wire protocol standard. The DDS specification also includes API definitions, with the explicit goal of making it easier to port.

In particular, DDS specifications include both a PIM (platform independent model) and a set of language PSMs (platform specific models). The PIM defines all the important user-visible API concepts including the DDS Entities (DomainParticipant, Topic, Publisher, Subscriber, etc.), their operations and behavior, the QoS, the listeners, and so on. All DDS implementations use the PIM. That means that the structure and major options in your code will translate unchanged to a different DDS implementation.

The PSMs are much newer and they are only unambiguously defined for C++ and Java. Other programming languages come from the IDL-derived mappings (also standard). If you used one of the new standard language PSM APIs, it’s even easier to port your application. Even if you didn’t, the common PIM makes it a straightforward process. Of course, in general, there are some differences between versions, both API variability and “non standard” options, features, and configuration. Still, compared to porting to a new middleware architecture, the work required to port is minimal.

How hard is it to port an existing application from OpenSplice DDS to the RTI Connext DDS in practice? 

It’s not trivial, but it’s not that bad, either. One significant difference between these two SDKs is in the IDL syntax. Although most of the IDL syntax is identical, there is a difference in how “key attributes” are indicated. Connext DDS uses the DDS-XTYPES standard format consisting of a “@Key” annotation placed inside a comment. OpenSplice uses a (non-standard) “#pragma keylist” directive. We can see this difference in the example below.

OpenSpliceDDS IDL Equivalent Connext DDS IDL
struct Coordinates {

   float latitude;

   float longitude;

   float altitude;

};
struct Flight  {

   string<32>  airlineCode;

   long        flightNumber;

   string<3>   departureAirport;

   string<3>   arrivalAirport;

   long long   departureTime;

   Coordinates currentPosition;

};
#pragma keylist Flight airlineCode flightNumber

struct Coordinates {

   float latitude;

   float longitude;

   float altitude;

};
struct Flight  {

   string<32>  airlineCode;  //@Key

   long        flightNumber; //@Key

   string<3>   departureAirport;

   string<3>   arrivalAirport;

   long long   departureTime;

   Coordinates currentPosition;

};

 

The first step in the migration is to modify the IDL files replacing #pragma keylist with the //@Key annotation, as shown above.

Once the IDL file(s) have been migrated, the rtiddsgen tool can be used to generate all the network marshaling/unmarshaling code. This allows the data to understood by any computer, independent of processor architecture, operating system, and programming language. The rtiddsgen tool is included with the Connext DDS SDK. In addition to generating the network marshaling code, rtiddsgen can generate example code and makefiles / build projects for the programming languages and platforms of your choice.

Beyond this there are a few application code edits that may be required. These affect the initialization of the TypeSupport, the use of the ReturnCode_t type, the use of the length() operation on types String and Sequence, and the way sequences are passed to the DataReader read/take operations.

RTI has experience doing this! Some RTI customers, including many of our largest and happiest customers, started with OpenSplice before they discovered the performance, quality, tools, and top-flight support from RTI. So, before you start this effort, contact us. We have a migration SDK including compatible headers, examples, and more. Better yet, we can offer you a cost-effective service to help you transition quickly and correctly.

“See” what is going on with your DDS System Reply

RTI - GUEST POST - SIMVENTIONS

 

simventions

Matt Wilson is the Vice President for Tools and Technology at SimVentions. Mr. Wilson has been designing, building, and integrating complex system and related software based tools for over 20 years. His experience with Naval Combat Systems, Human Systems Integration, and Systems Engineering projects has been the knowledge base for the design and development of a wide variety of innovative and useful tools.

 

Have you ever had to figure out what is going on inside of your DDS-based system and had no idea how to begin? You have many data topics being published to and subscribed by a multitude of data readers and data writers across a vast array of processes. There are a img1wide variety of scenarios from mis-matched properties, out of date message definitions, missing connections, etc. How do you begin to “look at” this problem?

SimVentions has developed a tool called InformeDDS (Info for me – DDS) that uses network graphing technologies and DDS discovery data to “show you” what is going on in your network. InformeDDS’ ability to auto-discover DDS entities (participant, publisher/subscriber, data writer/reader, topic) and then automatically provide an interactive visual representation of connections between DDS entities and all relevant attributes allows you to quickly “see” what is happening without impacting your network.

img2 Using one of the preset (or personally customized) display filters helps reduce complicated networks to a meaningful subset to allow you to focus on key issues. Each of these filters can be built, copied, tailored, and improved through a simple user interface to highlight attributes unique to your own system. We call these filters “View Profiles” which provide you the ability to set your own visualization business rules or use the ones that come out of the box.

InformeDDS has proven to be a useful tool for software developers during testing and debugging and for performing integration testing on complex systems. It has helped us (SimVentions) debug and troubleshoot networking issues for our own customers and can be quickly learned to bring the same value to your integration and testing efforts. During a recent visit to one of our large systems integration partners, an on-site engineer commented “Now THAT is an integration tool” after using InformeDDS.img3

SimVentions is proud to be an RTI partner. To try out InformeDDS with the latest version of RTI Connext DDS (5.2.0) please use the RTI Launcher to download and install the tool.

We continue to improve the InformeDDS technology and look forward to hearing your feedback about this capability. There are many other features in the tool that we will introduce to you in future stories. Until then, download and try the free trial of this innovative and useful debug and integration tool for your DDS network today.

For more information, visit the following sites or leave a note in the comments section of this post!

Rapid Data Transformations Are Moments Away! 4

When I started college, everybody was talking about “Information Technology.” At that point I had been programming for quite a while and it wasn’t clear to me what coding had to do with that fancy terminology. After a few more years of coding, I realized the connection: all I do, day in and day out, is move bytes (information) from one memory location to another. Copying the contents of a struct into the socket buffer and sending it out; getting the bytes from the socket buffer and deserializing them into a structure to pass them to the application logic. Well, that’s part of what communication middleware does for you!

RTI Connext DDS implements the OMG DDS Standard. DDS is data-centric middleware. We call those bytes being transferred data, and we assign a type to them. The type describes each byte in your data, so you (and your application) can make good use of them.

Designing a good distributed system means having a good type system in place. Often though, those types carry a lot of information, and it can become pretty difficult to deal with them. Also, you may not need all the information all the time. This often happens when you have heterogeneous systems with nodes that have different capabilities, such as bandwidth constraints or resource limits. In this case, some of the nodes in the system may not be able to handle a whole sample for certain data types, but they still need to be able to receive part of the information.

Let me present a possible scenario first and then suggest one way to solve it.

scenario

Figure 1: Scenario

Let’s say a new testing module for the International Space Station (ISS) has been shipped to space. The module has a device that collects thousands of data points, puts them all in a DDS sample with a specific type, and writes it on the topic, ComplexDataTopic. We will call this device the Emitter.

One of the many different data points this device collects is the temperature from 10 different sensors. It puts all the collected temperature values into a sequence.

enum Unit {
   F,
   C,
   K
};

struct Temp {
    long sensorId;
    double value;
    Unit unit;
}

struct Measurements {
  long deviceId;
  // thousand of more fields. 
  sequence<Temp,10> temperatures;
};

// Open file

The module also comes with a FancyReceiver (what we call a topic subscriber) that runs a UI; it gets all the data contained in each sample and creates statistics and charts so astronauts can easily understand the data. For example, looking that the temperature data, it could calculate the average (mean) or the median, or spot significant differences between sensors.

Let’s now say that back on Earth, NASA scientists need to know what the temperature is on the testing module. They’re not interested in pressure and humidity. Just the temperature. But, since the bandwidth between Earth and the ISS “is what it is”, it’s considered acceptable to receive aggregates of the date. For example, instead of sending all the values for the temperature the testing module will only send the average, which will save the ISS some bandwidth on the link to ground. Basically, they want something that looks like this:

enum Unit {
   F,
   C,
   K
};

struct SimpleMeasurements {
    long deviceId;
    double avgTmp;
    Unit unit;
}

// Open File

We have many options to achieve this result:

  1. We can create an ad-hoc application: it will subscribe to the ComplexDataTopic, get the data we need, create another topic, another type, calculate what we need, and send it over.
  2.  We can use RTI Routing Service: write a custom transformation library and be done with it.
  3.  We can use RTI Connector via the RTI Prototyper with Lua.
  4. We can use RTI Connector via Python or node.js.

I will now explain option 3: Use RTI Prototyper with Lua to easily transform your types. If you are already an RTI Connext user, you will have what you need in your distribution (under the “bin” directory if you are using RTI Connext 5.2, or under the “scripts” directory for older versions). Otherwise, check out this page to learn how to get your free copy of RTI Prototyper.

We will create a new component, the Transformer. The Transformer will subscribe to ComplexDataTopic (with type Measurements) and will Publish to SimpleDataTopic (with type SimpleMeasurements).

The Types

For simplicity, let’s say that the complex type is the following:

<enum name="Unit">
  <enumerator name="F"/>
  <enumerator name="C"/>
  <enumerator name="K"/>
</enum> 
<struct name= "Temp">
  <member name="sensorId" id="0" type="long"/>
  <member name="value" id="1" type="double"/>
  <member name="unit" id="2" type="nonBasic"  
        nonBasicTypeName= "Unit"/>
</struct>
<struct name= "Measurements">
  <member name="deviceId" id="0" type="long"/>
  <member name="humidity" id="1" type="double"/>
  <member name="pressure" id="2" type="double"/>
  <member name="temperatures" sequenceMaxLength="10" id="3" 
        type="nonBasic"  nonBasicTypeName= "Temp"/>
</struct>

<!--Open File -->

The simple type looks like this:

<enum name="Unit">
  <enumerator name="F"/>
  <enumerator name="C"/>
  <enumerator name="K"/>
</enum> 
<struct name= "SimpleMeasurements">
  <member name="deviceId" id="0" type="long"/>
  <member name="avgTmp" id="1" type="double"/>
  <member name="unit" id="2" type="nonBasic"  
        nonBasicTypeName= "Unit"/>
</struct>

<!-- Open File -->

The Topics and the Entities

All we need to do now is define an XML application file with a data reader for topic ComplexDataTopic and a writer to SimpleDataTopic:

<!-- Domain Library -->
    <domain_library name="MyDomainLibrary">
        <domain name="MyDomain" domain_id="0">
            <register_type name="Measurement" kind="dynamicData" 
                         type_ref="Measurement"/>
            <register_type name="SimpleMeasurement" kind="dynamicData" 
                           type_ref="SimpleMeasurement"/>
            <topic name="ComplexDataTopic"  
                          register_type_ref="Measurement"/>
            <topic name="SimpleDataTopic"   
                          register_type_ref="SimpleMeasurement"/>
        </domain>
    </domain_library>
<domain_participant name="Transform" 
                     domain_ref="MyDomainLibrary::MyDomain">
            <participant_qos base_name="QosLibrary::DefaultProfile"/>
                <publisher name="MyPublisher">
                    <data_writer name="MySimpleWriter" 
                                       topic_ref="SimpleDataTopic" />
                </publisher>
                <subscriber name="MySubscriber">
                    <data_reader name="MyComplexReader" 
                                       topic_ref="ComplexDataTopic" />
                </subscriber>
        </domain_participant>

<!-- Complete File -->

The Actual Logic

Once you have defined what your types are and how your entities are called, you just have to write a simple chunk of Lua code that will be executed by RTI Prototyper. Let’s have a look:

local myComplexReader = 
    CONTAINER.READER['MySubscriber::MyComplexReader']
local mySimpleWriter  = 
    CONTAINER.WRITER['MyPublisher::MySimpleWriter']
local instance        = mySimpleWriter.instance

myComplexReader:take()
for  i, sample in ipairs(myComplexReader.samples) do
    if (not myComplexReader.infos[i].valid_data) then
        print("\t invalid data!")
    else
        local deviceId = sample['deviceId']
        local avgTmp = 0
        local sum = 0;
        for i=1, sample['temperatures#'] do
            sum = sum + sample['temperatures[' .. i .. '].value'];
        end
        avgTmp = sum /  sample['temperatures#']

        --setting the instance
        instance['deviceId'] = deviceId
        instance['avgTmp']   = (avgTmp - 32) * (5/9);
        instance['unit'] = 1 -- C

        -- writing the simple instance
        print("Writing sample with avgTmp = " .. instance['avgTmp'] )
        mySimpleWriter:write()
    end
end

-- Open File

As you can see, you have to write very little code to transform your complex data type into a simple one using RTI Prototyper.

In the first 3 lines we are just getting the complex reader and the simple writer. We are then assigning the pre-allocated instance of the simple writer to the variable called instance.

Next we are doing a take and, if the received samples are valid, we iterate over them, getting only the field we care about and aggregating the data. (In this example, we calculate the average of all the temperatures sent in the original sample, and we ignore humidity and pressure.) We can also do some more intelligent transformations, such as transforming the incoming temperature from Fahrenheit to Celsius (line 20)!

Once we have what we need, we assign the values to writer instance (lines 19-21) and we write the sample (line 25 ). It’s that simple!!!

For your convenience I uploaded all the code and files to a GitHub repository here.

In the repo you will find the following directories:

  • Transformer: Contains both the XML and the Lua file described in this blog post. To run it just execute:
rtiddsprototyper -cfgFile Transformer.xml -luaFile Transformer.lua
  • EmitterEmu: Contains an XML file and a Lua file to emulate the behavior of your ISS module Emitter. To run it just execute:
rtiddsprototyper -cfgFile EmitterEmu.xml -luaFile EmitterEmu.lua
  • FancyReceiver: Contains an XML file and a Lua file to emulate your fancy dashboard running on board the ISS. To run it, just execute:
rtiddsprototyper -cfgFile FancyReceiver.xml -luaFile FancyReceiver.lua
  • LimitedReceiver: Contains an XML file and a Lua file to emulate your limited subscriber running back on earth. To run it, just execute:
rtiddsprototyper -cfgFile LimitedReceiver.xml -luaFile LimitedReceiver.lua

So, clone the repo, play with the examples, read the few lines of code, and you will right away understand the power of RTI Prototyper with Lua. If you want more information on RTI Prototyper check out the Getting Started Guide or the other blog posts about it here.

And if you have some time left check out the other solution for doing scripting with RTI Connext DDS using Python and node.js here.