“See” what is going on with your DDS System Reply

RTI - GUEST POST - SIMVENTIONS

 

simventions

Matt Wilson is the Vice President for Tools and Technology at SimVentions. Mr. Wilson has been designing, building, and integrating complex system and related software based tools for over 20 years. His experience with Naval Combat Systems, Human Systems Integration, and Systems Engineering projects has been the knowledge base for the design and development of a wide variety of innovative and useful tools.

 

Have you ever had to figure out what is going on inside of your DDS-based system and had no idea how to begin? You have many data topics being published to and subscribed by a multitude of data readers and data writers across a vast array of processes. There are a img1wide variety of scenarios from mis-matched properties, out of date message definitions, missing connections, etc. How do you begin to “look at” this problem?

SimVentions has developed a tool called InformeDDS (Info for me – DDS) that uses network graphing technologies and DDS discovery data to “show you” what is going on in your network. InformeDDS’ ability to auto-discover DDS entities (participant, publisher/subscriber, data writer/reader, topic) and then automatically provide an interactive visual representation of connections between DDS entities and all relevant attributes allows you to quickly “see” what is happening without impacting your network.

img2 Using one of the preset (or personally customized) display filters helps reduce complicated networks to a meaningful subset to allow you to focus on key issues. Each of these filters can be built, copied, tailored, and improved through a simple user interface to highlight attributes unique to your own system. We call these filters “View Profiles” which provide you the ability to set your own visualization business rules or use the ones that come out of the box.

InformeDDS has proven to be a useful tool for software developers during testing and debugging and for performing integration testing on complex systems. It has helped us (SimVentions) debug and troubleshoot networking issues for our own customers and can be quickly learned to bring the same value to your integration and testing efforts. During a recent visit to one of our large systems integration partners, an on-site engineer commented “Now THAT is an integration tool” after using InformeDDS.img3

SimVentions is proud to be an RTI partner. To try out InformeDDS with the latest version of RTI Connext DDS (5.2.0) please use the RTI Launcher to download and install the tool.

We continue to improve the InformeDDS technology and look forward to hearing your feedback about this capability. There are many other features in the tool that we will introduce to you in future stories. Until then, download and try the free trial of this innovative and useful debug and integration tool for your DDS network today.

For more information, visit the following sites or leave a note in the comments section of this post!

Rapid Data Transformations Are Moments Away! 4

When I started college, everybody was talking about “Information Technology.” At that point I had been programming for quite a while and it wasn’t clear to me what coding had to do with that fancy terminology. After a few more years of coding, I realized the connection: all I do, day in and day out, is move bytes (information) from one memory location to another. Copying the contents of a struct into the socket buffer and sending it out; getting the bytes from the socket buffer and deserializing them into a structure to pass them to the application logic. Well, that’s part of what communication middleware does for you!

RTI Connext DDS implements the OMG DDS Standard. DDS is data-centric middleware. We call those bytes being transferred data, and we assign a type to them. The type describes each byte in your data, so you (and your application) can make good use of them.

Designing a good distributed system means having a good type system in place. Often though, those types carry a lot of information, and it can become pretty difficult to deal with them. Also, you may not need all the information all the time. This often happens when you have heterogeneous systems with nodes that have different capabilities, such as bandwidth constraints or resource limits. In this case, some of the nodes in the system may not be able to handle a whole sample for certain data types, but they still need to be able to receive part of the information.

Let me present a possible scenario first and then suggest one way to solve it.

scenario

Figure 1: Scenario

Let’s say a new testing module for the International Space Station (ISS) has been shipped to space. The module has a device that collects thousands of data points, puts them all in a DDS sample with a specific type, and writes it on the topic, ComplexDataTopic. We will call this device the Emitter.

One of the many different data points this device collects is the temperature from 10 different sensors. It puts all the collected temperature values into a sequence.

enum Unit {
   F,
   C,
   K
};

struct Temp {
    long sensorId;
    double value;
    Unit unit;
}

struct Measurements {
  long deviceId;
  // thousand of more fields. 
  sequence<Temp,10> temperatures;
};

// Open file

The module also comes with a FancyReceiver (what we call a topic subscriber) that runs a UI; it gets all the data contained in each sample and creates statistics and charts so astronauts can easily understand the data. For example, looking that the temperature data, it could calculate the average (mean) or the median, or spot significant differences between sensors.

Let’s now say that back on Earth, NASA scientists need to know what the temperature is on the testing module. They’re not interested in pressure and humidity. Just the temperature. But, since the bandwidth between Earth and the ISS “is what it is”, it’s considered acceptable to receive aggregates of the date. For example, instead of sending all the values for the temperature the testing module will only send the average, which will save the ISS some bandwidth on the link to ground. Basically, they want something that looks like this:

enum Unit {
   F,
   C,
   K
};

struct SimpleMeasurements {
    long deviceId;
    double avgTmp;
    Unit unit;
}

// Open File

We have many options to achieve this result:

  1. We can create an ad-hoc application: it will subscribe to the ComplexDataTopic, get the data we need, create another topic, another type, calculate what we need, and send it over.
  2.  We can use RTI Routing Service: write a custom transformation library and be done with it.
  3.  We can use RTI Connector via the RTI Prototyper with Lua.
  4. We can use RTI Connector via Python or node.js.

I will now explain option 3: Use RTI Prototyper with Lua to easily transform your types. If you are already an RTI Connext user, you will have what you need in your distribution (under the “bin” directory if you are using RTI Connext 5.2, or under the “scripts” directory for older versions). Otherwise, check out this page to learn how to get your free copy of RTI Prototyper.

We will create a new component, the Transformer. The Transformer will subscribe to ComplexDataTopic (with type Measurements) and will Publish to SimpleDataTopic (with type SimpleMeasurements).

The Types

For simplicity, let’s say that the complex type is the following:

<enum name="Unit">
  <enumerator name="F"/>
  <enumerator name="C"/>
  <enumerator name="K"/>
</enum> 
<struct name= "Temp">
  <member name="sensorId" id="0" type="long"/>
  <member name="value" id="1" type="double"/>
  <member name="unit" id="2" type="nonBasic"  
        nonBasicTypeName= "Unit"/>
</struct>
<struct name= "Measurements">
  <member name="deviceId" id="0" type="long"/>
  <member name="humidity" id="1" type="double"/>
  <member name="pressure" id="2" type="double"/>
  <member name="temperatures" sequenceMaxLength="10" id="3" 
        type="nonBasic"  nonBasicTypeName= "Temp"/>
</struct>

<!--Open File -->

The simple type looks like this:

<enum name="Unit">
  <enumerator name="F"/>
  <enumerator name="C"/>
  <enumerator name="K"/>
</enum> 
<struct name= "SimpleMeasurements">
  <member name="deviceId" id="0" type="long"/>
  <member name="avgTmp" id="1" type="double"/>
  <member name="unit" id="2" type="nonBasic"  
        nonBasicTypeName= "Unit"/>
</struct>

<!-- Open File -->

The Topics and the Entities

All we need to do now is define an XML application file with a data reader for topic ComplexDataTopic and a writer to SimpleDataTopic:

<!-- Domain Library -->
    <domain_library name="MyDomainLibrary">
        <domain name="MyDomain" domain_id="0">
            <register_type name="Measurement" kind="dynamicData" 
                         type_ref="Measurement"/>
            <register_type name="SimpleMeasurement" kind="dynamicData" 
                           type_ref="SimpleMeasurement"/>
            <topic name="ComplexDataTopic"  
                          register_type_ref="Measurement"/>
            <topic name="SimpleDataTopic"   
                          register_type_ref="SimpleMeasurement"/>
        </domain>
    </domain_library>
<domain_participant name="Transform" 
                     domain_ref="MyDomainLibrary::MyDomain">
            <participant_qos base_name="QosLibrary::DefaultProfile"/>
                <publisher name="MyPublisher">
                    <data_writer name="MySimpleWriter" 
                                       topic_ref="SimpleDataTopic" />
                </publisher>
                <subscriber name="MySubscriber">
                    <data_reader name="MyComplexReader" 
                                       topic_ref="ComplexDataTopic" />
                </subscriber>
        </domain_participant>

<!-- Complete File -->

The Actual Logic

Once you have defined what your types are and how your entities are called, you just have to write a simple chunk of Lua code that will be executed by RTI Prototyper. Let’s have a look:

local myComplexReader = 
    CONTAINER.READER['MySubscriber::MyComplexReader']
local mySimpleWriter  = 
    CONTAINER.WRITER['MyPublisher::MySimpleWriter']
local instance        = mySimpleWriter.instance

myComplexReader:take()
for  i, sample in ipairs(myComplexReader.samples) do
    if (not myComplexReader.infos[i].valid_data) then
        print("\t invalid data!")
    else
        local deviceId = sample['deviceId']
        local avgTmp = 0
        local sum = 0;
        for i=1, sample['temperatures#'] do
            sum = sum + sample['temperatures[' .. i .. '].value'];
        end
        avgTmp = sum /  sample['temperatures#']

        --setting the instance
        instance['deviceId'] = deviceId
        instance['avgTmp']   = (avgTmp - 32) * (5/9);
        instance['unit'] = 1 -- C

        -- writing the simple instance
        print("Writing sample with avgTmp = " .. instance['avgTmp'] )
        mySimpleWriter:write()
    end
end

-- Open File

As you can see, you have to write very little code to transform your complex data type into a simple one using RTI Prototyper.

In the first 3 lines we are just getting the complex reader and the simple writer. We are then assigning the pre-allocated instance of the simple writer to the variable called instance.

Next we are doing a take and, if the received samples are valid, we iterate over them, getting only the field we care about and aggregating the data. (In this example, we calculate the average of all the temperatures sent in the original sample, and we ignore humidity and pressure.) We can also do some more intelligent transformations, such as transforming the incoming temperature from Fahrenheit to Celsius (line 20)!

Once we have what we need, we assign the values to writer instance (lines 19-21) and we write the sample (line 25 ). It’s that simple!!!

For your convenience I uploaded all the code and files to a GitHub repository here.

In the repo you will find the following directories:

  • Transformer: Contains both the XML and the Lua file described in this blog post. To run it just execute:
rtiddsprototyper -cfgFile Transformer.xml -luaFile Transformer.lua
  • EmitterEmu: Contains an XML file and a Lua file to emulate the behavior of your ISS module Emitter. To run it just execute:
rtiddsprototyper -cfgFile EmitterEmu.xml -luaFile EmitterEmu.lua
  • FancyReceiver: Contains an XML file and a Lua file to emulate your fancy dashboard running on board the ISS. To run it, just execute:
rtiddsprototyper -cfgFile FancyReceiver.xml -luaFile FancyReceiver.lua
  • LimitedReceiver: Contains an XML file and a Lua file to emulate your limited subscriber running back on earth. To run it, just execute:
rtiddsprototyper -cfgFile LimitedReceiver.xml -luaFile LimitedReceiver.lua

So, clone the repo, play with the examples, read the few lines of code, and you will right away understand the power of RTI Prototyper with Lua. If you want more information on RTI Prototyper check out the Getting Started Guide or the other blog posts about it here.

And if you have some time left check out the other solution for doing scripting with RTI Connext DDS using Python and node.js here.

RTI Services Delivery Partner (SDP) Program Reply

Partnership

Core values are critical. They shape how and why we do what we do, both personally and professionally. Here at RTI, we do our best to make decisions that are informed by our values, which ensures our strategic goals are consistent with our mission – to enable and realize the potential of smart machines to serve mankind.

Our Services Delivery Partner (SDP) program is no exception. It’s the realization of one of our core values: valuing the importance of working as a team. As with any organization, the value of teamwork is realized internally as well as externally. With SDP, the value of teamwork is embodied in how we work with our partners, who are critical to continued growth. That’s one of the many reasons I’m excited about the recent addition of Tech Mahindra and Remedy IT to our rapidly expanding SDP program.

The Value of Partnerships

I find Wikipedia’s definition of partnerships quite helpful in light of our SDP program vision. Wikipedia defines a partnership as “an arrangement where parties agree to cooperate to advance their mutual interests…to increase the likelihood of each achieving their mission and to amplify their reach.” By leveraging the power of two brands and our market-leading connectivity platform for the IIoT, our SDPs can offer their customers a broader variety of product and service choices for a truly best-in-class IIoT solution. The SDP program gives partners the ability to design, plan, implement, and support RTI-enabled solutions in their customers’ environments.

Needless to say, we see these partnerships as critical to healthy growth, and that is precisely why we’re actively growing our SDP program and driving our strategic partnerships to the next level.

Advancing Our Mutual Interests, Amplifying Our Reach

Our SDP program strategy is specifically designed to focus on:

  • Establishing a growth multiplier
  • Amplifying our reach

Establishing a Growth Multiplier

Companies are continually looking for effective ways to strengthen their brand and expand awareness. We’re no different at RTI, and plan to leverage the power of our partnerships to increase our market presence and technology adoption rate. In fact, right after our joint press release was issued announcing our new relationship with Tech Mahindra, we saw immediate coverage:

Aligning with our SDPs will create a catalyst for growth that allows us to benefit from our partners’ solutions-focused global presence, significant vertical expertise, and worldwide sales teams. We see this as a perfect complement to RTI’s connectivity platform for the IIoT. These partnerships enable us to be even more aggressive in driving adoption of our technology in more and more vertical solutions. By combining efforts, we can deliver solutions and services for the emerging and dynamic requirements of the IIoT market in key verticals around the world, including energy, automotive, avionics, healthcare, and telecommunications.

Amplifying Our Reach

In addition to expanding the adoption of our connectivity platform, we’re also looking to amplify our services reach. Today, RTI’s professional services organization delivers high-value expertise, but is inelastic given our current composition. We plan to leverage the size and vertical expertise of our partners to offer our existing and emerging customers additional value, and establish a more elastic delivery capability, which is vital to RTI’s continued growth.

The Way Forward

As you can probably tell, we’re quite excited about our SDP program, and we look forward to providing additional updates as the program grows. If you have any questions or you’re interested in becoming a Services Delivery Partner, don’t hesitate to contact us at partners@rti.com.