Internet of Things: Collect Device Data with MQTT & Use Device Data with DDS Reply

As David Barnett points out in his recent blog, the Internet of Things needs many protocols.  Two of them, MQTT and DDS, are rapidly becoming the best ways to connect large numbers of devices.  The confusion is understandable; the high-level positioning is similar.  Both can connect thousands of devices into real-time machine to machine (M2M) networks.

However, they target very different needs.

MQTT targets device data collection.  Its name, the Message Queue Telemetry Transport, points this out; the main purpose is telemetry or remote monitoring (see  The goal of MQTT is to collect data from many devices and transport that data to the IT infrastructure.   It targets large networks of small devices that need to be monitored or controlled from the cloud.  There is little attempt to enable device-to-device transfer, nor to fan out the data to many recipients.  Since it has a clear, compelling single application, MQTT is simple, offering few control options.  It also doesn’t need to be particularly fast; in this context, real-time is typically measured in seconds.

A hub-and-spoke architecture is natural for MQTT.  All the devices connect to a data concentrator server, such as the new IBM MessageSight appliance.  You don’t want to lose data, so the protocol works on top of TCP, which provides a simple, reliable stream.  Since the data is used by the IT infrastructure, the entire system is designed to easily transport data into enterprise technologies like ActiveMQ and ESBs.

DDS, on the other hand, targets device data use.  Its name, the Data Distribution Service, also points out its goal: to distribute data to other devices.  While interfacing with the IT infrastructure is supported, DDS exists to connect devices to other devices.

Devices demand data very differently than the IT infrastructure demands data.  First, devices are fast; real-time is often measured in microseconds.   Devices need to communicate with many other devices in complex ways, so the simple reliable point-to-point streams of TCP are far too restrictive.  Instead, DDS offers detailed quality of service control, multicast, configurable reliability, and pervasive redundancy.  Fan out is a key factor; DDS offers powerful ways to filter and select exactly which data goes where — which can be thousands of simultaneous destinations.  Some devices are small, so there are lightweight versions of DDS that run in constrained environments.  But there are also powerful full-featured versions that support large participants in the device network.

Hub-and-spoke is completely inappropriate for device data use.  Rather, DDS implements direct device-to-device bus communication with a relational data model.  RTI calls this a databus, because it is the networking analog to a database.  Similar to the way a database controls access to stored data, a databus controls data access and updates by many simultaneous users.  This is just what many high-performance devices need to work together as a single system.

Bottom line: Device data use is a fundamentally different use case from device data collection.  Data distribution (DDS) enables devices to use other devices’ data to work together as a coherent system.  Data telemetry (MQTT) enables IT infrastructure to monitor and collect data from many devices, thus connecting them to the cloud.  There is some overlap: DDS can serve and receive data from the cloud, and MQTT can send information back out to devices.  But the fundamental goals differ, the architectures differ, and the capabilities differ.  Both protocols are critical to the (rapid) evolution of the Internet of Things.

Practical Freedom and Open Business – As applied to DDS and Open Source 1

The open source movement, and notably the Free Software Foundation, rightly positions its value proposition as really about freedom.  Their lofty ideals, such as guaranteed access to source, unrestricted sharing, and open collaboration, drive the success of many key works.  “Free speech” software is a beautiful concept.

Unfortunately, these lofty ideals are often not enough in practice.  Open source software (OSS) famously struggles to find successful business models.  Without financial support, many projects languish.  Users, especially users of emerging core infrastructure software, need more.  They need the practical freedoms and open policies that ensure a healthy supplier relationship and mutual success.  These freedoms are no less profound.

What works for the mass, mature, server market may not apply to emerging niche applications.  That really matters to you, if one of those emerging niches is the core underpinnings of your $1b project. RTI has decades of experience in complex networking software for mission-critical projects.  By listening deeply to hundreds of teams, we have learned which freedoms are truly critical.  We recently made the most important announcement in our history, our new “infrastructure community model”.  That model implements both the practical freedoms and open business policies that our customers need.

Maybe it comes down to a platitude: Freedom isn’t free.  It also isn’t one thing.  It’s time we take a look at the practice of freedom and open business…

Source Freedom

Source code for a complex system is useful for understanding, for debugging, and to expose potential bugs to many “eyes”.  Contributions in the form of application, services, tools and other layered products are very valuable.  Likewise, using common architectures and platforms across an organization or a supply chain dramatically lowers software costs.  So, the freedom to share modules and resulting platforms is critical.

“Free beer” products, which do not carry any costs, are also critical to seed early exploration.  RTI now offers free beer, and it’s fresh, it’s the market-leading best brew, and we offer it with professional service (support).  You can modify it, share it with your cohorts, and ship it in your products.  There is no charge.  If you’re opening a brewery, it should be compelling.  Bottoms up!

How can we afford that?  Because free beer is best only for smaller, short term projects.  If your system is complex, long lived, and mission critical, there is danger in distancing yourself from the source of the infrastructure.  If I’m betting my career and $1b project on infrastructure software, I damn well want my vendor to know who I am, care about my needs, and aggressively maintain the software to ensure my success.  I do not want to hack that software myself, thereby cutting me off from effective support and future updates; “escrow” is a desperate last resort.  And, I’m willing to pay a fair price to ensure the software’s, and the vendor’s, continued excellence.  In short, I expect my vendor to see my success as a responsibility, and one that is shouldered in exchange for reasonable consideration.

Risk Freedom

These technical aspects are critical.  But the risks of system development go well beyond source and sharing.

The first consideration is freedom from legal strings.  The tar pit of copyleft is well known.  But, some code bases contain hundreds of different copyrights and license terms; it’s hard to know if the vendor has the right to grant you a license at all.  This problem, called code provenance, is a nasty trap waiting to spring on the unsuspecting user.  It’s the reason many open source vendors will not offer IP indemnification for their products…they make you take the risk.

Unintended patent licenses are an even more subtle problem.  The GPL and LGPL, for instance, contain well-meaning patent license clauses.  Unfortunately, these threaten unintentional licensing of your patents if you distribute the code.  The problem is sufficient to cause many organizations (ironically) to preclude use of LGPL code.

Of course, a vendor’s business model is more than a license.  Consider freedom from frustration.  Pulling your hair out late at night preparing for your system acceptance test is no time to consider support and guidance.   Support for current and old versions, both free and paid, is crucial.  That support should be available during extended hours at known and bounded cost.  Consider also the quality and availability of deep guidance.  Our experience with hundreds and hundreds of project teams means that we’ve made and learned from mistakes that you don’t want to repeat.

Speaking of mistakes, consider freedom from being stranded.  Fresh from yet another colonoscopy from a Fortune 50 vendor qualification, I can assure you that infrastructure customers expect crisp and complete quality processes, deep testing, financial health, and good corporate governance.  Some OSS companies offer this…others roam the wild west.

Freedom from stranding does not just mean that your vendor survives.  The vendor must proactively strive to meet your needs.  Great software requires ongoing investment in features, usability, quality, platforms, and standards.  The business model must reward more than just hours.  It must also allow and encourage you to leverage the latest technology, even if parts of your system are using free or old versions.  These are not trivial.  But without them, you risk long-term lack of the features, tools and services that you need to succeed.

Relationship Freedom

Perhaps the most important freedom is freedom from surprise.  RTI commits to a high bar: absolute honesty; it’s written into our founding core values.  This does not mean we will tell you all our plans.  It does mean that we commit to you, to our employees, and even to our competitors that we will never intend to deceive.  More to the point, if there’s something you need to know to deal fairly with us, we will not hide that information.  We strive for no surprises.

We are very open with our license; it’s posted on our website.  Our “open community source” concept is not “genuine” open source, and we do not claim that it is.  However, it strives for the elegance and freedom seen in the best of the OSS licenses, albeit within an IC.  The IC concept allows us to offer you a great deal: almost all projects qualify to be in an IC, within the IC we are very open, we ship the latest version of core library free or paid, and we support everything we ship. We genuinely believe this model is in your best interest.

You cannot divorce the license model from the pricing model.  Few pure-play OSS users expect to be someday faced with the Hobson’s choice of living with a critical flaw in their infrastructure or paying a $250k bill.  Few users who start with free-but-limited versions expect to someday have to upgrade to a pricing model they have never considered.  These primrose paths are, unfortunately, significant sources of revenue for many companies.

RTI matches straightforward licensing with straightforward pricing.  Like our software, our source of revenue is completely open, predictable, and known.  Your cost depends on only one metric (developers).  When you purchase all you need from our broad product line, you will pay only a few $k each.  We request only predictable, fair consideration for extraordinary value (another core value).

Our customers count on us for weighty things:  The core of multi-$b product lines.  The success of critical programs.  The survival of their compatriots.  Infrastructure middleware is a responsibility, and one that we at RTI take very seriously.  Our product quality, performance, and reliability are covenants with our customers.

We also consider both our license and our pricing as fundamental covenants with our customers.  We are fully dedicated to an open, honest, successful relationship.  There will be no surprises.  We will walk that extra mile, with you, to ensure success.

That is also freedom; maybe it should be called freedom from loneliness.  Because, in the end, selecting and using infrastructure software is a long journey.  It’s a trip you don’t want to take alone.

You are likely betting your career and project on infrastructure software.  We strive to be worthy of that trust.

Learn more about OSS business models and open community source at my upcoming webinar entitled “Open Source 2.0?”; register here:  It’s free.  🙂

Introducing RTI Connext and The Next Era in Operational Technology Reply

Today is one of the most significant days in RTI’s history. Today, we are unveiling a new product line, a new corporate strategy, and our vision of a new industry direction.

Our new products target the challenging problem of building connected systems that will work together as never before.  The RTI Connext product line spans the gap from deeply-embedded devices, through high-performance real-time systems, all the way to enterprise IT integration.  It really is the first edge-to-enterprise “real-time SOA” platform.  We’ve been working on the base technology for decades; it’s truly exciting to bring the vision together—and the capability to market—for the first time.

Our new strategy is to leverage our “operational” technology into a unified system-wide infrastructure.  Our software is the key nervous system for over 500 major mission-critical projects in several industries including defense, industrial automation, automotive, and healthcare.  Our new products expand upon this footprint, both down into smaller devices, and up into system-of-systems integration.  We can now assemble large networks of mixed devices and applications into working systems, and hook those (and other) systems together into a connected whole.

Critically, we can also now connect operational systems to the enterprise IT infrastructure. Why is that valuable?  Because it enables a new way to run and control systems.  This is an important trend across many industries.  As networks become more powerful, and as technologies like ours make interconnection more possible, the worlds of IT and OT (operational technology) can, for the first time, interact. RTI Connext builds operational systems, but with our new Integrator product, it can also connect those systems with enterprise technologies in real time.  Since business intelligence and resource management are so well developed for the enterprise, this makes it possible to feed back smart directions to the operational system.  And that makes everything more efficient.

Thus, we let you use, control, and optimize all your systems as a holistic entity.  Our new tag line sums it up nicely: Your Systems. Working as One.

This is a big, complex vision.  Implementing it fully will take years.  But today’s launch represents a culmination of what we’ve learned addressing the needs of mission-critical applications for so many years. Our base technology has the performance, 24×7 reliability, and scale to succeed as the core transport for demanding real-time systems.  We are now adding IT-like technologies: central console administration, multiple messaging patterns, and a true service bus.  The result is a unique approach that enables a new level of system function.  We’re excited both by the current breakthrough and its implications for the future.

It’s great to kick off 2012 with such big news! Today’s announcement is the first step towards our goal of making all your systems work together as one. I’d like to offer my congratulations to the RTI family, as well as the customers and partners who made this day possible.