Skip to the main content.

Did you know?

 

RTI is the world’s largest DDS supplier and Connext is the most trusted software framework for critical systems.

Success-Plan-Services-DSSuccess-Plan Services

Our Professional Services and Customer Success teams bring extensive experience to train, problem-solve, mentor, and accelerate customer success.

Learn more

Developers

From downloads to Hello World, we've got you covered. Find all of the tutorials, documentation, peer conversations and inspiration you need to get started using Connext today.

Try the Connectivity Selection Tool ⇢

Resources

RTI provides a broad range of technical and high-level resources designed to assist in understanding industry applications, the RTI Connext product line and its underlying data-centric technology.

Company

RTI is the infrastructure software company for smart-world systems. The company’s RTI Connext product is the world's leading software framework for intelligent distributed systems.

Contact Us

News & Events
Cooperation

2 min read

Persisting data in a Real-Time distributed system

Persisting data in a Real-Time distributed system

Most of the distributed systems we deal with at RTI have performance constraints at their core. Either the system is pushing the limits of the available resources, or the action-reaction timing is critical for a given event. In other words the constraint might be on throughput or latency (or increasingly latency vs. throughput). In these kinds of systems persisting data is a real challenge. In many systems it is becoming a requirement that the distributed data is persisted. Take for example flight systems and automated trading systems, where persisting data is necessary to adhere to regulatory demands. At RTI we have set out to make persisting distributed data as minimally intrusive, performant and configurable as possible.

Recording Data

The simplest case of persisting data in a distributed system is recording the data to a file system. This can be done locally for each node, or centrally on a "recording" node. Each system setup solves different problems. Persisting local state on each node allows you to recover from a local crash by reading that state during recovery. Recording all the data on a centralized node allows you to query the entire set of data recorded (not just the subset any node would see) to find patterns or anomalies, or do a post-mortem on a system failure.

Locally each RTI Data Distribution Service publisher or subscriber keeps samples in queues. To allow for local persistence a mirror of these in-memory queues can be kept on disk. After a crash the publishers and subscribers can then be created with the same state as they had before the crash.

Centrally you can run RTI Recorder that can be configured to subscribe to interesting data (or all data for that matter) and write it to a file system. This data can then be queried either in real-time or during off-peak hours. As an aside RTI Data Distribution Service and RTI Recorder have countless flight hours on mission critical systems!

Connecting the Distributed System to an Enterprise Database

In some cases simply having the data on a file system is not enough — the full power of an enterprise database is needed. At RTI we have solved this by having RTI Real-Time Connect for ODBC enabled databases. Now you can use RTI Data Distribution Service to implement a distributed database, or utilize a database for your persistent storage needs. This allows you to use any tool or query you are familiar with to work with a distributed set of data. The possibilities are literally endless!

No matter what the use-case — recovery, logging, post-mortem analysis or database integration — persisting real-time distributed data is here to stay. The challenge is persisting the data with minimal intrusiveness on the real-time aspects of the system. At RTI we work hard to make that happen.

Learn More:

Autonomous Vehicle Production »

What is DDS? »

Connext DDS Pro »

Connext DDS Cert »

In the Age of Machines, We'll All be Programmers »