The Data-Centric Modus Operandi: Part 2 6

An earlier post of mine, The Data-Centric Modus Operandi, has garnered a couple of good comments recently. I was in the process of responding to one of them when it occurred to me that (1) a response longer than the WordPress comment field probably didn’t belong there and that (2) more readers might have the same question, and an answer in the comments wouldn’t be very visible. So here it is, my feature-length response:

Frans Ott says:

Good piece of work Rick! Does this mean that practising the DDS-perspective, I will respect the business domain model more as it is? And that, as a consequence, less domain-burden will be drawn into the middleware?

This question is an important and perceptive one. My answer has a couple of parts.

First, DDS is not a panacea. Adopting DDS but using it in the same way you used your old messaging middleware can bring you incremental benefits in terms of performance, scalability, and flexibility. But it cannot by itself provide you transformative benefits in terms of time-to-market and long-term maintainability.

Those larger benefits accrue as you adopt data-centric architecture — that is, as you begin to model your business domain and tie your distributed applications to that model instead of to each other. You can start to do this with any middleware, not just with DDS, and doing so will allow you to more smoothly integrate capabilities over time without perturbing existing behavior. But when you use a technology like DDS, which unlike other middlewares provides functionality to directly support this type of architecture, you get an additional benefit: the middleware can start to automate things that you used to have to do yourself.

There’s my second point: when you tell your infrastructure what you’re trying to do, it can help you do it better. I used the word “model,” which might sound frightening and esoteric. But everyone does this anyway! They just don’t call it that. When people design a system in a message-centric way, they write long documents (aka “models”) describing who has to send messages to whom, in what format, at what times. Then they email these documents to each other and implement the various components based on what the documents say. In the process, they write a lot of code to encode and decode data, validate it, monitor and respond to the system’s status, manage application state, and so on. And once they’ve done all of this work, they plug their components in and find out whether the documents had any ambiguities in them and whether everyone has correctly implemented all of those infrastructure pieces I enumerated.

The truth is, from the point of view of the real business logic, “the middleware” isn’t just a messaging or data-distribution component purchased off the shelf. It also includes all of the stuff your in-house infrastructure developers had to build on top of that to make the applications work right. A key innovation in DDS is the answer to this question: what if we took a lot of the ad hoc content out of all of those word-processing design documents and replaced it with a formal structure that the off-the-shelf middleware could understand? The answer is that the off-the-shelf component can start to do a lot of what the in-house “middleware-above-the-middleware” used to have to do — based on configuration rather than on custom code. The result is a more portable, interoperable system of higher quality and performance that was developed and integrated in less time. In other words: faster, cheaper, better.

You can use DDS as just a fast, interoperable way to send messages from here to there. Lots of people do. But DDS can do more than that if you’re willing to design differently. DDS does for data in motion something like what an RDBMS does for data at rest. It can’t make you create a good data design. But if you teach it what your design is, it can help you enforce that design, and it can reduce the amount of code you have to write in between your “middle”ware and your business logic.

6 comments

  1. “based on configuration rather than on custom code”.

    OMG! I can’t believe you said that. Using an RTI IRAD license, I recently built and tested a simple adapter between our app layer and your product – only exposing the QoS features that we needed. In parallel, a colleague built a “free” homegrown contraption directly on top of (blech!) sockets using Boost.asio to lessen the effort. Early on, I encountered a mysterious “lost messages” problem at high topic rates that went undetected by both the DDS publisher and subscriber. After calling your support line, I solved the problem by simply changing the default history QoS “configuration” setting to “keep all history” on the publisher side so that fast writers wouldn’t overwhelm the transport layer – and the problem disappeared. Weeks and $$$ later, we’re still trying to solve the equivalent problem in our homebrew with “custom code” diddling socket buffer sizes in the bowels of the system. It makes me want to cry and wring some necks.

    Old school people just don’t get it. It reminds me of the olden days where developers would write their own Rube Goldberg RTOS rather than shelling out some dough for a solid commercial product written by experts in the domain. Boo Hoo.

    Like

  2. The role of middleware is mainly solving the shared data requirement from the large application vendor’s point of view. Modelling ‘their world’ with a more prominent role for middleware will lead, and I’ll put this nicely, to some rimples in the pond.

    Giving away a piece of the action will be felt in someone’s pocket, not? Some major application vendors that started their own middleware solution are, in my observations, a bit struggling in this new field of expertise.
    Catching-up with new technology developments, attrackting new customers, etc are a push for middleware, while supporting the existing customerbase is, of course, got their first priority.

    DDS seems to push the middleware in a stronger and more prominent midfield place.
    I think, the rise of DDS will increase the pressure on the business and enterprise modellers, in order deliver the right design to a promissing model. And did we do a good modelling job so far? Did the ‘invisible middleman’ gained the trust by the desisionmakers?
    Customers nowadays tend to choose for standards and out-of-the-box software. Still, modelling the customer’s domain is of major importance. Because, that’s also how we still keep up to new requirements and non-functional necessities.

    After reading serveral DDS documents I think I got an understanding of where it’s aiming for.
    But in order make DDS mainstream, we have to overcome some conflict’s of interest first. I think the key for succes lies in delivering the right DDS design; it’s this new deliverable we need to present to our managers in order to tempt them for going DDS al-the-way, don’t you agree?

    Like

  3. Frans,
    You are right to take the conversation back in the business direction. The right approach for “selling” DDS to your managers/stakeholders depends on their level of comfort with the technology and on your relationship with them and with your infrastructure provider(s). In the end, it all comes down to risk management: which methodologies and tools are going to give the highest chance for project success at an acceptable cost? You or your customers might be interested in this study by Embedded Market Forecasters, which found that projects that took advantage of off-the-shelf middleware were more likely to meet their design goals and at lower cost: http://www.rti.com/mk/commercial-middleware-vs-roll-your-own.html.

    For new systems and significant updates to existing systems, RTI frequently recommends a data-centric architectural approach as the lowest-risk path to success over the project lifecycle — in particular because of the “shared data requirement” you mention. The lifecycle includes developing and deploying an initial set of capabilities as well as the integration of new capabilities over time. The approach should be supported by appropriate standards-based technologies, DDS among them. I think this is what you’re getting at when you speak of “the right DDS design … for going DDS all the way.”

    Other project and/or other managers will require a more incremental approach. Teams with expertise in a message-centric approach can use DDS in that way and quickly observe benefits in terms of CPU and network utilization, one-to-many scalability, and fault tolerance. They don’t need to go “all the way” in terms of refactoring their system design; they can see incremental gains based on their existing design. Case in point: one RTI customer recently ported an existing system from web-services-based messaging to DDS-based messaging and saw a 10x increase in data-processing capacity with 5x _fewer_ machines. This obviously represents a significant cost savings as well as a significant functional improvement.

    RTI can help customers with DDS adoption, whether large or small, in several ways:

    * Our Routing Service component (http://www.rti.com/products/dds/routing-service.html) can help you integrate DDS with other communication technologies you may have, such as legacy socket-based protocols or other middlewares. You can adopt DDS on a subsystem-by-subsystem basis; you don’t need a “big bang.”

    * Our Message Service component (http://www.rti.com/products/jms/index.html) adapts DDS with the JMS API: engineering teams as well as software components that are JMS-ready can also be DDS-ready is short order.

    * Our Professional Services organization (http://www.rti.com/services/) is available to help with architecture studies, design support, and even technology implementation.

    I hope that helps put things in perspective. DDS has the potential to be transformative, but people considering adopting it shouldn’t fear that they _have to_ transform in order to take advantage of it. Adoption can be quite straightforward; just download and see for yourself: http://www.rti.com/downloads/dds.html.

    Like

  4. Pingback: New Video: DDS in a Nutshell « Blogs from RTI

Submit a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s