Derived and Revised Messages
Gerry Tyra - April 2015
A messaging system does not exist in a vacuum, it was created in response to the needs of an environment. As that environment changes, the messaging system must change as well. But change is more than just adding new messages. Change also includes a need to extend and modify existing messages to better represent the content that the producers and consumers of that data have available and need.
Change happens. As technology improves, or new requirements emerge, the operational characteristics of any system will change. Whether sub-systems are modified or replaced doesn't matter, the change happens. And the change may not be to greater complexity. A convergence of factors may create a new smaller, lighter system that is functionally more capable, at the expense of some heavy elements. Consider using a quad-copter in place of a Global Hawk. The quad-copter has only a small fraction of the payload capability, but it can be launched and controlled locally, when and where it is needed.
All of these changes result in changes to the data that can, or should, be transferred within the system. In order to maintain efficient communications, the messaging system need to evolve to represent the data.
But very few systems exist by themselves. Rather, they exist within a larger environment where the cost of inventing and integrating a new messaging protocol is prohibitive. In these cases, reuse of an existing system, evolving messages only where needed, is a rational approach.
The Need for Coherency:
When talking of changing or growing messaging systems, one factor that must be remembered is data coherency. Just because all of the data that a given consumer (be it a process, thread or service) needs is scattered over N different messages, this may not be good enough. In a real time environment, various groups of data must be time correlated and time stamped relative to Now. During a final approach, knowing my vertical velocity as of 10 milli-seconds ago doesn't tell me much if my last altimeter reading was 5 seconds ago.
Keeping related data together, with a common aging on that data, makes the design and implementation of the consuming task simpler and more reliable.
Forms of Change:
The two easiest ways to change a messaging system are to add messages and tack on new data elements to the back of an existing message. Both of these have limitations.
Adding new messages requires that any other system that needs the data will also have to be updated. And, in respect for the limits of available bandwidth, there is usually a limited number of message identifiers (once, 32 bit IP addresses were adequate). Adding data elements to existing messages allows existing systems to correctly interpret the older portion of the message, but at the cost of growing message clutter and possible redundant data.
As it was shown in “Software Implementation to Reduce the Impact of Interface Revision” , another approach exist for revising messages. With a new revision, the older versions are deprecated and eventually dropped as other systems are updated to the newest revisions. This has the limitation that a revision implies that the previous version has be deprecated. But, what if the change was for a complementary system and not a replacement (e.g. changing a message to handle a color imager rather than an Infra-Red sensor). Both systems could coexist on the same platform, or on different platforms operated by the same unit, for an extended period of time.
The need for the coexistence of similar messages is the bases for the concept of a derived message. The approach is based in the C++ implementation of a base class, with derived classes used to specialize the functionality. So, a set of derived messages share a common core message, with extensions required to meet specific requirements. All systems would be able to interpret the base message, and there is nothing to preclude an application from providing interfaces to multiple derived messages.
Without the concept of a derived message, there are only two viable approaches to handling a set of similar messages, either create a single message that is the union of all the messages, or create a unique message for each case.
With derived messages, the set intersection of the specialized messages is represented by a common base class. As such, the majority of the implementation is done only once. A new derivation need only describe the difference between the base message and the additional required functionality. This simplify the design, implementation and maintenance of extensions.
This also opens the ability of a process having several receiving functions, each accepting either the base message type or specific derived messages. This allows an application to accept and use data that is appropriate, while rejecting messages that it can not use effectively.
The logical extension of this is that has elements of a sub-system group evolve, revisions to the derived messages can be created to keep the messaging system appropriate to its intended purpose.
The capability to provide revisions, intended to handle evolutionary change, and derivations, to handle variations within a generation of a given sub-systems, permits controlled tailoring of message sets to current real world requirements.