Integration Platform For JMPS Using DDS

Integration Platform for JMPS using a DataOriented ArchitectureOriented Architecture

Supreet OberoiVice President of Engineering

Real-Time Innovations, Inc.

About Joint Mission Planning System

The principle objectives of JMPS are to create a scalable framework for mission planning systems and promote collaborative inter-service mission planning. JMPS will be Defense Information Infrastructure. The Joint Mission Planning System (JMPS) program was designed to replace legacy planning systems and provide aircrews with well-structured automated flight planning tools for aircraft, weapons, and sensors. JMPS provides support for unit-level mission planning of all phases of military flight operations, with goals to support Air Force, Navy, Marine Corps, Army, and U.S. Special Operations Command fixed and rotary wing aircraft, weapons, and sensors, including precision guided munitions (PGMs), cruise missiles, and unmanned aerial vehicles. Pilots use the JMPS to develop route plans and weapons data load inputs prior to flying each mission. These files can be extensive and many require change prior to each flight. In addition, Force-level JMPS users attempt to optimize the individual aircraft plans into larger groups of assets. The pilots and other users require real-time file transfer with simultaneous joint file manipulation between geographically separated operational units, in order to support increasingly complex and time-critical mission planning processes. JMPS provides support for geographically separated pilots (mission planning users) operating at the individual Combat Unit level, with each user planning a single mission for a specific aircraft. These users are often under the command of different forces and Combatant or Expeditionary Commanders. JMPS comprises COTS, GOTS, and newly developed software. It uses software components as basic building blocks for the system, the JMPS framework providing the entire component management and infrastructure support needed to build such a component-based system. Mission application components can plug into the framework to create the functionality required for planning specific missions. The system interfaces with the Defense Information Infrastructure Common Operating Environment (DII COE), and is designed to handle system-high operations at the Top Secret level and above. Users collaborate on the route information by updating an XML-based “.JRT” file. The finished route is available as an information service in XML. Under the Framework 1.3/1.4 JMPS will provide additional mission-planning capabilities. In addition, JMPS will migrate to a service-oriented architecture and expand its use of Web services. This evolving technology will enable JMPS to interoperate with command-and-control systems, an important feature that supports more tactical mission planning. Framework 1.3/1.4 is particularly significant because the software forms the core underlying architecture on which software "plug-ins" that create complete mission-planning systems must conform.

JMPS promises to…

l Provide a framework to enable legacy systems and new COTS products to collaborate– Legacy systems include Portable Flight Planning Software

(PFPS), Air Force Mission Support Systems (AFMSS), Tactical Aircraft Mission Planning Systems (TAMS)

– COTS and GOTS products could include IM, Lotus Notes, – COTS and GOTS products could include IM, Lotus Notes, databases, messaging systems…

Goals of JMPS essential to the success of any Integration Platform

JMPS promised to reduce life-cycle costs, and provide a smooth migration from legacy systems such as Portable Flight Planning Software (PFPS), Air Force Mission Support System (AFMSS), and Tactical Aircraft Mission Planning System (TAMPS). The two key components of the JMPS are the collaboration technology and Mission Binders, which is the technology that allows end users to pull in all associated products into a single directory and share them. JMPS v1.4 mandates addition of SOA extensions and Web Services. JMPS v1.3 uses Universal Pluggable Component (UPC) model for mission planners to choose the modules that they want. … more on PFPS, AFMSS, TAMPS on the next slide

Goal 1: Leverage Existing Systems

l Utilize significant legacy investments:– Portable Flight Planning Software (PFPS)– Air Force Mission Support System (AFMSS)– Tactical Aircraft Mission Planning System (TAMPS)

l Harness COTS technology and messaging technologies:– Email, Chat– File Transfer, Phone…

The Portable Flight Planning Software, or PFPS is a suite of software for planning air missions built by TYBRIN Corporation. The suite includes the FalconView mapping system, Combat Flight Planning Software (CFPS), Combat Weapon Delivery Software (CWDS), Combat Air Drop Planning Software (CAPS) and several other software packages built by various software contractors. The Air Force Mission Support System (AFMSS) provides automated mission-planning support for Air Force aircraft (fixed and rotary wing) and guided munitions. AFMSS will operate either as a stand-alone system, or linked with other command information systems, to aid the mission planner in selecting an optimal route through hostile territory. Mission planning tools include: Flight, Route and Airdrop Planning Weapons Delivery Target Area Planning Radar Predictions Threat analysis Route Fly Through Perspective Views Cartridge/Data Transfer Devices (DTDs) Loading TAMPS is a computerized method of planning and optimizing mission routes against hostile targets. TAMPS is employed extensively by embarked Navy air wings and Marine Corps aviation units to provide planners a common automated system for rapidly processing large quantities of digitized terrain, threat and environmental data, aircraft, avionics, and weapon systems parameters. The system has an intended capability to meet the tactical mission planning and digital data upload requirements of fixed and rotary wing aircraft, standoff weapons, avionics systems mission support systems and unmanned air vehicles.

Goal 2: Share Relevant Data

l Route plans, saved as XML-based “.JRT” files can be changed prior to each flight.

l The changes made by COTS or legacy applications need to be disseminated. Challenges in supporting “joint file manipulation” include:file manipulation” include:– Broadcasting the entire file for even a small change– Tuning dissemination of changes based on nature of the

network links (connected/disconnected, LAN/wireless/radio)– Identifying set of parties affected by the change– Identifying parties classified to see the data– Supporting conflict management when multiple application

change the same data

Pilots use the JMPS to develop route plans and weapons data load inputs prior to flying each mission. These files can be extensive and many require change prior to each flight. Users collaborate on the route information by updating an XML-based “.JRT” file. The finished route is available as an information service in XML. This requirement addresses the need to support “joint file manipulation”. There are multiple challenges in sharing data in a context-relevant, automated manner across application technologies (email, IM, proprietary), network links (satellite, LAN, radio), organization topologies, and application needs. To address some of these concerns, JMPS and AFMSS use a file-based data structure, which integrates with application technologies such as Internet Messenger (IM), email, and web-services applications. To send the data relevant to the application, JMPS uses a file format standard which captures all the fields an application in the collaboration suite may use. This technique, while presented in an over-simplified manner, suffers from some significant architectural flaws: The file format, to preserve its mandate to capture all fields, is bulky and is not efficiently normalized. As a result, even for small (but semantically significant) change to a data structure, the entire file needs to be transported. Unlike a data structure in a middleware, a file structure is monolithic, making it very difficult to update and merge small changes to it in a distributed system. Transporting large files to share changes in the flight plan makes poor use of network links, especially if the network links involve high-latency or low bandwidth. Ideally, the middleware should determine: What attribute was changed? Who should be notified of the change? Send change to affected parties File-sharing does not address challenges in conflict management when multiple users update the same data structure. This challenge is typically addressed by distributed caching, for which a file-based implementation is not adequate. Given that different actors in the (distributed) collaboration system will need changes to the flight plan with different sense of urgency, we need a middleware that places a higher priority, and allows for special tuning, on consumers of data with more real-time needs. Emails, IM messages, or even events from traditional client-server systems like Lotus Notes do not address the needs of a real-time system with heterogeneous systems, network topologies, and organizational units.

Goal 3: Automate Integration

l Human-in-the-Loop design for integrating legacy and COTS applications is error-prone, inefficient– What is required is a data-centric design, where each

application gets data that they need, when then need, and howthey need in an automated manner

l Cannot force changing the application to automate integration– Not possible in many cases (e.g., COTS products)– Increases cost of QA, deployment and software maintenance

l Automate transformation of data between different applications

While using file is a cheap way to superficially integrate different systems such as IM and email, it is neither efficient nor reliable in disseminating the red-lines (changes) within that file, to different systems. So, each time the operator of the system gets a new version of the file, she needs to (conceptually) perform the following tasks: Parse document for updates Determine if the changes are relevant to her system Apply changes to the system Some of these tasks are manual, thus becoming both slow and error-prone. In an ideal collaboration framework, the system will seamlessly integrate changes between different systems in an automated manner by using newer technologies such as Enterprise Service Bus (ESB), Complex Event Processing (CEP) or other SOA-based applications.

Goal 4: Support Diverse Platforms

l Optimize use of network resources– Each application may need data with a different sense of

urgency– Data may need to be processed differently for sending over

different network links such as radio, wireless, LAN

l Transport Protocols– UDP, Radio, TCP, Shared Memory, Infiniband…

l Operating Systems & Programming Languages

Different applications within a collaboration system may need data with different urgency. For example, a pilot in-flight may need to receive updates to flight plans in a real-time, whereas a monitoring application intending to capture flight changes for post-mission analysis is not as sensitive to the urgency of data dissemination. Similarly, the collaboration framework should differently optimize sending data over radio links as compared to sending data over LAN networks. Many COTS collaboration suites, such as Lotus Notes, Yahoo IM, email, are inherently designed for reliable (TCP) LAN links, and do not perform optimally when the network characteristics change (need multicast on UDP, satellite link with > 350ms latency delay, …) In addition, if only a few attributes of a (large) data structure have been updated, there is no reason to publish the entire data structure. Heterogeneous platforms To enable a collaboration platform that can be used for disseminating complex data structure in real time over a variety of messaging platforms such as IM, email, video, voice, we cannot assume a lowest common denominator in terms of technology. There will be different transport protocols, operating systems with unique characteristics, and networks with very restrictive profiles (allow network communication only one way, and support lossy networks). The collaboration framework should enable collecting data across different domains, and not be excluded because of the unique attributes of the platform.

The Way Forward…

What is needed is a holistic approach that enables integration of all messaging tools, creates a unified picture from all information sources, and analyzes and combines all aspects on information in a manner that best utilizes the network links, and investments in existing collaboration applications. Recent technologies such as the Data-Distribution Service (DDS) and Complex Event provide the missing links. DDS enables the creation of a global data space where information can posted and accessed in real-time from any number of nodes. CEP provides a sophisticated way to process and analyze live information to detect events and patterns, and fire events. Combining all these technologies presents a unique opportunity to create a cohesive approach to building a collaboration framework that can merge information from many sources, correlate different events to infer new events of interest and disseminate this information to relevant users and applications.

Net-Centric Objective

Goal: Build fully integrated real-time net-centric systems.

Need: Seamless integration of many platforms, systems,and their data, overmany transportsusing a variety of technology.

RTI origins in the Stanford Robotics Lab led the U.S. Military to choose RTI’s NDDS software for their unmanned ground and aerial vehicles on the battlefield.

Net-centric Challenges

- Interoperabilitylimitations

- Inflexible implementations

- Performance &- Performance &Scalability Limitations

- Reliability, Availability, and Bandwidth

- Security concerns

- Cost

History: DDS the Standard

l Data Distribution Service for Real-Time Systems– Adopted in June 2003– Finalized in June 2004– Revised June 2005, June 2006– Specification of API for Data-Centric Publish-Subscribe in real-time

distributed systems.

l Interoperability wire protocol– Adopted in July 2006– Revised in July 2007

l Related specifications– UML Profile for DDS– DDS for Light-Weight CCM

l Multiple (7+) Implementations

BoeingFuture Combat

E2C HawkeyeBoeingAWAKS

DDS Adoption – Aerospace & Defense

NorthtropQinetiqSAAB

12

LockheedAEGIS Boeing/Insitu

Unmanned Air Vehicles

Future CombatSystems

RaytheonSSDS

Net-Centric Integration Problem

Net-Centric Implementation FrameworkPart 4: Node Design Guidance Figure 1. c

Air traffic control scenario

Airplane LAN

Alarms

Avionics

Real-Time

Real-Time GatewayGateway

Edge

Control Tower LAN

Airport LAN

Alarms

Arrival timeEnterprise Gateway

Real-Time

Enterprise

Backbone

Gateway

Enterprise

The Solution: DDS Global Data Space

l Data accessible to all interested applications:– Data distribution (publishers and subscribers): DDS– Data management (storage, retrieval, queries): SQL– ESB Integration, Business process integration: WSDL– Rich QoS, automatic discovery and configuration– Real-time and/or high-performance access to data

DBMS

DBMSDBMS

Global Data Space

DistributedNode

DistributedNode

DistributedNode

DistributedNode

DistributedNode

SQL SQL

DDS SQL

DDSWSDL

D T

Why Data-Centric Middleware?

5.0 Communications

2.0 Sensors

3.0 Fusion

4.0 BMC2

6.0 Sensor Control

1.0 Common Servicesl Hawkeye has functionally

oriented software modulesl Each module talks to many

other modulesRIP TRK MSI

WAC TDA

ESM SAFERDR IFF

SEN DSCL4 L16L11

DIA NAV IPCCMCP

MUX

FIL TDM

l Adding new functionality cascades integration re-work across many other

CEC

5.0 Communications

2.0 Sensors

3.0 Fusion

4.0 BMC2

6.0 Sensor Control

1.0 Common Services

RIP TRKCEC MSIWAC RAIDERTDA

DWC

CHAT

ESM SAFERDR IFF

SEN DSCD

istributed Data F

ramew

orkIPv6L4 L16L11

DIA NAV IPCCMCP

MUX

FIL TDM aADNS TIS

1.0 Common Services

8.0 Training7.0 Visualization

l Grouping the modules into functional clusters does nothing to change that reality and ease software integration

UNCLASSIFIED

HMI ACIS

across many other modules 8.0 Training7.0 Visualization

Distributed D

ata Fram

ework

HMI ACIS T4O

l Changing the communication between the modules can ease integration, when the new ‘Publish Subscribe’ approach is used – each module publishes its output w/o regard to who is receiving it, in contrast to the point-to-point approach of traditional inter-process communication

It’s about an architecture that can assimilate evolving functionality, rather than remaining set in time

Top reasons to use DDS

l Flexibility and Power of the data-centric model

l Performance & Scalability

l Rich set of built-in servicesl Rich set of built-in services

l Interoperability across platforms and Languages

l Provides/integrates Pub-Sub into SOA

#1 RTI Data-Centric Model

“Global Data Space” generalizes Subject-Based Addressing– Data objects addressed by DomainId, Topic and Key– Domains provide a level of isolation – Topic groups homogeneous subjects (same data-type & meaning) – Key is a generalization of subject

l Key can be any set of fields, not limited to a “x.y.z …” formatted string

Data WriterData Writer

Data WriterData Writer

Data ReaderData Reader

Data Reader

Data ReaderData Writer

Data Object

Key (subject)Topic

Topic: “Market Data”

FieldPayload

Topic: “Order Entry”

Subscriptions: By Topic, Subject, Content

SourceField

Value

Symbol Type ExchangePayload

* * * * *

Volume Bid Ask …

Company Confidential

Subject Filter (for a Reader)

Field

Value

Symbol Type ExchangePayload

* * NYSE *

Subject Filter (for a Reader)

SourceField

Value

Symbol Type Exchange Payload

REUTERS * EQ NYSE Volume > x, Ask < y

Payload Filter (for a Reader)

Topic: “Market Data”

Symbol OrderKind Stop LimitOrderNumber …

#2 Performance & Scalability

l DDS was designed to support high performance

l RTI DDS was developed to maximize performance and minimize jitter

l Advanced techniques employed:– Pre-allocation of memory

l Never allocate/free memory in the critical pathl Never allocate/free memory in the critical path– Use dedicated threads per receive port

l Minimize thread switchingl Avoid expensing operating system calls (e.g. select())

– Maximize concurrencyl Carefully design critical sectionsl Patented concurrent mutex-free thread-safe data structures

– Employ high-performance data-access APIsl Read data by array (no additional copies)l Scatter/gather APIs to access transport. l Buffer loaning for zero copy access

Study on impact of WS technologies for future European ATC: XML is not suitable for European Flight Data Distribution

l Data size explodes 10X vs. CDR– Flight Plans go from 100KB to 1MB

l Communication speed drops 20X

Not good for Radios!!

Source: Christian Esposito and Domenico Cotroneo, Dario Di Crescenzo.

SELEX-SI/Consorzio SESM/University of Naples.

“Flexible Communication Among DDS Publishers and Subscribers”

July 2008, Real-Time Systems Workshop, Washington, DC

#3 Powerful Services & Tools

– High-Availability– Persistent Data– Recording service– Relational Database bridge– Relational Database bridge– Development & Monitoring Tools

#4 Interoperability between platforms & languages

l Data accessible to all interested applications:– Data distribution (publishers and subscribers): DDS

– Data management (storage, retrieval, queries): SQL

– ESB Integration, Business process integration: WSDL

– Legacy Java Integration: JMS

DBMS

DBMSDBMS

Global Data Space

DistributedNode

DistributedNode

DistributedNode

DistributedNode

DistributedNode

SQL JMS

DDS SQL

DDSWSDL

D T

DDS: Multi- Architecture Support

Solaris

RTI DDS

Windows

RTI DDS

Linux

RTI DDS

RTOS

RTI DDS

• Same API for all platforms• Language Independence: C, C++, Java, C#, .NET, ADA• Enterprise and Embedded Support

VxWorks, INTEGRITY, LynxOS

Linux, Solaris, Windows

• Prototype on any platform

RTI DDS: Pluggable Transports

• Enables non-IP centric transports (e.g InfiniBand)

• Allows for multiple transports on same node

• Provides high-performance (zero-copy interface)• Saves bandwidth (compact messages & encapsulation)

Standard IP network(Ethernet, Wifi, etc.)

IPv4 & IPv6

UDP

SharedMemory

InfiniBandCustom

(e.g. Radio)

RTI DDS

Real-time Applications

#5 Provides Real-Time Pub-Sub in SOA

Real-TimeDevices Fault

ToleranceAuditing & Recording

WS-DDS

Tools & Visualization

Database

EventProcessing

Real-Time Pub-Sub/Caching/MessagingSOA &

Real-TimeWeb Services

Real-Time SOA Architecture/Implementation

RT Architecture/Technologyl High Performancel Event-Driven/Publish-Subscribel Small footprintl Quality of Servicel Support for embedded

environmentsl Support for unreliable & low-

DDS Data Bus

l Support for unreliable & low-bandwidth networks

Traditional Enterprisel Low Performancel Client-Serverl Centralized (Server-based)l TCP based

Demo – Integrating COTS with DDS

l Display live DDS Data in Excell Perform real-time computations and chartsl Publish DDS data from Excel

blank history saved demo QoSShapesDemo

Integrating with DDS (1/3)

l Direct DDS integration (programmatic)– Most efficient approach to integration– APIs available in C, C++, Java, .NET (C#, C++/CLI)

l Direct JMS integration– RTI provides peer-peer interoperability between DDS and RTI’s – RTI provides peer-peer interoperability between DDS and RTI’s

Java Message Service driver

l Bridging via a Database– Synchronize between global data space and RDBMS– Support for Oracle RDBMS, Oracle TimesTen, MySQL

Integration with DDS (2/3)

l Web Services– Web Services interface to DDS (WS-DDS)– Mapping between DDS types and XML Schema

l Complex Event Processing– Designed for running queries of high-throughput streaming data– Designed for running queries of high-throughput streaming data– Provide a broad-selection of off-the-shelf input and output

adapters, lowering the cost of integration– Excellent technology to systems with network constraints,

requiring processing of high-throughput data in near-real time

Integrating with DDS (3/3)

Type PerformanceImpedance matching

Integrationeffort

Access to DDS Capabilities

Direct DDS P2P Highest DDS filters, history cache and QoS

Requires programming

All

Direct JMS P2P High DDS filters, history cache

Configuration only

Basic

Summary

history cache and QoS

only

Database bridge

Service-based

Medium to low

SQL in addition to DDS filters, history cache and QoS

Configuration only

Almost all

WS bridge Service-based

Low DDS filters, history cache and QoS

Configuration only

Almost all

CEP as bridge

Service-based

Medium CEP extensions to SQL in addition to DDS filters, history cache and QoS

Minimal Basic

l DDS is a mature international Standard from OMG– Platform Neutral: Operating systems and Programming

Languages– Deployed worldwide in Military systems and other

Demanding real-time applications

l DDS Is mandated by US DoD for Publish-Subscribe

Conclusions

l DDS Is mandated by US DoD for Publish-Subscribe and data-distribution applications

l DDS is ideally suited to Network-Centric Systems– Highly Tunable via Quality of Service (QoS)– Can accommodate unreliable & low-latency transports– Rich services (persistence, filtering, high-availability)

Integration Platform For JMPS Using DDS

Technology

Transcript of Integration Platform For JMPS Using DDS