DDS over Low Bandwidth Data Links:Tactical Radios, Satellite, etc.

Connext Conference - London

October/08/2014

Jaime Martin LosaCEO eProsima

JaimeMartin@eProsima.com+34 607 91 37 45 www.eProsima.com

eProsima in one shot

Experts on middleware, focused on DDS. OMG Members.

– RPC over DDS, Web Enabled DDS, DDS Security (Supporter)

Army Interoperability Standards Contributor– Spanish Army: Tactical Data Interface (IDT)– MIP: Adem Model

RTI Spanish Distributor

.

Agenda

Initial Motivation, solution and results DDS main concerns in Low Bandwidth Links eProsima Low Bandwidth Plugins for DDS (Now RTI DIL

Plugins)– Discovery Plugin– Compression Transport Plugin– RTPS Header Reduction Transport Plugin– Link Simulation Plugin

Some More hints - Questions Appendix:

– About eProsima– eProsima Success Cases

C2 Interoperability Low Bandwidth Data Links

Initial Motivation, Solution and Results

DDS in Low Bandwidth Enviroments

Initial Motivation

Bring DDS communications to the C2 Tactical level (Spanish Army):– Low Bandwidth Tactical Radios:

From 4800 bps shared

– Frequent disconnections. – High packet loss rate.

Solution: Low Bandwidth Plugins for DDS– Simplified Discovery– Optimized RTPS headers– Compression

Results

DDS is mandated by the spanish Army for C2 interoperability in Tactical Radios, Satellite and Lan

Formal Interoperability Specification released, specifying how to use DDS and the information model (Spanish Tactical Data Interface, IDT)

Next Step: International C2 Interoperability: MIP ADEM

MIP: C2 International Interoperability (27 Countries)– Large and complex model

ADEM: Alternative Data Exchange Method (Simplified MIP Model)– Uses DDS for the Low

Bandwidth profile

Real Performance Example: ADEM

ADEM MIP International Test (Jan 2014)– 2 C2 Nodes, Tactical Radios,

4800 bps SHARED = 600 bytes/sec

– 12 Unit Positions/Second– Payload 44 bytes/position: 528 bytes/sec– Add RTPS & IP Headers: >100% of

available bandwidth used: Batching & Compression Optimized RTPS headers Simplified Discovery

Spanish Army Performance Test

Spanish Army Testing Example– VHF Radios– Bandwidth 4800 Bits/second SHARED– Number of nodes: 6

Example running 70 minutes:– 5 nodes: Each one sending its position and 3 more unit

positions every 30s (5*4=20 positions in total/30 sec)– 1 node sending its position and 11 more (12 in total/30 sec)– 32 Positions/30 Sec (45 bytes/position)– Other traffic:

1 alarm every 180 sec 1 tactical message every 360 sec 2 obstacles (6 point area) every 1200 sec 2 Tactical line (6 point area) every 1200 sec 2 installations (6 point area) every 1200 sec

DDS main concerns in Low Bandwidth Links

DDS in Low Bandwidth Enviroments

DDS main concerns in Low Bandwidth Links

Automatic Discovery is very chatty. – For N nodes the number of messages is proportional

to N^2– DDS is not going to work well Out of the box in

bandwidths bellow 64kbits DDS does not include data compression The protocol (RTPS) headers are big for this

kind of networks. Fortunately, we can use the Low Bandwidth

plugins for DDS to solve the problem

Low Bandwidth Discovery Plugin

DDS in Low Bandwidth Enviroments

Overview

What is discovery? Discovery phases

– Participant discovery phase– Endpoint discovery phase

Low Bandwidth Discovery Plugin (LBDP)

What is discovery?

The process by which domain participants find out about each other’s entities– Each participant maintains database on other

participants in the domain and their entities Happens automatically behind the scenes

– “anonymous publish-subscribe” Dynamic discovery

– Participants must refresh their presence in the domain or will be aged out of database

– QoS changes are propagated to remote participants

Discovery phases

Two consecutive phases– Participant discovery phase

Participants discover each other Best-effort communication, multicast (default)

– Endpoint discovery phase Participants exchange information about their datawriter

and datareader entities Reliable communication, unicast(default)

Steady state traffic to maintain liveliness of participants

Participant discovery phase

Participants periodically announce their presence using RTPS DATA message– Contains participant GUID, transport locators, QoS– Initially sent to all participants in “initial peers” list,

then sent periodically to all discovered participants – Sent using best-effort

Peer 1 (up)

Peer 2 (up)

Peer 3 (down)

Hello!

Hello!

Hello!

Initial peers:Peer 1Peer 2Peer 3

Endpoint discovery phase

Conversation between each pair. DataWriter/DataReader discovery

– Send out pub/sub DATA to every new participant– NACK for pub/sub info if not received from a known

participant – Send out changes/additions/deletions to each

participant Uses reliable communication between

participants DDS matches up local and remote entities to

establish communication paths

Discovery start-up traffic

Node A Node B

Participant created on ASend DATA to peer hosts

Participant created on BSend DATA to peer hosts

DATA participant A

DATA participant B

DATA participant A

DATA participant B

User creates Data Writer FooSend DATA to participantsin database

DataWriter DATA Foo Add publication C to databaseof remote publications

random sleep

random sleep

Add B to database of participants

Add A to database of participants

Already know about B

(sent reliably)

Discovery Implementation

Discovery is implemented using DDS entities known as Built-in Data Writers and Built-in Data Readers– Uses same infrastructure as user defined Data

Writers/Data Readers– Participant data is sent best effort & multicast – Publication/subscription data is sent reliably &

unicast

Three Built-in topics (keyed):– DCPSParticipant– DCPSPublication– DCPSSubscription

Discovery Entities

Participant 1

Participant Built-inData Reader

PublicationBuilt-inData Writer

SubscriptionBuilt-inData Reader

ParticipantBuilt-in Data Writer

SubscriptionBuilt-inData Writer

PublicationBuilt-inData Reader

Participant 2

ParticipantBuilt-inData Reader

PublicationBuilt-inData Writer

SubscriptionBuilt-inData Reader

Participant Built-inData Writer

SubscriptionBuilt-inData Writer

PublicationBuilt-inData Reader

Subscription Data MsgParticipant Data Msg Publication Data Msg

Best Effort, Multicast Reliable, Unicast

LBDP

Goals:– Reduce the discovery information transmitted.– Reduce net traffic: Less Packets.

Scenario:– We now most details of the participant applications in

advance.

Solution:– Suppress second discovery phase.– Mixed static and dynamic behavior.– Information about endpoints stored in XML files or

Databases

LBDP: Discovery Entities

Participant 1

Participant Built-inData Reader

PublicationBuilt-inData Writer

SubscriptionBuilt-inData Reader

ParticipantBuilt-in Data Writer

SubscriptionBuilt-inData Writer

PublicationBuilt-inData Reader

Participant 2

ParticipantBuilt-inData Reader

PublicationBuilt-inData Writer

SubscriptionBuilt-inData Reader

Participant Built-inData Writer

SubscriptionBuilt-inData Writer

PublicationBuilt-inData Reader

Participant Data Msg

Best Effort

XML XML

XML XML

Low Bandwidth Compression transport

DDS in Low Bandwidth Enviroments

LBCT: Overview

Compression at Transport Level Several compression libs used Several modes of operation

LBCT: Compression at Transport Level

Compression at Transport Level– Stackable: Use it over any transport: UDP, Serial, Ad

hoc…

LBCT:Several compression libs

Several compression libs can be used:– ZLIB– BZIP2 – LZO : LZO1X, LZO1B & LZO1F – UCL : UCL_NRV2B, UCL_NRV2D & UCL_NRV2E

Easy to add more by the user.– Through Public API.

Several modes of operation

Several modes of operation:– Fixed Algorithm– Algorithm depending on packet size.– Automatic: when CPU is not the bottleneck, the

plugin select the best algorithm for each package.

Results

40% improvement in standard Discovery traffic– Applies also to Static Discovery

60-80% improvement in Data– We used several sample applications

from command and control systems of Spanish Army.

Low Bandwidth RTPS Transport

DDS in Low Bandwidth Enviroments

LB RTPS: Overview

Optimized RTPS for low bandwidth scenarios Implemented as a transport.

LB RTPS: Optimized RTPS

RTPS Optimizations:– RTPS Header from 20 bytes to 1 byte.– RTPS SubmessageHeader from 4 to 1 byte.– RTPS extraflags for DATA and DATA_FRAG

eliminated (1 byte)– ReaderID and WriterID from 4 to 1 byte each (so 2^3

writers or readers per participant)– SequenceNumber from 8 to 5 or less bytes (more

than enough for these scenarios) Save more than 30 bytes!

LB RTPS: Implemented as a transport

Implemented as a transport Stackable:

– Can be used with any transport and it is stackable, so for example you could use:

– LB RTPS -> UDP– LB RTPS -> Compression Transport -> UDP

Low Bandwidth Simulation Transport

DDS in Low Bandwidth Enviroments

LB Simulation Transport: Overview

Simulate your low bandwidth scenario Implemented as a Transport plugin (stackable) Two operation modes:

– Simple Channel mode: Easy to Set Up The bandwidth is controlled for each node independently

– Advanced Channel mode: Bandwidth controlled accounting the activity in all the

nodes.

LB Simulation Transport: Simulate your low bandwidth scenario

Simulate your low bandwidth scenario:– Designed to cover a variety of devices:

Tactical Radios Satellite links

– General purpose.

Low Bandwidth: Hints

DDS in Low Bandwidth Enviroments

Low Bandwidth Scenarios: Hints

Use Best Effort or NACK Based Reliability Use Multicast in Radio Scenarios Flow controllers Optimize Types.

– Sparse Types.

Call us

Low Bandwidth Scenarios: Hints (II)

eProsima Smart Flow Controller for DDS– Take into account the network state in real time

The product calculates the available bandwidth in real time with the latencies & packet loss

– Assign real priorities and bandwidth resources to your DDS Topics In Terms of the available bandwidth.

Want to know more?

www.eProsima.com

Youtube: https://www.youtube.com/user/eprosima

Mail: JaimeMartin@eProsima.com Phone: +34 607913745

Twitter: @jaimemartinlosa http://es.slideshare.net/JaimeMartin-eProsima

Appendixes

About eProsima

About eProsima

Experts on middleware, focused on DDS. OMG Members.

Army Interoperability Standards– Spanish Army: Tactical Data Interface (IDT)– MIP: Adem Model

.

About eProsima: Products And Services

eProsima Products:– DDS based: Plugins, add-ons, adaptors, etc

Services:– Communication modules, App development, DDS

training, Support. R&D:

– R&D Projects with enterprises and universities. Quality: ISO 9001

– Design, Development, Marketing and Support of Software.

Customers (I)

Amper Programas:– BMS– Simacet (Main Spanish C2 System)

Cassidian: – UAVs - Neuron, Atlante

Ground Station Comm Server

– Comfut INDRA:

– Defense (BMS, UAV PASI)– Air Traffic Control, – SESAR, ATC Interoperability– Energy (InSpeed)

Spanish Army:, – IDT :Tactical Data Interface

Customers (II)

Isdefe Spanish Army: JCISAT, DGAM CATEC-FADA: R&D Aerospatial Santa Barbara: Armoured Vehicles RTI GMV

Customers (III)

Tecnobit: COSMOS, Reserved Projects. IKERLAN: R&D. Navantia: F105 (Aegis) Boeing: Atlantida, Swim suit

eProsima Products.- Index

eProsima Smart Flow Controller for DDS.– Flow control for Low Bandwidth

eProsima RPC over DDS:– Remote procedure calls framework over DDS.

eProsima Fast Buffers– Fast Serialization engine.

eProsima Dynamic Fast Buffers.– Fast Serialization engine. No IDL Required,

serialization support is generated at runtime. eProsima DDS Non-Intrusive Recorder.

– Stores DDS communication history in a data base.

Ongoing Project

FP7: KIARA, Future Internet Middleware– FI-WARE 1st open call– Based on eProsima RPC over DDS & OMG DDS– Lots of new features:

Improved IDL Direct Use of Application native types New formats of marshalling (SOAP, RestFul) Web Services compatibility Protocol negotiation Extended transport support High performance dispatching agent (RPC)

eProsimaSuccess Cases

RTI DDS DIL Plugins:Disconnected and Intermitent Links

eProsima developed the plugins for the Spanish Army Tactical Radios, and later were adquired by RTI.

Allow the use of DDS in very low bandwidth links, such as Tactical Radios and Satellite.– Tested from 2400 bps

Tactical Data Interface: Spanish Army

C2 Interoperability Comm layer:– Tactical Radios

From 2400bps

– Satellite

Mandated for all the Spanish Army C2 systems.– Already implemented in the

their main C2 systems

eProsima developed the army C2 comm layer using RTI Connext DDS optimized for low bandwidth enviroments. The project included the design of the Data Model and QoS requisites for the Army.

C2 Systems: INDRA & Amper

eProsima Provides a DDS based comm layer for INDRA and Amper C2 Systems.

eProsima implemented the mandated Spanish Army Tactical Data Interface for Simacet (Main Spanish Army C2 System, Amper) and BMS (Tanks C2 System, INDRA & Amper)

Tactical Messaging Bridge

Unified mail and chat: Internet, NATO and Tactical for the Spanish Army.

Enable Complete Messaging on the tactical radio network.

SESAR - INDRA ATC

eProsima provides middleware research and prototyping for ATC Interoperability

Among the different middleware technologies studied, DDS and WS are the SESAR proposed technologies for ATC interoperability.

Cassidian: nEURon and Atlante GS

eProsima provides the comm layer for the ground station comm server.

eProsima Non-Intrusive Recorder is used to record the communications for later analisys.

FI-WARE Middleware

eProsima has been selected to develop Future Internet Middleware in the FI-WARE programme.

DDS will be the core technology

Fi-WARE is a consortium of over more than 30 companies and universities including Telefonica, Siemens, SAP, Atos…

eProsima will partner in this project with the German Universities DKFI and CISPA and the Swiss ZHAW.

Remote Application

Client / Server,Publisher /Subscriber

Application

API / Data Access

Marshalling

TransportMechanism

s

Wire- Protocols

Transport Protocols UDPTCPTLS, DTLS

Shared Memory

Backplane/ Fabric

XML JSON CDR

SDN Plugin

Data Transfer

Compile time orEmbedded Runtime Compiler/Interpreter

Data / Function Mapping

Declarative Data/Function descritption

Security / QoS Policy

Security / QoSParameter

Function

Stub

Function Skleleton

QoS

DataWriter

DataReader

-DDS / RTPS

REST / HTTP

RPC Pub/Sub

Negotiation

Publisher SubscriberRPC

ServerRPC ClientPrepare Initialize

IDLParser

• IDLbased on OMG IDL

• WADL

Security

Dispatching

I2ND GE

FI-WARE Middleware: DDS Based

Thank you!

Jaime Martin LosaCEO eProsima

JaimeMartin@eProsima.com+34 607 91 37 45 www.eProsima.com