DDS Made Simple

OpenSplice

DDS

Angelo CORSARO, Ph.D.Chief Technology Officer OMG DDS Sig Co-Chair

[email protected]

DDS Made Simple

Copyrig

ht 2013, PrismTech – A

ll Rights Reserved.

OpenSplice

DDS

What I’ll Cover

☐ Understand DDS and its Data Sharing abstraction

☐ Learn about the benefits that Data Sharing provide over plain Messaging

☐ Learn how much simpler is to build distributed systems with DDS than with messaging technologies

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Distributed System Definition

A Distributed System is a collection autonomous software components distributed across a network that coordinate their activities and share system resources to run as a “single system”

N.B. It is useful to remark how this definition does not mention message passing or shared memory, those are implementation details

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Distributed Systems Challenges A number of challenges are inherent to the nature of distributed systems:

☐ Transparency

☐ Scalability

☐ Dependability

☐ Performance

☐ Flexibility

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Building Distributed Systems

Several abstractions have been proposed to build Distributed Systems:

☐ Distributed Shared Memory

☐ Message Passing/Queueing

☐ Distributed Objects

☐ Tuple Spaces

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Message Queueing☐ Message Queueing, as

suggested by its name, provide a coordination abstraction based on the exchange of messages between distributed processes

☐ Message delivery is often mediated by brokers

☐ A Message is composed by a header and a body, where the body is opaque (often binary) data

Broker

BodyHeader

Message

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Tuple Space☐ Tuple Space provide the

abstraction of a Global Tuple Space that can be used by autonomous applications to coordinate by inserting (out), reading (rd) and removing (in) tuples

☐ The Tuple Space model was initially introduced by Gelernter and Carrero’s Linda and extended over the years

<a,b,c>

<x,y, <j, k>><x,y, <j, k>>

out(<a,b,c>) rd(<a,_,_>)

in(<15,_>)eval(...)

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Tuple Spaces vs Messaging☐ Tuple Spaces provides spatial

and temporal decoupling along with content awareness

☐ Tuples remain in the tuple space up to when they are not removed

☐ Tuple Spaces allow application to share information

☐ Overall Tuple Spaces provide a high-level abstraction for building distributed systems

☐ Messaging doesn’t provide content awareness and usually provides limited temporal decoupling

☐ Messages usually disappear once they are delivered to consumers

☐ Messaging allows application to exchange messages. Information has to be reconstructed from messages

☐ Messaging provides a relatively low-level abstraction for building distributed systems

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Why is Messaging so Popular?At this point you may ask yourself:

If Tuple Spaces are so cool, why are messaging technologies so popular?

The answer is actually pretty simple:

Tuple Spaces where originally quite inefficient and this lead people to adopt different approaches, such as messaging and distributed objects.

Nowadays, however, there are plenty of technologies deriving from the tuple space concept that have very high performance (often superior than messaging).

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Why is Messaging so Popular?

At this point you may ask:

OK, that makes sense. Yet there are still plenty of people using messaging technologies. Why?

The answer is again pretty simple:

The adoption of Tuple Space inspired technologies is growing steadily, but many people are still using messaging since this is what they are familiar with. But enlightenment will eventually hit them ;-)

OpenSplice

DDS

DDS

OpenSplice

DDS Ba

DDSics

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Data Distribution Service

☐ Topics: Unit of data sharing

☐ DataWriters: data producers

☐ DataReaders: data consumers

DDS provides a relaxed Tuple Space Abstraction abstraction based on:

DDS Global Data Space

...

TopicA

TopicBTopicC

TopicD

Data Writer

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

For Real-Time Systems

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Data Distribution Service

☐ DataWriters and DataReaders are automatically and dynamically matched by the DDS Dynamic Discovery

☐ A rich set of QoS allows to control existential, temporal, and spatial properties of data

DDS Global Data Space

...

TopicA

TopicBTopicC

TopicD

Data Writer

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

For Real-Time Systems

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Fully Distributed Tuple Space☐ DDS Implements a fully

distributed Tuple Space with relaxed consistency.

☐ DDS provides eventual consistency as opposed to strong consistency

☐ In addition, DDS provides a local “take” and a global “dispose” in place of the “in” operation

<a,b,c>

<x,y, <j, k>>

<x,y, <j, k>>

out(<a,b,c>) rd(<a,_,_>)

in(<15,_>)eval(...)

Traditional Tuple Space

DDS Data SpaceLogical Global Data Space

Physical Local Data Space

The local Data Space is a projection of the Logical Global Data Space. This projection reflects the interest of the given

application.

The Global Data Space is an abstraction in DDS built by

the composition of Local Data Spaces.

Copyrig


ll Rights Reserved.

OpenSplice

DDS

What DDS Provides☐ Location Transparency => Dynamic Discovery

☐ Anonymity and Temporal Decoupling => Tuple Space

☐ Data Centricity => Tuple Space

☐ Dependability => Fault-Tolerant and Secure

☐ Scalability => Fully distributed architecture easy to scale-out

☐ Portability => OS and Programming Language Independence

☐ Interoperability => Standardized Wire Protocol

Copyrig


ll Rights Reserved.

OpenSplice

DDS

IDL

DDS Topics☐ A Topic defines a class of

streams ☐ A Topic has associated a user

defined extensible type and a set of QoS policies

☐ The Topic name, type and QoS defines the key functional and non-functional invariants

☐ Topics can be discovered or locally defined

DURABILITY,DEADLINE,PRIORITY,

…

“TVehicleDynamics”

TopicTypeName

QoS

struct VehicleDynamics { long vid; long x; long y; long dx; long dy;};#pragma keylist VehicleDynamics vid

VehicleDynamics

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Instances

☐ Each unique key value identifies a unique topic instance

☐ DDS not only demultiplexes “streams” but provides also lifecycle information

☐ A DDS DataWriter can write multiple instances struct Person {

long ssn; String name; String surname; };#pragma keylist Person ssn

<101,Leslie, Lamport >

<010,Edsger, Dijkstra>

<110,Haskell, Curry>

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Anatomy of a DDS Application

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Running App

☐ During this webcast we’ll be looking at a few DDS aspects through the iShapes applications

☐ The advantage of targeting the iShapes application is that we can actually “see” what our application will do

Copyrig


ll Rights Reserved.

OpenSplice

DDS

OpenSplice Mobile

Copyrig


ll Rights Reserved.

OpenSplice

DDS

In Action...

Copyrig


ll Rights Reserved.

OpenSplice

DDS

iShapes Information Model“Circle”

TopicType

NameQo

S

ShapeType

IDLstruct ShapeType { string color; long x; long y; long shapesize;};#pragma keylist ShapeType color

“Square”

TopicType

Name

QoS

ShapeType

“Triangle”

TopicType

Name

QoS

ShapeType

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Anatomy of a DDS ApplicationDomain Participant

[ISO C++ API]

auto dp = DomainParticipant(domainId);

Copyrig


ll Rights Reserved.

OpenSplice

DDS


Topics


// Create a Topicauto topic = Topic<ShapeType>(dp, “Circle”);

[ISO C++ API]

Copyrig


ll Rights Reserved.

OpenSplice

DDS


Topics


// Create a Publisherauto pub = Publisher(dp);


Publisher

// Create a Subscriberauto sub = Subscriber(dp);

Subscriber

[ISO C++ API]

Copyrig


ll Rights Reserved.

OpenSplice

DDS


Data Writer

Topics

// Create a DataWriterauto writer = DataWriter<ShapeType>(pub, topic);


// Create a Publisherauto pub = Publisher(dp);


Publisher

// Create a Subscriberauto sub = Subscriber(dp);

Subscriber

Data Reader // Create a DataWriterauto reader = DataReader<ShapeType>(sub, topic);

[ISO C++ API]

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Putting it all together 1 int main(int argc, char* argv[]) { 2 try { 3 DomainParticipant dp(0); 4 Topic<ShapeType> topic(dp, "Circle"); 5 Publisher pub(dp); 6 DataWriter<ShapeType> dw(pub, topic); 7 10 for (int i = 0; i < N; ++i) {11 ShapeType bc = {"RED", i, i, 60};12 ShapeType rc = {"BLUE", N-i, N-i, 60};13 dw.write(bc);14 // You can also write with streaming operators!15 dw << rc;16 std::cout << "." << std::flush;17 usleep(sleep_time);18 }19 20 } catch (const dds::core::Exception& e) {21 std::cout << e.what() << std::endl;22 }23 return 0;24 }

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Anatomy of a DDS Application

Domain Participant

Data Writer

Topics

// Create a DataWriterDataWriter<ShapeType> writer = pub.createDataWriter<ShapeType>(topic);

DomainParticipant dp = dpf.createParticipant(domainId);

// Create a PublisherPublisher pub = dp.createPublisher();

// Create a TopicTopic<ShapeType> topic = dp.createTopic(“Circle”,ShapeType.class);

Publisher

// Create a SubscriberSubscriber sub = dp.createSubscriber();

Subscriber

Data Reader

[Java 5 API] BootstrapServiceEnvironment env = ServiceEnvironment.createInstance(this.getClass.getClassLoader);DomainParticipantFactory dpf = DomainParticipantFactory.getInstance(env)

// Create a DataWriterDataReader<ShapeType> reader = sub.createDataReader<ShapeType>(topic);

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Anatomy of a DDS ApplicationDependency Injection through Implicits

Data Writer

Topics

// Create a DataWriterval writer = DataWriter[ShapeType](topic)

[Scala API]

import dds.config.DefaultEntities._

// Create a Topicval topic = Topic[ShapeType](“Circle”)

Data Reader // Create a DataWriterval reader = DataReader[ShapeType](topic)

OpenSplice

DDS

Writing Data

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Writing Data in DDS

writer.write(SpapeType(“RED”, 10, 20, 30));

// -- or -- writer << SpapeType(“RED”, 10, 20, 30);

[ISO C++ API]

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Writing Data in DDS[Java 5 API]

writer.write(new SpapeType(“RED”, 10, 20, 30));

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Writing Data in DDS[Scala API]

writer.write(new ShapeType(“RED”, 10, 20, 30))

// -- Or equivalently:

writer write (new ShapeType(“RED”, 10, 20, 30))

OpenSplice

DDS

Reading Data

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Reading Data// Read Dataauto data = reader.read();

// -- you can also read with streaming operatorsauto data = reader << read();

// Print the received data on the consolestd::for_each(samples.data().begin(), samples.data.end(), demo::printShape)

[ISO C++ API]

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Reading Data// You can read data on your favorite container using iteratorsauto samples = std::vector<Samples<ShapeType>>(MAX_SHAPES);

// Read data into the provided containerreader.read(samples.begin(), MAX_SHAPES);

// Print the received data on the consolestd::for_each(samples.data().begin(), samples.data.end(), demo::printShape)

[ISO C++ API]

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Reading Data

// Read Data Sample.Iterator<ShapeType> iterator = reader.read(); while (iterator.hasNext()) { // Assuming ShapeType ovrrides toString

out.println(iterator.next().getData()); }

[Java API]

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Reading Data

// Read Data reader.read foreach { s => println(s.data()) }

[Scala API]

OpenSplice

DDS

Selecting Samples

OpenSplice

DDS

Data Selectors

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Cherry Picking in DDS

☐ DDS provides some very flexible mechanisms for selecting the data to be read:☐ Data Content☐ Data Status

☐ These mechanisms are composable

OpenSplice

DDS

Content-Based Data Selection

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Filters and Queries☐ DDS Filters allow to control what gets

into a DataReader cache

☐ DDS Queries allow to control what gets out of a DataReader cache

☐ Filters are defined by means of ContentFilteredTopics

☐ Queries operate in conjunction with read operations

☐ Filters and Queries are expressed as SQL where clauses

DataReader Cache

DataReader

...... ... ...

Filter

Query

Application

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Querystruct VehicleDynamics { long vid; long x; long y; long dx; long dy;};#pragma keylist VehicleStatus vid

// Define the query and the parameters std::vector<std::string> p;p.push_back("100");p.push_back("100");dds::core::Query q("x < %0 AND y < %1", p.begin(), p.end());

auto data = reader .select() .content(q) .read();

[DDS C++ API 2010]

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Instances☐ DDS provides a very efficient way of reading data belonging to a

specific Topic Instance

☐ Obviously, one could use queries to match the key’s value, but this is not as efficient as the special purpose instance selector

// C++auto data = reader .select() .instance(handle) .read();

// Scalaval data = reader read(handle)

OpenSplice

DDS

State-Based Selection

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Sample, Instance, and View State☐ The samples included in the DataReader cache have associated

some meta-information which, among other things, describes the status of the sample and its associated stream/instance

☐ The Sample State (READ, NOT_READ) allows to distinguish between new samples and samples that have already been read

☐ The View State (NEW, NOT_NEW) allows to distinguish a new instance from an existing one

☐ The Intance State (ALIVE, NOT_ALIVE_DISPOSED, NOT_ALIVE_NO_WRITERS) allows to track the life-cycle transitions of the instance to which a sample belongs

Copyrig


ll Rights Reserved.

OpenSplice

DDS

State Selector in Action

// Read only new samplesauto data = reader .select() .state(status::DataState::new_data()) .read()

// Read any samples from live instancesauto data = reader .select() .state(status::DataState::any_data()) .read()

C++// Read only new samplesval data = reader read

// Read any samples from live instancesval data = reader read(SampleSelector.AnyData)

Scala

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Putting all Together

☐ Selectors can be composed in a flexible and expressive manner

C++auto data = reader ! .select() .instance(handle) ! ! .state(status::DataState::new_data()) ! ! .content(q) .read();

OpenSplice

DDS

QoS

Copyrig


ll Rights Reserved.

OpenSplice

DDS

QoS Model☐ QoS-Policies control local and

end-to-end properties of DDS entities

☐ Local properties controlled by QoS are related resource usage

☐ End-to-end properties controlled by QoS are related to temporal and spatial aspects of data distribution

☐ Some QoS-Policies are matched based on a Request vs. Offered Model thus QoS-enforcement

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

......

QoS QoS

Type Matching

DomainParticipant DomainParticipant

QoS QoS

Copyrig


ll Rights Reserved.

OpenSplice

DDS

QoS PoliciesQoS Policy Applicability RxO Modifiable

USER_DATATOPIC_DATA

GROUP_DATADURABILITYDURABILITY

SERVICEHISTORY

PRESENTATIONRELIABILITYPARTITION

DESTINATION ORDER

LIFESPAN

DP, DR, DW N Y

ConfigurationT N Y ConfigurationP, S N Y

Configuration

T, DR, DW Y N

Data AvailabilityT, DW N N

Data Availability

T, DR, DW N N

Data Availability

P, S Y N

Data Delivery

T, DR, DW Y N

Data DeliveryP, S N Y Data DeliveryT, DR, DW Y N

Data Delivery

T, DW N Y

Data Delivery

[T: Topic] [DR: DataReader] [DW: DataWriter] [P: Publisher] [S: Subscriber] [DP: Domain Participant]

Copyrig


ll Rights Reserved.

OpenSplice

DDS

QoS PoliciesQoS Policy Applicability RxO ModifiableDEADLINELATENCY BUDGET

TRANSPORT PRIORITY

TIME BASED FILTER

OWNERSHIPOWNERSHIP STRENGTHLIVELINESS

T, DR, DW Y Y

Temporal/Importance

Characteristics

T, DR, DW Y YTemporal/

Importance Characteristics

T, DW N YTemporal/

Importance Characteristics

DR N Y

Temporal/Importance

Characteristics

T, DR, DW Y NReplicationDW N Y Replication

T, DR, DW Y N Fault-Detection

[T: Topic] [DR: DataReader] [DW: DataWriter] [P: Publisher] [S: Subscriber] [DP: Domain Participant]

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Setting QoS Policies

C++// Setting Partition QoS-Policy on Publisherqos::PublisherQos pubQos;pubQos << policy::Partition("Partition");Publisher pub(dp, pubQoS);

// Setting various QoS-Policy on a Topicqos::TopicQos tqos;tqos << policy::Reliability::Reliable() << policy::Durability::Transient() << policy::History::KeepLast(5);

Topic<VehicleDynamics> topic(dp,"Partition", tqos);

OpenSplice

DDS

Ipse Dixit

For non latin speakers: "He, himself, said it"

Copyrig


ll Rights Reserved.

OpenSplice

DDS

The “Ipse Dixit” Dejavu☐ As an historical reference“Ipse

Dixit” was used by the Holy Inquisition to push back any argument that would contrast with established knowledge

☐ A famous example was the argument around the Geo-Centric model of the Solar System

☐ What has this to do with Middleware... Let me show you!

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Copyrig


ll Rights Reserved.

OpenSplice

DDS

What’s Wrong with this Paper?☐ When looking at DDS the

Authors did not refer to the state of the art

☐ DDS is discarded as being too complicated (really?!?!) -- they did not check out the new APIs available since 2010

☐ Beyond that, I think the authors where comparing a bit apples and oranges since DDS provides far more than ZeroMQ!

Copyrig


ll Rights Reserved.

OpenSplice

DDS

DDS vs. 0MQ -- What’s Simpler?import dds.config.DefaultEntities._


// Create a DataWriterval writer = DataWriter[ShapeType](topic)

// Create a DataWriterwriter write (new ShapeType(“RED”, 1,2,3))

import zmqimport time

// Create a Contextcontext = zmq.Context()

// Create a Socketpublisher = context.socket (zmq.PUB)

// Bind the socket to a namepublisher.bind ("ipc://nasdaq-feed")

// Send Datapublisher.send (message)

Sending Data

Copyrig


ll Rights Reserved.

OpenSplice

DDS

DDS vs. 0MQ -- What’s Simpler?import zmqimport time

// Create a Contextcontext = zmq.Context()

// Create a Socketsubscriber = context.socket (zmq.SUB)

// !!! MANUALLY CONNECT IT !!! What if your // topology changes?!?subscriber.connect ("tcp://192.168.55.112:5556")subscriber.connect ("tcp://192.168.55.201:7721")

// Define subscrptionsubscriber.setsockopt (zmq.SUBSCRIBE, "NASDAQ")

// Receive messagemessage = subscriber.recv()// ... Now decode the msg and do something // with it

Receiving Data

import dds.config.DefaultEntities._


// Create a DataReaderval reader = DataReader[ShapeType](topic)

// Create a DataWriterreader.read() foreach(s => println(s.data))

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Concluding Remarks

☐ DDS provides a powerful and feature-rich abstraction for building distributed systems

☐ This technology is widely used in mission and business critical systems and it being swiftly adopted in data-centric/big-data systems

Copyrig


ll Rights Reserved.

OpenSplice

DDS

Copyrig


ll Rights Reserved.

OpenSplice

DDS

¥@prismtech

¥@acorsaro

¥youtube.com/opensplicetube ¥slideshare.net/angelo.corsaro

¥ [email protected]¥[email protected]

¥opensplice.com ¥forums.opensplice.org

¥opensplice.org ¥[email protected]

:: Connect with Us ::

http://www.youtube.com/opensplicetube

http://www.slideshare.net/angelo.corsaro



mailto:[email protected]




http://www.opensplice.com




http://forums.opensplice.org/

http://forums.opensplice.org/

http://www.opensplice.org








DDS Made Simple

Technology

Transcript of DDS Made Simple