RDA3 Transport

Joel Lauener on behalf of the CMW team

26th June, 2013

Joel.Lauener@cern.ch

• Introduction• Architecture• Connection management• Messaging• Conclusions

Outline

1

Introduction

• Provide connection management and asynchronous bi-directional communication

• Keep it simple & stupid!• Abstraction of the underlying communication

library• Independent from other librairies

2

Introduction

Public API

3

Architecture

Dispatcher

• Push sockets from high level to transport• Callback from transport to high level

4

Architecture

Channels

• Admin socket:– Management– Small message size– Small message count

• Data socket:– Application data– Might block/overflow

5

Architecture

Frames

0x20 CLIENT_CONNECT0x21 CLIENT_REQ0x22 CLIENT_HB

0x01 SERVER_CONNECT_ACK0x02 SERVER_REP0x03 SERVER_HB

6

Architecture

Message

• Used to pass data between high layer and transport layer

• Wrapper around ZMQ message with custom free function

0 copy

7

Architecture

Connect (success)

8

Connection Management

Heartbeating

9

Connection Management

Connect (fail)

10

Connection Management

Client reconnect

11

Connection Management

Server disconnect

12

Connection Management

Client close

13

Connection Management

Connection states

14

Connection Management

Request

15

Messaging

Replies

16

Messaging

• Each data point can be multiplexed• Authorization layer (RBAC)• Cannot send first updates through the same

channelWe do not use pub/sub for notifications

• But we have 2 use cases for a pub/sub based transport: Timing and Orbit Feedback

What about pub/sub?

17

Messaging

• Powerful, easy to use, well documented• However…– In terms of error handling ROUTER is inconsistent

compared to DEALER, REQ and REP– ZMQ doesn’t give means to inspect/modify the

content of queues

ZeroMQ

18

Conclusions

Future plans

• Improve disconnection detection on the server side (ZMQ monitor or async HB)

• Implement graceful close• Investigate pub/sub• Think about replies priority• Performance testing and profiling• Should be ready by September ’13

19

Conclusions

• Transport is almost implemented– Working version in C++ and Java– 90% done, 90% of work left ;)– Some work left on error detection

• Pub/sub is not used for sending notifications• No showstoppers on the ZeroMQ side• Preliminary scalability tests are promising

Summary

20

Conclusions