The NaradaBroker: A Flexible Messaging Infrastructure

Rahim Lakhoo (Raz)

DSG Seminar 12th April 2004

2

Outline

• Introduction• Underlying Technologies

–Java Messaging Service (JMS)

• Overview of the NaradaBroker• Brokers Capabilities• Brokers Current Usages• Live Demo (screenshots)• Summary• Conclusions• Future Work

3

Introduction

• NaradaBrokering was developed by Community Grids Lab at Indiana University

• Distributed messaging infrastructure– Message orientated Middleware (MoM)

– Notification framework

• Originally designed to provide uniformed multicasting for supporting real-time collaboration via the publish/subscribe architecture

• NaradaBrokering is defined as a network of cooperating brokers

4

JMS• Unified messaging infrastructure from Sun Microsystems

• Support for messages containing Java objects• Provides guaranteed message delivery:

– Once-and-only-once delivery semantics– Client acknowledge mode or transacted sessions can be used to ensure transmission

• Used to provide Message Orientated Middleware• JMS domains can be classified as either:

– Point-to-Point– Publish/Subscribe

• Includes common API – email

5

JMS – Point-to-Point (PTP)• Point-to-Point messaging:

– Concept of FIFO message queues– Messages are addressed to a specific queue– Clients extract messages from the queue

6

JMS – Publish/Subscribe• A Topic is a JMS-managed object, which manages the

message flow from publisher to subscriber• Publish/Subscribe messaging:

– Messages are addressed to a topic with a content hierarchy– Generally topics are anonymous and dynamic– Can provide the ability for software multicast via topics

7

JMS Topics• Topics can be published as

– Tag pair value• Make=Ford, Model=Focus, Colour=*

– “/” separated string • /UoP/dsg/chat

– Integer• 10

• JMS also handles topic sessions

• JMS does not specify how to organise topics

• Messages can be missed due to latencies when a subscriber subscribes to a topic

8

JMS Message• Header:

– Contains ID– Used for routing messages

• Properties:– Optional Header field

– Extra Descriptions for the payload

• Body:– Stream message – Java primitive types

– Map message – Tag value pairs

– Text message – String type

– Object message – Java object

– Bytes message

9

What JMS does not do

• Load balancing

• Fault tolerance

• Error notification

• Administration

• Security

• Wire protocol

• Message repository

• Language for creating message definitions

10

NaradaBroker• NaradaBrokering is a collection of cooperating brokers• Implementation of JMS (1.0.2b compliant)• Event based messaging:

– Events are time stamped messages

• Asynchronous communications• Capable of supporting:

– Grid services– Web Services– P2P– Audio/Video conferencing (Java Media Framework)

• Publish/Subscribe architecture with XML based Topics

11

NaradaBroker continued• Supports multiple protocols:

– TCP/IP (blocking & non-blocking)– UDP– Multicast– SSL– HTTP– RTP– HHMS (PDA’s, mobile phones)

• Provides an interface to a reliable storage

• Compress/decompress messages (zlib)

• Capable of fragmenting large files (>1GB)

• Supports JXTA interactions

12

NaradaBroker continued• Cluster based architecture:

– Attempts to map the virtual network to the physical network via cluster groups

– Lowers the strain on the network infrastructure

– Each broker is assigned a logical address e.g. [1.1.1.1]

• Can bypass firewalls and work across NAT’s or proxies:

– Contains a transport probe that scans ports that are usable/open

13

NB Topology• Brokers are arranged hierarchically in clusters:

– Cluster– Super-Cluster– Super Super-Cluster

• Brokers each hold Broker Network Maps (BNM):

– Used to calculates the cost for traversals across connections

14

NaradaBrokering

15

Broker Locators• Broker Locators are used by clients to find Brokers to connect to

• The Broker Locator propagates the Brokers information to the client:

– Hostname/IP

– Listening port

– Transport protocol

• Client uses this information to transparently connect to the Broker

• Clients may have connections to multiple Brokers

• This provides load balancing

16

Broker Locators

17

Profiles•Profiles at a node signify an interest in events which match certain templates

• Includes a constraint that events need to satisfy before being routed

•Every profile has a unique ID •Each profile has a matching destination•This is generally considered as a subscription

•Subscriptions are in a hierarchy which can potentially overload a super super-cluster controller - solution is to add more controllers

18

Organisation & Routing

• Client subscriptions stored in a hierarchy within the system:

– Broker maintains client subscriptions– Cluster controller maintains broker subscriptions

• Events received are matched (matching engine) against stored profiles and destinations are computed:

– Cluster controller manages broker destinations– Broker manages client destinations

• Each broker has a set of destinations, it computes the best destination based on broker hops

19

Organisation & Routing

20

Event messaging

21

NB Matching Engines•Possibility for multiple matching engines at each node, depending on what needs to be supported

•Different types of constraints can be used on subscriptions/profiles for matching:

– String based Topics separated by “/”– Integer topics– Tag = value pairs– XPath queries– String based coupled with SQL like queries– Regular expressions

22

Types of Matching Engines• String based:

– Separated by “/” e.g. /UoP/dsg/chat

– Subscriptions to a topic does not include all sub-topics

• String based coupled with SQL like queries:– First matches the string based topic

– Second matches with SQL92 like query against the events properties

– Reduces amount of times SQL matches are calculated

• Tag = value pairs:– “,” separated <tag value> pairs

– Also allows for * type wildcard

• Integer based

23

Types of Matching Engines•XPath based queries:

– XPath query language searches, locates and identifies parts of an XML document

– Searches for XML advertisements across the Broker network

•Regular Expressions:– For text based content

24

Narada Transport Framework

Transport InterfacesLink

PerformanceData

TransportHandler Link

Factory

LinkFactory

Links

Data accumulated byMonitoring Service

Brokernode Administrative Link (HTTP)

Optimal Transport

TransportInterfaces

(Application andContent Dependent)

Negotiated with infoexchanged over

Administrative Link

Brokernode

MonitoringService

25

NB transport framework• User Channels:

– Applications interface with the broker– Supports the publish/subscribe architecture

• Links:– Implements a data protocol– Different underlying transport protocol– Administration link negotiates best available

communications protocol

26

Performance Monitoring• Performance monitoring:

– Monitors links originating from a node– Displays a set of metrics:

• Average delays• Round trip latency• Jitter• Loss rates• Throughput

–The metrics are returned to a performance aggregator• The metrics are available either via the built in application or

through the integrated web server• The performance data gathered is an XML document

27

Performance Aggregator

BrokerNode

LinkData

BrokerNode

LinkData

Performance AggregationService

Control MessageExchange

Aggregates infofrom nodes in acertain domain

MonitoringService

28

Security Framework

29

Security Framework• KMC – Key Management Centre:

– Includes Authorization module– Manages keys associated with entities and topics– Ensures secure communications with entities via SSL– Entities register their public key with the KMC– Multi-KMC also being investigated

• Comprises of two basic parts:– Authentication for all Publishers and Subscribers– Secured Publishing and Receiving

• ACL’s are associated to topics published• Similar to X.509 certificate chaining, KMC hierarchy designed to verify signatures

30

Security Framework• Supports different cryptographic algorithms with various different key sizes:

– DES/3DES– AES– RSA– RC2

• Designed to support:– JMS messages– JXTA messages– Audio/video conferencing

• KMC only a prototype and thus currently is a centralised implementation instead of distributed

31

JXTA• JXTA stands for Juxtapose, meaning side-by-side

• JXTA is a Peer-to-Peer protocol specification which defines a standard for the behavior JXTA peers

• Six protocols define the JXTA platform:– Peer Resolver Protocol (PRP) allows peers to send search

queries– Peer Discovery Protocol (PDP) allows peers to discover

advertisements– Peer Information Protocol (PIP) allows peers to poll the status

of another peer– Pipe Binding Protocol (PBP) allows peers to bind a pipe to a

peer– Endpoint Router Protocol (ERP) allows peers to request

routing information– Rendezvous Protocol (RVP), here peers subscribe to a

propagation service

32

NB-JXTA• JXTA is included with the NaradaBroker

• JXTA peers do not communicate with NaradaBrokers direct

•NaradaBroker provides a JXTA proxy

•This process is transparent to JXTA peers

•NaradaBroker adds its own header to JXTA messages

33

NB-JXTA

34

NB Web Services•WS-Eventing:

– Allows Web Services to subscribe to or accept subscriptions for event notification messages

•WS-ReliableMessaging:– Allows messages to be delivered reliably in the

presence of software component, system or network failures

•SOAP:– Support for SOAP messages will be provided in

version 1.0 of the NaradaBroker

35

NB Web Services

36

NB GridFTP•Provides support for GridFTP transfers

•Data from client stored in a local temporary file

•Data in temporary file is then read and sent over the NaradaBrokering system to the endpoint

•Temporary stores provide support for failures

•Seems to be a prototype implementation

37

NB GridFTP

38

Screenshots BrokerCommand used to access BrokerConnection made to

another broker

Link ID

Node Address Request

Distance calculation with hops

Operations available

New logical address

39

Screenshot JMS ChatInitial Chat application connectionLink registration

Publishing Topic

Subscribing Topic

Conversations between 3 clients across 3 brokers

40

Screenshots Video Transmitter

Receiver

Subscription to topic

41

Summary• JMS used for unified messaging and topics

• NaradaBroker supports multiple protocols

• Cluster based topology

• Only has partial capabilities for bypassing firewalls

• Only has a prototype security framework

• Capable of video conferencing

• Supports JXTA clients

• Prototype support for GridFTP

• Emerging Web Service support

• NaradaBroker is still pre version 1.0

42

Conclusions• Parts of the NaradaBroker has been hard coded for the Windows OS, which hampers its portability

• The Broker has the potential for bridging services

• Firewalls still cause issues for the broker

• Some stability issues are caused when links are abruptly terminated

• Will the integration of Grid Services still be feasible considering GT4 is based on Web Services?

• Version 1.0 is to be released soon, hopefully this will be more refined

43

Future work•Further investigation of the services provided by the NaradaBroker

• Investigate the feasibility of GridP2P with the NaradaBroker

•Collaborate with other universities to establish a UK testbed

•Evaluate the NaradaBroker across a Wide-Area-Network (WAN)

44

•University of Portsmouth•Members of the DSG•Community Grids Lab

Acknowledgements

45

Links• JMS - http://java.sun.com/products/jms/

• JXTA - http://www.jxta.org/

•NaradaBrokering – http://www.naradabrokering.org/

• JMF - http://java.sun.com/products/java-media/jmf/index.jsp

•GridFTP - http://www.globus.org/datagrid/gridftp.html