Messaging for IoTJBUG November 2015

Martyn Taylor, Dejan Bosanac

Presenters

Martyn Taylor Senior Software Engineer at Red Hat Apache ActiveMQ Committer

Mainly working on Apache Artemis Keen interest in IoT

Bosanac Dejan Senior Software Engineer at Red Hat Apache ActiveMQ committer and PMC member Co-author of ActiveMQ in Action

Agenda

IoT messaging basics Tasks and challenges Patterns and protocols

IoT messaging brokers Apache ActiveMQ ActiveMQ Artemis

IoT messaging scaling Vertical and horizontal Qpid Dispatch Router Scalable deployments

IoT Messaging Basics

IoT Topology Messaging tasks Messaging challenges

IoT Topology Big Data

Messaging Infrastructure

Analytics

Enterprise Middleware

Devices

IoT Messaging Infrastructure

Task Provides connectivity between devices and back end systems

Challenges Interoperability Deployment environment High Availability Scalability

IoT connectivity patterns

IoT Communication Patterns Telemetry, Command & Control, Enquiry, Notifications

Protocols and Technologies JMS MQTT AMQP

Telemetry

• Device → Service• One way (push)• Failures tolerable• Lots of small data

Command and Control

• Service → Device• Two way (Req/Resp)• Less, more important data• 1 → 1 and 1 → many

• grouping

Enquiry

• Device → Service• 1 → 1• Two way (Req/Resp)

Notification

• Service → Device• One Way (push)• 1 → Many

• Grouping

Requirements on the transport

Communication from/to devices 1 → 1

Device Addressing 1 → Many

Group Addressing Push Request → Response

IoT Networks

Unreliable Handle network failures

Expensive and Constrained Low network overhead

IoT Devices

Limited Resources Low processing overhead

Large scale and varied Inter-operable

Battery powered Work offline

IoT Environment Challenges

Interoperable Connection Failure

Handling Offline Low network overhead Low Processing overhead

JMS

Addresses majority of requirements 1-1, 1-Many,Grouping,Device address

Queues, Topics, Message Selectors Push Req/Resp

Reply to queues Inter-operable

JVM Connection failures

Durable Messages and Various ACK Modes

JMS

Inter-operability Java Runtime only Protocol vs API

Vendor specific Rich feature set

Complexity on the client Large footprint

Vendor protocols High network overhead

MQTT

OASIS standard (v3.1.1) Created by IBM Light weight

Low network message Simple protocol

Pub / Sub Quality of Service Connection failures Very popular in IoT scenarios

QoS Levels

Overheads

Sending / Receiving messages QoS 0 = 8 bytes QoS 1 = min 12 bytes QoS 2 = min 20 bytes

Clients have small code foot print Simple protocol Burden on the broker

Failures

Reconnects

Subscriptions

Topics Hierarchical addresses

e.g. building/core/floor/2/office/1 Supports wild cards

building/core/floor/# building/core/floor/+/office/+

QoS per topic Outlives a connection

AMQP 1.0

International Standard (ISO/IEC ISO 19464) Binary Protocol Rich feature set:

Conversation multiplexing Advanced flow control Type system QoS Guarantees

Symmetrical message exchange No Broker required

QoS Guarantees

At most once Fire and Forget

At least once Retry

Exactly once 3 Way Ack

Type System

Highly Interoperable Rich Type Set

Primitives integer, long lists, maps

Descriptive types Allow application defined types

Mappings in most languages

Symmetrical Protocol

Not Client -> Broker Broker less message

Client -> Client Client -> Router

Interesting features Smart routing

Flow control & multi channels

Limit producer and consumer flow Each channel (link) can be controlled individually Limit telemetry flow for command messages

Protocols Overview Lots of IoT protocol choices including several not discussed here The right choice depends of the scenario Scenarios may combine protocols

CoAPTarget usecase Long-haul (&

local)Long-haul Local (& long-

haul)Long-haul

Compactness High Medium Highest Verbose

Security SSL SSL DTLS SSL

Flow control No Yes No No

Structured message

No Yes No Yes

Complexity Medium High Low Lowest

IoT Messaging Brokers

Apache ActiveMQ

ActiveMQ Artemis

Apache ActiveMQ

Apache ActiveMQ is a high-performance, scalable messaging broker

Started as an open source JMS broker

Moved to Apache Software Foundation in 2006

Now widely used open source messaging system

Apache ActiveMQ

Multi-Protocol/Multi-Language Support

Invented Stomp protocol in 2006

Now supports AMQP 1.0, MQTT 3.1.1, HTTP, STOMP protocols

Broad range of supported client libraries for all kinds of device hardware

Apache ActiveMQ

Benefits Feature rich Battle-tested in many production environments Good security support Flexible Configurable Embeddable

Apache ActiveMQ

Limitations Broker core getting old Written using synchronous thread-locking model

Limits vertical scalability of the broker Number of threads raise with number of connections

and destinations

ActiveMQ Artemis

Multi Protocol Broker AMQP, MQTT, STOMP, OpenWire, Artemis Core JMS (API)

Started as HornetQ JBoss project in 2009 Embedded WildFly (JBoss AS) JMS messaging

service In 2014 donated to Apache ActiveMQ

Sub project ActiveMQ Artemis. Latest Release 1.1.0

ActiveMQ Artemis: Core

Contains Broker Business logic Core maintains a tight scope

Message Routing Persistence Protocol utility API HA and Scaling

Highly Performance Protocols are plugged in

ActiveMQ Artemis: Architecture

ArtemisCore

Apache Artemis Summary

Great performance due to Reactive Architecture Efficient Append only Journal

Multi - Protocol Broker AMQP, MQTT, STOMP, CORE, OpenWire, JMS

HA and Scalability built in

Apache Artemis Next Steps

Take features from ActiveMQ Enhance OpenWire for ActiveMQ compatibility JDBC Journal implementation More protocols

CoAP Your protocol here….

IoT Messaging Scaling

Broker scaling Vertical Horizontal

Qpid Dispatch Router Scalable Deployment

Broker – Vertical Scaling

Give broker enough resources Reduce thread usage

NIO transport Reactive architecture

Improve monitoring under stress Advisory messages Selective Mbean registration

Broker – Horizontal Scaling

One broker can only do so much Horizontal scaling

Networks of brokers Load balance connections

Limitations All destinations on all brokers Broker network is the bottleneck

Qpid Dispatch Router

Lightweight AMQP 1.0 message router written in C http://qpid.apache.org/components/dispatch-router/

Provides flexible and scalable interconnect between AMQP endpoints

Qpid Dispatch Router

It is not a broker It never owns a message It propagates AMQP transfer, settlement and

disposition frames between endpoints Message based or link based routing

Router/device1

/device2

Qpid Dispatch Router

Deployed in multiple router-broker-endpoint topology Redundant paths

RouterBroker

Router

Qpid Dispatch Router

Cost-based route computation

RouterBroker

Router

Qpid Dispatch Router

Automatic re-routing on failure

RouterBroker

Router

Qpid dispatch router

Benefits Better scaling due to more focused tasks Smart routing can be used to partition the traffic Ideal candidate for gateway into the system

Scalable deployments

Smarter scaling techniques using routers Connections concentration Destinations concentration Destination sharding Smart routing

Scalable deployments

Combination of brokers and routers provides powerful tool box

Brokers should focus on storing messages

Routers should do the rest

Allows for horizontal scaling topologies that can solve IoT challenges

Connection Concentration

Router

Broker

• Challenge: Reduce the number of connections on the broker

Destination Concentration

Router

Broker

/building1/room1

/building1

/building1/room2

/building1/room2

• Challenge: Reduce the number of destinations on the broker

Destination Sharding

Router

Broker

Queue.A

Broker

Queue.B

Challenge: Distribute destinations across brokers

Deployments – Smart Routing

Router

BrokerQoS 0

Challenge: Decrease the load of messages on the broker

Deployments – Smart Routing

Router

BrokerQoS 1

Summary

IoT offers a new set of problems for communication Messaging is a good fit We need new tools and technologies to help

Protocols such as AMQP and MQTT help address some of these problems

ActiveMQ and Artemis are evolving to meet IoT demands

New tools such as Apache Dispatch Router allow us to do some novel things to help scale to really high numbers

Resources

• Slides• http://http://www.slideshare.net/dejanb/messaging-for-iot

• MQTT• Spec: http://www.mqtt.org • Libraries and tutorials: http://www.eclipse.org/paho/

• AMQP• Spec: http://www.amqp.org/ • Libraries and tutorials: http://qpid.apache.org/

• ActiveMQ and Artemis• http://activemq.apache.org/

• Dispatch Router• http://qpid.apache.org/components/dispatch-router/