Messaging for IoT
-
Upload
virtual-jboss-user-group -
Category
Software
-
view
2.316 -
download
0
Transcript of Messaging for IoT
![Page 1: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/1.jpg)
Messaging for IoTJBUG November 2015
Martyn Taylor, Dejan Bosanac
![Page 2: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/2.jpg)
Presenters
Martyn Taylor Senior Software Engineer at Red Hat Apache ActiveMQ Committer
Mainly working on Apache Artemis Keen interest in IoT
Bosanac Dejan Senior Software Engineer at Red Hat Apache ActiveMQ committer and PMC member Co-author of ActiveMQ in Action
![Page 3: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/3.jpg)
Agenda
IoT messaging basics Tasks and challenges Patterns and protocols
IoT messaging brokers Apache ActiveMQ ActiveMQ Artemis
IoT messaging scaling Vertical and horizontal Qpid Dispatch Router Scalable deployments
![Page 4: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/4.jpg)
IoT Messaging Basics
IoT Topology Messaging tasks Messaging challenges
![Page 5: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/5.jpg)
IoT Topology Big Data
Messaging Infrastructure
Analytics
Enterprise Middleware
Devices
![Page 6: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/6.jpg)
IoT Messaging Infrastructure
Task Provides connectivity between devices and back end systems
Challenges Interoperability Deployment environment High Availability Scalability
![Page 7: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/7.jpg)
IoT connectivity patterns
IoT Communication Patterns Telemetry, Command & Control, Enquiry, Notifications
Protocols and Technologies JMS MQTT AMQP
![Page 8: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/8.jpg)
Telemetry
• Device → Service• One way (push)• Failures tolerable• Lots of small data
![Page 9: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/9.jpg)
Command and Control
• Service → Device• Two way (Req/Resp)• Less, more important data• 1 → 1 and 1 → many
• grouping
![Page 10: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/10.jpg)
Enquiry
• Device → Service• 1 → 1• Two way (Req/Resp)
![Page 11: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/11.jpg)
Notification
• Service → Device• One Way (push)• 1 → Many
• Grouping
![Page 12: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/12.jpg)
Requirements on the transport
Communication from/to devices 1 → 1
Device Addressing 1 → Many
Group Addressing Push Request → Response
![Page 13: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/13.jpg)
IoT Networks
Unreliable Handle network failures
Expensive and Constrained Low network overhead
![Page 14: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/14.jpg)
IoT Devices
Limited Resources Low processing overhead
Large scale and varied Inter-operable
Battery powered Work offline
![Page 15: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/15.jpg)
IoT Environment Challenges
Interoperable Connection Failure
Handling Offline Low network overhead Low Processing overhead
![Page 16: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/16.jpg)
JMS
Addresses majority of requirements 1-1, 1-Many,Grouping,Device address
Queues, Topics, Message Selectors Push Req/Resp
Reply to queues Inter-operable
JVM Connection failures
Durable Messages and Various ACK Modes
![Page 17: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/17.jpg)
JMS
Inter-operability Java Runtime only Protocol vs API
Vendor specific Rich feature set
Complexity on the client Large footprint
Vendor protocols High network overhead
![Page 18: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/18.jpg)
MQTT
OASIS standard (v3.1.1) Created by IBM Light weight
Low network message Simple protocol
Pub / Sub Quality of Service Connection failures Very popular in IoT scenarios
![Page 19: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/19.jpg)
QoS Levels
![Page 20: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/20.jpg)
Overheads
Sending / Receiving messages QoS 0 = 8 bytes QoS 1 = min 12 bytes QoS 2 = min 20 bytes
Clients have small code foot print Simple protocol Burden on the broker
![Page 21: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/21.jpg)
Failures
![Page 22: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/22.jpg)
Reconnects
![Page 23: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/23.jpg)
Subscriptions
Topics Hierarchical addresses
e.g. building/core/floor/2/office/1 Supports wild cards
building/core/floor/# building/core/floor/+/office/+
QoS per topic Outlives a connection
![Page 24: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/24.jpg)
AMQP 1.0
International Standard (ISO/IEC ISO 19464) Binary Protocol Rich feature set:
Conversation multiplexing Advanced flow control Type system QoS Guarantees
Symmetrical message exchange No Broker required
![Page 25: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/25.jpg)
QoS Guarantees
At most once Fire and Forget
At least once Retry
Exactly once 3 Way Ack
![Page 26: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/26.jpg)
Type System
Highly Interoperable Rich Type Set
Primitives integer, long lists, maps
Descriptive types Allow application defined types
Mappings in most languages
![Page 27: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/27.jpg)
Symmetrical Protocol
Not Client -> Broker Broker less message
Client -> Client Client -> Router
Interesting features Smart routing
![Page 28: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/28.jpg)
Flow control & multi channels
Limit producer and consumer flow Each channel (link) can be controlled individually Limit telemetry flow for command messages
![Page 29: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/29.jpg)
Protocols Overview Lots of IoT protocol choices including several not discussed here The right choice depends of the scenario Scenarios may combine protocols
CoAPTarget usecase Long-haul (&
local)Long-haul Local (& long-
haul)Long-haul
Compactness High Medium Highest Verbose
Security SSL SSL DTLS SSL
Flow control No Yes No No
Structured message
No Yes No Yes
Complexity Medium High Low Lowest
![Page 30: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/30.jpg)
IoT Messaging Brokers
Apache ActiveMQ
ActiveMQ Artemis
![Page 31: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/31.jpg)
Apache ActiveMQ
Apache ActiveMQ is a high-performance, scalable messaging broker
Started as an open source JMS broker
Moved to Apache Software Foundation in 2006
Now widely used open source messaging system
![Page 32: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/32.jpg)
Apache ActiveMQ
Multi-Protocol/Multi-Language Support
Invented Stomp protocol in 2006
Now supports AMQP 1.0, MQTT 3.1.1, HTTP, STOMP protocols
Broad range of supported client libraries for all kinds of device hardware
![Page 33: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/33.jpg)
Apache ActiveMQ
Benefits Feature rich Battle-tested in many production environments Good security support Flexible Configurable Embeddable
![Page 34: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/34.jpg)
Apache ActiveMQ
Limitations Broker core getting old Written using synchronous thread-locking model
Limits vertical scalability of the broker Number of threads raise with number of connections
and destinations
![Page 35: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/35.jpg)
ActiveMQ Artemis
Multi Protocol Broker AMQP, MQTT, STOMP, OpenWire, Artemis Core JMS (API)
Started as HornetQ JBoss project in 2009 Embedded WildFly (JBoss AS) JMS messaging
service In 2014 donated to Apache ActiveMQ
Sub project ActiveMQ Artemis. Latest Release 1.1.0
![Page 36: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/36.jpg)
ActiveMQ Artemis: Core
Contains Broker Business logic Core maintains a tight scope
Message Routing Persistence Protocol utility API HA and Scaling
Highly Performance Protocols are plugged in
![Page 37: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/37.jpg)
ActiveMQ Artemis: Architecture
ArtemisCore
![Page 38: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/38.jpg)
Apache Artemis Summary
Great performance due to Reactive Architecture Efficient Append only Journal
Multi - Protocol Broker AMQP, MQTT, STOMP, CORE, OpenWire, JMS
HA and Scalability built in
![Page 39: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/39.jpg)
Apache Artemis Next Steps
Take features from ActiveMQ Enhance OpenWire for ActiveMQ compatibility JDBC Journal implementation More protocols
CoAP Your protocol here….
![Page 40: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/40.jpg)
IoT Messaging Scaling
Broker scaling Vertical Horizontal
Qpid Dispatch Router Scalable Deployment
![Page 41: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/41.jpg)
Broker – Vertical Scaling
Give broker enough resources Reduce thread usage
NIO transport Reactive architecture
Improve monitoring under stress Advisory messages Selective Mbean registration
![Page 42: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/42.jpg)
Broker – Horizontal Scaling
One broker can only do so much Horizontal scaling
Networks of brokers Load balance connections
Limitations All destinations on all brokers Broker network is the bottleneck
![Page 43: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/43.jpg)
Qpid Dispatch Router
Lightweight AMQP 1.0 message router written in C http://qpid.apache.org/components/dispatch-router/
Provides flexible and scalable interconnect between AMQP endpoints
![Page 44: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/44.jpg)
Qpid Dispatch Router
It is not a broker It never owns a message It propagates AMQP transfer, settlement and
disposition frames between endpoints Message based or link based routing
Router/device1
/device2
![Page 45: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/45.jpg)
Qpid Dispatch Router
Deployed in multiple router-broker-endpoint topology Redundant paths
RouterBroker
Router
![Page 46: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/46.jpg)
Qpid Dispatch Router
Cost-based route computation
RouterBroker
Router
![Page 47: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/47.jpg)
Qpid Dispatch Router
Automatic re-routing on failure
RouterBroker
Router
![Page 48: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/48.jpg)
Qpid dispatch router
Benefits Better scaling due to more focused tasks Smart routing can be used to partition the traffic Ideal candidate for gateway into the system
![Page 49: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/49.jpg)
Scalable deployments
Smarter scaling techniques using routers Connections concentration Destinations concentration Destination sharding Smart routing
![Page 50: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/50.jpg)
Scalable deployments
Combination of brokers and routers provides powerful tool box
Brokers should focus on storing messages
Routers should do the rest
Allows for horizontal scaling topologies that can solve IoT challenges
![Page 51: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/51.jpg)
Connection Concentration
Router
Broker
• Challenge: Reduce the number of connections on the broker
![Page 52: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/52.jpg)
Destination Concentration
Router
Broker
/building1/room1
/building1
/building1/room2
/building1/room2
• Challenge: Reduce the number of destinations on the broker
![Page 53: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/53.jpg)
Destination Sharding
Router
Broker
Queue.A
Broker
Queue.B
Challenge: Distribute destinations across brokers
![Page 54: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/54.jpg)
Deployments – Smart Routing
Router
BrokerQoS 0
Challenge: Decrease the load of messages on the broker
![Page 55: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/55.jpg)
Deployments – Smart Routing
Router
BrokerQoS 1
![Page 56: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/56.jpg)
Summary
IoT offers a new set of problems for communication Messaging is a good fit We need new tools and technologies to help
Protocols such as AMQP and MQTT help address some of these problems
ActiveMQ and Artemis are evolving to meet IoT demands
New tools such as Apache Dispatch Router allow us to do some novel things to help scale to really high numbers
![Page 57: Messaging for IoT](https://reader031.fdocuments.in/reader031/viewer/2022030317/5871406c1a28ab55588b4577/html5/thumbnails/57.jpg)
Resources
• Slides• http://http://www.slideshare.net/dejanb/messaging-for-iot
• MQTT• Spec: http://www.mqtt.org • Libraries and tutorials: http://www.eclipse.org/paho/
• AMQP• Spec: http://www.amqp.org/ • Libraries and tutorials: http://qpid.apache.org/
• ActiveMQ and Artemis• http://activemq.apache.org/
• Dispatch Router• http://qpid.apache.org/components/dispatch-router/