MixIT 14 - How to choose your message broker
description
Transcript of MixIT 14 - How to choose your message broker
How to choose your message broker
Guillaume Arnaud @guillarnaud
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
Offtopic
�2
commercial products low latency
benchmarks ESB
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
Benchmarks
�3
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
Exchange information !
�4
notifications
RPC
log centralized
dashboard
synchronization
worker
metrics
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
Once upon a time JMS…
�5
API (!=protocol)
JMS ∈ JEE
100% Java
JMS 2.02001
Transaction
decoupling
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
JMS: client side
�6
connexion
session
broker
client
clientsessiontcp!transport
buffer
buffer
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
Beware of wrong asynchronism
�7
connection = new ActiveMQConnectionFactory( "failover:(tcp://localhost:61616)?timeout=1000") .createConnection(); … producer.send(queue,message);
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
Exit strategy
�8
bounded
ABC
A consumer
queue dead letter queue
B
C
Time To Live
10 seconds
=10 seconds
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
JMS: queue vs topic
�9
ABC
A
B
C
consumer 1
queue
consumer 2
consumer 3
ABC
A
C
consumer 1
topic
consumer 2
consumer 3
BC
ABC
B
consumer 3
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
AMQP: le challenger
�10
100% Protocol
interoperability
routing
2006
0.9.1 vs 1.0
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
broker
AMQP
�11
exchange
queue
publisher
subscriber
subscriberqueue
queue.a.b
queue.a.c
queue.a.*
binding
virtual host
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
AMQP: topology
�12
exchange
queue
queue
queue
queuefanout
exchange queue.a.b
fanout
direct
exchangequeue.*.#
queue topic
exchangequeue.a.b (ignored)
queueheader1: value1 header2: value2 …
headers
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
STOMP: light messaging
�13
textual
light
Simple Text Orientated Messaging Protocol
2010 (?)
incompatibility
heartbeat
js + ws
security
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
STOMP: message structure
�14
COMMAND header1:value1 header2:value2 !Body^@
CONNECT accept-version:1.0,1.2 host:myhost heart-beat: 10,20 !̂@
CONNECTED version:1.2 heart-beat:10,20 !̂@
SEND destination:/queue/a content-type:text/plain !hello ^@
SUBSCRIBE id:0 destination:/queue/a ack:client !̂@
MESSAGE subscription:0 message-id:003 destination: /queue/a content-type:text/plain !hello ^@
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
MQTT
�15
Device
QoSCompact
2007
Last Known Good
semantic
Highly constrained
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
MQTT
�16
sender serverpublishQoS 0: at most once delivery
client serverpublish
ack
QoS 1: at least once delivery
QoS 2: exactly one delivery
receiver
receiver
publish
publish
client server
publish
receive
receiverpublish
complete
release
idid
id
id
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
Kafka
�17
distributed
complexity
MessageSet
ZeroCopy
2011
consumer oriented
ordering
append only
λ & μ-service architecture
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
Kafka
�18
producer
consumer consumer
partitionpartition partition partition
serveur serveur
consumer consumer
consumer consumerconsumer groupconsumer group
topic:partition topic:partition
zookeeper
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
Distribution/high availability
�19
broker 1
broker 2
shared fs/db
load balancing
High Availability!actif/actif!
actif/passif
broker 1
broker 2
broker 3
broker 4
P1
C1
Network of brokers!(a topology example)
failover
Zookeeper!(robust distribution) idempotency
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
Performance
�20
persistance mandatory ?
filesystemdatabase
distribution
yes
no
batchprefetch
ack
no serialization
headers
specialized queue
flow control
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
My subjective solutions
�21
• EBIA ALLIANCE = XEBIA + XEBIALABS + THIGA + UX REPUBLIC
Conclusion
�22
decoupling
semantic buffer
routingheartbeat
ordering
batch/prefetch exit strategy
idempotency flow control