RabbitMQ & Hutch
-
Upload
peter-hamilton -
Category
Technology
-
view
1.357 -
download
1
description
Transcript of RabbitMQ & Hutch
RabbitMQ & HutchPete Hamilton - GoCardless
About Me
• Developer at GoCardless
• CompSci Student at ICL
• Github: https://github.com/petehamilton
• Blog: pete-hamilton.co.uk
• Twitter: @peterejhamilton
SOA CommunicationA (very) brief overview
Awesome Service
SOA CommunicationA (very) brief overview
Awesomer Services
SOA CommunicationA (very) brief overview
Awesomer Services
SOA CommunicationA (very) brief overview
?Client - Server?
Message Queues?
RabbitMQ
RabbitMQWhy?
• Nice Learning Curve
• Reliable
• Active Community
• Polyglot
• Scalable
• Proven itself in battle production
RabbitMQWhy?
• Payments are pretty serious stuff
• Async = Potential problems
• Network Failure?
• Consumer Crashes?
• Duplication/redelivery?
• RabbitMQ helps with confirms, acknowledgements, mandatory flags
RabbitMQ Routing
P X
C1
C2
Q1
Q2
Fanout Exchange
RabbitMQ Routing
P X
C1
C2
Q1
Q2
Direct Exchange
RabbitMQ Routing
P X
C1
C2
type=direct
fireflystar_trek
big_bang_theory
Q1
Q2
Topic Exchange
RabbitMQ Routing
P X
C1
C2
type=topic
tv.scifi.*
tv.comedy.*
C3
tv.#
Q1
Q2
Q3Sample Message Routing Keys:- tv.comedy.big_bang_theory- tv.scifi.firefly- tv.scifi.star_trek
RabbitMQ
• Very Flexible, but...
• Which data structure do you use?
• How do you serialise your data?
• Which fields do you require in your messages?
• How should messages be routed? fan out? direct? topically?
• How do new nodes set themselves up?
• What form should “Boiler Plate” code take?
A system for processing RabbitMQ messages in Ruby
• Our Conventions (Warning: Opinions!)
• Topic Exchange Routing
• Serialisation w/ JSON
• Message IDs and Timestamps
• Simple to use
• Configurable
• Specific ruby files or full Rails apps
Publisher
Consumer
https://github.com/gocardless/hutch
RabbitMQScaling
• Good guide to clustering online
• Potential for issues with multiple nodes
RabbitMQScaling
• Good guide to clustering online
• Potential for issues with multiple nodes
S S
M
C1
RabbitMQScaling
• Good guide to clustering online
• Potential for issues with multiple nodes
S S
M
x C1
RabbitMQScaling
• Good guide to clustering online
• Potential for issues with multiple nodes
S S
M
x C1
RabbitMQScaling
• Good guide to clustering online
• Potential for issues with multiple nodes
S S
M
C1
RabbitMQScaling
• Good guide to clustering online
• Potential for issues with multiple nodes
S S
M
C1
x
RabbitMQScaling
• Good guide to clustering online
• Potential for issues with multiple nodes
M S
M
C1
x
RabbitMQScaling
• Good guide to clustering online
• Potential for issues with multiple nodes
• Clients exist for load testing, but...
• We wanted ++ parallelism
• Built “AMQPC”
AMQPC
• Written in Go
• Producer & Consumer modes
• Traffic Modes
• One off burst
• Intermittent burst
• Consistent load
AMQPC
https://github.com/gocardless/amqpc
Useful Links
• Hutch - www.github.com/gocardless/hutch
• AMQPC - www.github.com/gocardless/amqpc
• RabbitMQ - http://www.rabbitmq.com/getstarted.html
Questions?