Messaging for Modern Applications
-
Upload
tom-mccuch -
Category
Technology
-
view
2.377 -
download
1
description
Transcript of Messaging for Modern Applications
Taking the next step shouldn’t be scary …
1
Messaging for Modern Applications
Tom McCuch Staff Systems Engineer, VMware Email: [email protected] Twitter: @spring_tom Source Code for these demos: http://github.com/spring-tom/spring-integration-samples (applications/café)
© 2011 SpringOne 2GX. All rights reserved. Do not distribute without permission.
Session Agenda
• Modern Applications – Trends and Impact on Messaging – Impact on Application Architecture
• Spring Integration – Messaging Use Cases – Messaging DSL for Spring Applications
• AMQP – Why AMQP? – AMQP Architecture
• RabbitMQ – Why RabbitMQ? – RabbitMQ Architecture
Modern Application Trends
• Written with frameworks – Developer productivity and innovation
• New Application Types – Mobile, SaaS, Social – Apps released early and often
• Data intensive – Emerging requirements: elasticity, multi-cloud – Web orientation drives exponential data
volumes
• Deployed on virtual and cloud infrastructure – Trending toward hybrid (public and private)
infrastructure
Modern Applications need Modern Messaging
Modern approaches to messaging embrace the unpredictable, dynamic,
and transient nature of the web
Traditional approaches to messaging imply predictable, static interaction
between well-known end-points
Synchronous Architecture is Brittle
Place Order
Fulfill Order
Manage Inventory
Schedule Delivery
If any component is unavailable within this synchronous flow, all in-flight messages will be affected, regardless of what
component they are currently in -- and since state is maintained across the components, the system can be complex to reset
X X
X X
X X
X X
X
An exception occurred,
please resend
Asynchronous Architecture Rules the Web
In this asynchronous flow: • all state is kept in the simple messages being passed • each stateless component interacts only with the broker cloud • if a component is lost, only the in-flight messages within that
component must be re-delivered • components can be quickly provisioned on-demand
I have an Order
I can fulfill part of
that Order
I’ll schedule delivery of that order
I can fulfill part of
that Order Your order will be delivered next Tuesday
The status of your order is
…
How is Messaging Used within Modern Apps?
8
§ Event I need to know when to do something
§ Fragment I need to only give you a piece of this data
§ Route Control who gets which message, without changing sender or receiver
§ Batch Producer and consumer can run at independent times
§ Publish Tell everyone who wants to know about this
§ Modularity Deploy, Scale, and Version a distributed system
Event
I need to know when to do something …
9
<service-activator input-channel="hotDrinks” ref="barista" method="prepareHotDrink" output-channel="preparedDrinks” />
An Event-Driven Consumer is one that is automatically handed messages as they’re delivered on the channel. The receiver acts like the message delivery
is an event that triggers the receiver into action.
Fragment
I need to only give you this piece of data …
10
<int:splitter input-channel="orders” expression="payload.items” output-channel="drinks” />
Use a Splitter to break out the composite message into a series of individual messages, each containing data related to one item.
Route
Control who gets what message
11
<int:router input-channel="drinks” expression="payload.iced ? 'coldDrinks' : 'hotDrinks'” />
A Router consumes a Message from one Channel and republishes it to a different Channel channel depending on a set of conditions.
Batch
Producer and Consumer can run at independent times
12
<int:channel id="preparedDrinks"/> <int:aggregator input-channel="preparedDrinks" method="prepareDelivery" output-channel="deliveries” > <bean class="org.sf.integration.samples.cafe.xml.Waiter"/> </int:aggregator>
Use a stateful filter, an Aggregator, to collect and store individual messages until a complete set of related messages has been received. Then, the Aggregator
publishes a single message distilled from the individual messages.
Publish
Tell everyone who wants to know about this
13
<int-event:outbound-channel-adapter channel="eventChannel"/>
All messages sent to the 'eventChannel' channel will be published as ApplicationEvents to any relevant ApplicationListener instances that are
registered within the same Spring ApplicationContext.
Modularity
Deploy, Scale, Version a distributed system
14
Distributed <int-amqp:outbound-gateway id="coldDrinksBarista" request-channel="coldDrinks” reply-channel="preparedDrinks” routing-key="ordered.drinks.cold” /> <int-amqp:outbound-gateway id="hotDrinksBarista" request-channel="hotDrinks” reply-channel="preparedDrinks” routing-key="ordered.drinks.hot” />
Spring Integration doesn’t force you to finalize decisions about deployment before testing begins. Individual modules can be versioned, deployed, and scaled
independently at any time through configuration, not code.
Local <service-activator input-channel="coldDrinks” ref="barista” method="prepareColdDrink” output-channel="preparedDrinks” /> <service-activator input-channel="hotDrinks” ref="barista" method="prepareHotDrink" output-channel="preparedDrinks” />
15
Demo – Introducing the Spring Integration Café Demo
Advanced Message Queuing Protocol
16
SMTP AMQP
HTTP IIOP
unreliable reliable
async
sync
“Like email, but you can send money over it” Open, Ubiquitous, and
Adaptable
Why AMQP?
17
A Protocol, not an API • A defined set of
messaging capabilities called the AMQ model
• A network wire-level protocol, AMQP
On commodity hardware • 10-‐25 thousand messages
per second is rou>ne * • The NIC is usually the
boDleneck
* Non-‐persistent messages
Why AMQP?
18
AMQP security • Brokers support separate
virtual hosts • Three levels of permissions • AMQP over SSL is supported
Designed to scale horizontally • Dozens of clustered brokers
are common • JPMorgan sends 1 billion
AMQP messages per day
AMQP Architecture
19
new.order
queue
M3 M1 M2 new.order binding
new.order routing key
AMQP Architecture
20
café NA deliveries
queue
café deliveries
queue M3 M1 M2
M3 M1 M2
café WW deliveries
queue M3 M1 M2
AMQP Architecture
21
hot_drinks
queue
cold_drinks
queue
1 2
2
3
1
drink.cold
Message Routing Keys: 1. drink.hot 2. drink.cold 3. drink.warm
all_drinks
queue
22
Demo – Scaling the Café Demo with AMQP in Cloud Foundry
Legacy Middleware not suited for Modern Apps
23
Costly to Scale Up Complex to Scale Out
Existing middleware impacts agility by tightly coupling applications to underlying application
server and other middleware components
Apps & Middleware
Licensed by Peak Physical Servers
RabbitMQ – Messaging that Just Works
design goal: no bloat - “it just works” mission: “be everywhere”
AMQP LEADER
11st:4EK-$?+%([:\(!"##$%(>,:+/484?$,.(@%0`
!"##$%&'($/%,?-"%,.(*$%+(C4.%(4F(
%+,(C"$/(0,3,84E,-(E8"i4-C.
!"##$%(0$.%-$#G%,0(#K(C4.%(@$/G9(%KE,(E8"i4-C.
A,(4^,-(E":T"?,.(F4-(C"/K(4%+,-(=Q(E8"i4-C.
!"##$%&'`j<>("/0(&$:-4.4R(2g7(E"-%/,-.+$E
;84G0("/0(WG$80H!G/H&"/"?,(*$88(0-$3,(C4-,(."8,.
&"-T,%(,/%-Ke("??-,..$3,(m4E,/](0$.%-$#G64/(u(EG88(F4-(."8,.("/0(:4CC,-:$"8(=<&
Monday, 17 August 2009
Incredible Community Lowers Your Cost of Use
Enterprise Users, such as this Tier-1 Telco
Enterprise Features, such as Clustering
27
RMQ Cluster1
RMQ2
RMQ1 queue-1
queue-2
RMQ3 queue-3
1
3
2
1. queue-1 2. queue-3 3. queue-2
Enterprise Features, such as Fault Tolerance
ESX1
RMQ1 hot-drinks cold-drinks
slave master
VM2
RMQ2 hot-drinks cold-drinks cold-drinks
VM1
RMQ1 hot-drinks cold-drinks
Our New Mission: “It’s Just There”
Downloadable Micro Cloud
Cloud Foundry vFabric
Service Provider
Private Cloud
vSphere
Hybrid Cloud
vSphere
Cloud Foundry vFabric Cloud Foundry vFabric
Ruby App
Java App
1. Build your messaging application
2. Deploy it, Scale it using Messaging Service
3. Move on or off premise as needed
RabbitMQ Hybrid Cloud Deployment
30
new.order
queue M3 M1 M2
cafe-direct
RabbitMQ Linux / VMware Fusion
cafe-remote
RabbitMQ Linux / Amazon EC2
Federated
amqp
new.order
31
Demo – Deploying the Café Demo with RabbitMQ in a Hybrid Cloud
Summary • Modern Applications
– Use frameworks, less emphasis on servers – The explosion of connected mobile devices
requires a new solution for messaging
• Spring Integration – A DSL for introducing messaging abstractions
into your spring application – Externalizes messaging concepts from code
• AMQP – A TCP/IP protocol not an API – Fast, Reliable, Open, Secure, Scalable
• RabbitMQ – AMQP Leader – Backed by VMware
33
Thank You!