AGNIAGNIMESSAGING. ON FIRE.MESSAGING. ON FIRE.

Rick Dillon

Yep. ⇧ That dude is a god.

WHY DO WE NEED MESSAGING?WHY DO WE NEED MESSAGING?Web apps should be fast, so...

do work asynchronously!And break out cross-cutting concerns into services

HOW DO SERVICES COMMUNICATE?HOW DO SERVICES COMMUNICATE?HTTP, of course.

But then, two problems emerge:

Traffic can be burstySystems go down for maintenance

So we thought:

“ Wouldn't it be great if all the requests coming inwere queued somewhere? ”

So we built Agni.

DESIGNDESIGNAgni is built around three principles:

SimplicityQueuingPriority

SIMPLICITYSIMPLICITYOne primitive: the queue. Avoid the complexities of AMQP.

ChannelsConsumersExchangesBindingsRoutesRoute keys

QUEUINGQUEUINGAsynchronous-by-design: just because no one is subscribed toa queue doesn't mean that messages can't wait for aconsumer.Durability is baked in.Flexible architecture: publishers don't need to know aboutwho is consuming published data.

PRIORITYPRIORITY'Asynchronous' doesn't mean 'not important'. Prioritize the stuff

that needs to be done fast.

The Messenger object is used for both publishing andsubscribing.

It's easy to make one.

require 'agni'm = Agni::Messenger.new('amqp://localhost')

Subscribing is a one-liner.

m.subscribe('test_queue') {|m,p| printf p}

So is publishing.

1.upto(100).each{|n| m.publish("test#{n}", 'test_queue')}

Sending prioritized messages adds only a single parameter.

1.upto(100000).each{|n| m.publish("test#{n}", 'test_queue', n%10)}

BEHIND THE SCENESBEHIND THE SCENESBased on RabbitMQ via Use ten queues, each with their own AMQP channelConfigure each channel with its own prefetch valueUse an in-memory Fibonacci heap to priortize the incomingprefetch streamsProcess messages off of the heap

ruby-amqp

LIMITATIONSLIMITATIONSEach message is only consumed once (by design)EventMachine-Fast (5,000 messages/sec)Supports ten levels of priority

STATUSSTATUSAgni has been in production use at Apartment List for close to

six months, and delivers millions of messages daily.

It is being made available publically for the first time this week.

Agni is Open Source, available under the BSD license. Check itout online at:

Try it out, file bugs, fork it, and send pull requests!

https://github.com/apartmentlist/agni