Finagle-Based Microservices at SoundCloud
Phil Calçado http://philcalcado.com
@pcalcado
Quick Report:
> 11 hours of audio uploaded every minute
~ 300 million people every month
Like everybody else around that time…
Like everybody else around that time…
Because ${reasons}, we decided to break the monolith. But how?
The Mothership
SoundCloud.com
The Mothership
Sounds ˝& Sets
Social Graph
Premium ˝Features Search
Activity Stream
API
Reminds me of something…
Monoliths exist when building within it is cheaper than the overhead associated with coordination of multiple components. !
!
We need an inexpensive services framework
Realistic options
Summary in a picture
Phase 1: Finatra
Good stuff
• Model is very intuitive to people coming from other languages
• Production-ready service in minutes
Main issues we had
• Not always on latest Finagle version
• Doesn't compose with Filters
• Internals aren’t very flexible
• twitter-server is too far away from our infra
Phase 2: Filters and Services
Where the gold was
We followed suit
(114 LOC)
Routes are defined in a single file
Each controller is a Service
Lots of filters
Today
• 100% Traffic hitting Finagle-based services
• Redis, memcached, MySQL drivers used in production
• About one month to ramp-up non-Scala developers
What is still hard
• Config flags implementation isn't very flexible, we use env vars not properties
• API surface space for Finagle is too large. What’s user land, what’s test, what’s internal?
• Bespoke scatter-gather code all over our apps
• Every organization builds their own Finagle routing
Up next
Containers for Continuous Delivery
Thrift/Protocol Buffers?
}JSON parsing?
phil calçado
www.soundcloud.com
http://philcalcado.com
@pcalcado
http://soundcloud.com/pcalcado
Top Related