Scaling Twitter

Alex & Britt(from Twitter)

Preconceived notions.

The story.

So, what happened?

What we did.

More boxes.

‣ Not so many then. But now...‣ Mongrels across 19 cores.‣ Message processing/delivery across 16 cores.‣ Jabber across 2 cores.‣ MySQL on one big 8 core box.‣ 16GB+ for memcache across a bunch of

machines.‣ 32 cores total.

That’s a lot.

Why?

‣ 200 - 300 connections per second.‣ Spiking at 800 connections per second.‣ We’ve done 11,000 connections per second.‣ MySQL has spiked to 2,400 queries per second.‣ Alexa says we have a whole mess o’ traffic, and

they’re not even seeing the API traffic.

memcache.

memcache.

memcache.

Cache the hell out of everything.

Denormalize.

find(:all, :conditions => "substring( truncate(id,0),-2,1) = #{@fugly_dist_idx}", :limit => 10)

Starling.

Community.

YOU CAN HAS QUESTIONS