Case Study: Digging into the Technology Behind the Development of Digg
18
description
Transcript of Case Study: Digging into the Technology Behind the Development of Digg
What is digg?What is digg?
The idea - Oct. 04The idea - Oct. 04
Slashdot - User created content Del.icio.us - Bookmark / Share / Save Friendster - Social hookups, large
groups, no task
Slashdot - User created content Del.icio.us - Bookmark / Share / Save Friendster - Social hookups, large
groups, no task
$2,000 to launch$2,000 to launch
Developed a project spec - elance Open source technologies - LAMP Basic utilitarian design $99 a mo. hosting
Developed a project spec - elance Open source technologies - LAMP Basic utilitarian design $99 a mo. hosting
Feature DecisionsFeature Decisions
Stay away from me too features Simple and rewarding - one click
(digg, bury, #1) Used “ajax” where it made sense Tools to connect to other sites (blog
this, javascript widget) Experiment - spy, “cloud view”,
visualizations (stack, swarm)
Stay away from me too features Simple and rewarding - one click
(digg, bury, #1) Used “ajax” where it made sense Tools to connect to other sites (blog
this, javascript widget) Experiment - spy, “cloud view”,
visualizations (stack, swarm)
The “digg effect”The “digg effect”
Case Study: From Startup to Enterprise
Case Study: From Startup to Enterprise
One Server (Dec 2004-Mar 2005).
One Server (Dec 2004-Mar 2005).
Monitoring is “someone is working on the site most of the time.”
Standard LAMP architecture. Growth -> optimizing queries ->
denormalizing.
Monitoring is “someone is working on the site most of the time.”
Standard LAMP architecture. Growth -> optimizing queries ->
denormalizing.
GrowthGrowth
Paris Hilton Silverorange/Digg 2.0 Word of mouth, minimal advertising Steady growth with occasional insanity
Paris Hilton Silverorange/Digg 2.0 Word of mouth, minimal advertising Steady growth with occasional insanity
Seed Funding. Seed Funding.
Small Number of Servers (May 2004-Sep 2004) - web server, mysql master, mysql slave.
Ad-hoc monitoring. 1 Dedicated operations person. Silverorange (Digg gets a professional
design). Growth continues to outstrip hardware,
though now more constrained by developer resources.
Small Number of Servers (May 2004-Sep 2004) - web server, mysql master, mysql slave.
Ad-hoc monitoring. 1 Dedicated operations person. Silverorange (Digg gets a professional
design). Growth continues to outstrip hardware,
though now more constrained by developer resources.
Series A. Series A.
$100k on Servers. Spares! Everyone in the same place Operations Department (currently 4
people). Hire senior developers.
$100k on Servers. Spares! Everyone in the same place Operations Department (currently 4
people). Hire senior developers.
Description of Digg Architecture
Description of Digg Architecture
LAMP + memcached. Rough estimate of lines of code
(historical trend). I.e. Digg 1.0 = 5k lines of code. Digg 2.0 = 50k. Digg 3.0 = 200k (will fill in actual numbers).
Diagram (something like timeless’s “shards” diagram).
LAMP + memcached. Rough estimate of lines of code
(historical trend). I.e. Digg 1.0 = 5k lines of code. Digg 2.0 = 50k. Digg 3.0 = 200k (will fill in actual numbers).
Diagram (something like timeless’s “shards” diagram).
Monitoring (Nagios/Cacti). A couple of screenshots - Cacti
Graphs.
Monitoring (Nagios/Cacti). A couple of screenshots - Cacti
Graphs.
Development Process. Subversion, Bugzilla, Wiki.
Trac.1 Developer --> 3 --> 7. Talk a little about managing
that.
Development Process. Subversion, Bugzilla, Wiki.
Trac.1 Developer --> 3 --> 7. Talk a little about managing
that.
Office. Pictures? Moved in May 2006, already
outgrowing.
Office. Pictures? Moved in May 2006, already
outgrowing.
StatsStats
1M+ Registered users 10M+ Pages per day 1.2M+ Daily unique visitors 100+ Linux boxes (break
this down)
1M+ Registered users 10M+ Pages per day 1.2M+ Daily unique visitors 100+ Linux boxes (break
this down)
New Features/Announcements
New Features/Announcements
Future PlansFuture Plans
Party!!! Party!!!
