Fast and Good: Alternate Approaches to Quality at Etsy - STPCon fall 2011
54
Noah Sussman STPCon Fall 2011, Dallas TX #STPCon Alternative Approaches to Quality at Etsy fast and good
-
Upload
noah-sussman -
Category
Technology
-
view
2.761 -
download
0
description
Transcript of Fast and Good: Alternate Approaches to Quality at Etsy - STPCon fall 2011
Noah Sussman STPCon Fall 2011, Dallas TX
#STPCon
Alternative Approaches to Quality at Etsyfast and good
Our mission is to empower people to change the way the global economy works. We see a world in which very-very small businesses have much-much more sway in shaping the economy, local living economies are thriving everywhere, and people value authorship and provenance as much as price and convenience. We are bringing heart to commerce and making the world more fair, more sustainable, and more fun.
September 2011: 489 Web deployments
110 people committed to trunk 4,519 changesets in all 9.2 changesets average per production deployment
September 2011: 489 Web deployments
80 people deployed to production 20 or fewer committers in each deployment 5.7 committers average per deployment
10 million registered members25 million unique visitors per month800,000 active sellers150 different countries10 million listed items$314 million in sales in 2010
In 2010, MTTR was 6.4 minutes for severity one outages
Six change-related outages in 2010Out of thousands of production deployments
“All existing revision control systems were built by people who build installed software.”
~Paul Hammond
All canonical software life cycles were designed by people who build installed software
“In the online world… there is one consumer of the software: you. The users are consumers of the service built atop the software.”
~ Theo Schlossnagle
Optimize for recovery rather than focusing on preventing failure.
Failure is inevitable.
Default to open access
No blame
Use your own judgement
“an airport without an air traffic controller”Chad Dickerson
faster deploys = smaller changesets = fewer change-related outages
Self-serve deployment
bleeding edge code,production data and services
princess
Shared development environment
Single dev databaseEveryone commits to trunk
low-ceremony process
A test is a design document first
Many small deployments, many small tests
Beyond developer testing1. Test planning2. GUI-driven regression testing3. Performance testing4. Security / Penetration testing5. Nagios
Make it easy to write and run tests
Make it painless to use fixtures and mocks
Most bugs, most of the time, are easily nailed given even an incomplete but suggestive characterization of their error conditions at source-code level.
~ESR, “How Many Eyeballs Tame Complexity,” 1997
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
~ Brian Kernighan
Test to address technical debt
Keep the feedback loop short
Graph or it didn’t happenAt Etsy we monitor over 40,000 real-time metrics
Business Metrics
Application Metrics
System Metrics
We hire our users
Tests are one of the tools we use to detect and prevent failure
There will always be bugs in production
Unexpected failures tend to be catastrophic
Anomaly detection is hard
Just ship
How do you deploy large features?
Releasing a feature is decoupled from deploying code
if ($cfg["new_search"]) {// new hotness$resp = search_solr();
} else {// old busted$resp = search_grep();
}
$cfg = array( 'checkout' => true, 'homepage' => true, 'profiles' => true, 'new_search' => false,);
Dark launchesAdmin-only rolloutsA/B campaignsetsy.com/prototypesWe are always testing
A culture of responsibility
A community of artisans
You are a participant observer
etsy.com/careers
photo creditspenguins: paulmcdee http://www.flickr.com/photos/paulmcdee/2806715569aerial photograph of snowy road: K. Leidorf http://leidorf.blogspot.com/search/label/20D03855death in the mirror: Richard Avedon for the New YorkerWomen at work on C-47 Douglas cargo transport http://www.flickr.com/photos/library_of_congress/2179137415/in/photostream/Woman aircraft worker http://www.flickr.com/photos/library_of_congress/2179930812/cool bike with wooden handlebars: Bertelli http://www.bertellibici.com/product.php?works_id=112clover keys: Brian Cerveny http://www.flickr.com/photos/briancerveny/48889960/in/photostream/Etsy office monitor wall and Etsy bike rack: Mashable's Cubicle Spy http://mashable.com/2010/10/13/etsy-office-picshorse fly: Lukjonis http://www.flickr.com/photos/38628972@N05/5884111163/in/photostream/Gandalf vs balrog: the brothers HildebrandtJust Ship t-shirt: Kellan http://www.etsy.com/shop/kellanem/soldbarn raising: dogpose http://www.flickr.com/photos/8371074@N05/2865798475/model train coupling: David E. Smith http://www.flickr.com/photos/david_e_smith/3566697122/sizes/l/in/photostream/
If I used your image and didn't credit you, it's because I couldn't figure out who you were -- email me and I will gladly credit you. If you object to my using an image to which you own the rights, please contact me and I will remove that image from the deck.
