Rails in the Large - Neal Ford

68
ThoughtWorks ThoughtWorks NEAL FORD software architect / meme wrangler ThoughtWorks [email protected] 3003 Summit Boulevard, Atlanta, GA 30319 www.nealford.com www.thoughtworks.com blog: memeagora.blogspot.com twitter: neal4d PAUL GROSS software developer / consultant ThoughtWorks [email protected] 200 E. Randolph St, 25th Floor, Chicago, IL 60601-6501 [email protected] www.pgrs.net www.thoughtworks.com Rails in the Large: Building the Biggest (Enterprise) Rails Application in the World

description

Neal Ford - Agile East 2011

Transcript of Rails in the Large - Neal Ford

Page 1: Rails in the Large - Neal Ford

ThoughtWorksThoughtWorks

NEAL FORD software architect / meme wrangler

ThoughtWorks

[email protected] 3003 Summit Boulevard, Atlanta, GA 30319

www.nealford.com

www.thoughtworks.com

blog: memeagora.blogspot.com

twitter: neal4d

PAUL GROSS software developer / consultant

ThoughtWorks

[email protected] 200 E. Randolph St, 25th Floor, Chicago, IL 60601-6501

[email protected]

www.pgrs.net

www.thoughtworks.com

Rails in the Large:Building the Biggest (Enterprise)

Rails Application in the World

Page 2: Rails in the Large - Neal Ford

coming R S N

Page 3: Rails in the Large - Neal Ford

time & space

metrics

feedback loops

automation

communication non-

intuitivity

demonstration

Page 4: Rails in the Large - Neal Ford

what OVE.com does

Page 5: Rails in the Large - Neal Ford
Page 6: Rails in the Large - Neal Ford
Page 7: Rails in the Large - Neal Ford

the pursuit

Go for the one that’ll beat the one that’ll beat the one you last did

Page 8: Rails in the Large - Neal Ford

.NET Rails

Page 9: Rails in the Large - Neal Ford
Page 10: Rails in the Large - Neal Ford

Business AnalystProject manager

DeveloperTech Lead

quick start: october 2006

Page 11: Rails in the Large - Neal Ford

Added one pair every 2 weeks

Started with 2 pairs

8 or 9 pairs by July

inception: Jan 17, 2007

Page 12: Rails in the Large - Neal Ford

6 quality assurance

11 pairs of developers 8 business analysts

client principleproject manager

now

iteration manager

Page 13: Rails in the Large - Neal Ford

spikes are your friends!

technology isn’t as important as responsiveness to business needs

don’t try to convince too early

demonstration over arguments

lessons learned

Page 14: Rails in the Large - Neal Ford

infrastructure

Rock is for Rookies: males have a

tendency to lead with Rock on their

opening throw

Page 15: Rails in the Large - Neal Ford

pairing workstations

XServe (Selenium Grid)

physical infrastructure

BA

standalone QA

integrated QA

UAT (sneak peak)

Page 16: Rails in the Large - Neal Ford

deployment stack

10 web boxes 2 image servers

background server memcache

4 database servers

Page 17: Rails in the Large - Neal Ford

technical statsfeedback

loops

Page 18: Rails in the Large - Neal Ford

environment

Page 19: Rails in the Large - Neal Ford
Page 20: Rails in the Large - Neal Ford

Mac OS X rocks

scale infrastructure opportunistically...

...but don’t wait too long

have fun

lessons learned

Page 21: Rails in the Large - Neal Ford

testing

Scissors on First:play scissors as your

opening move against a more experienced player

Page 22: Rails in the Large - Neal Ford

disconnected unit tests

UnitRecord and the evolution of unit tests that don’t hit the database

http://github.com/dan-manges/unit-record

Page 23: Rails in the Large - Neal Ford

unit tests

Page 24: Rails in the Large - Neal Ford

the rule:

unit tests don’t hit the database

mock everything

Page 25: Rails in the Large - Neal Ford

functional tests

Page 26: Rails in the Large - Neal Ford

no mocking allowed in functional tests

tests that hit the database are slooooow

Page 27: Rails in the Large - Neal Ford

DeepTest

http://github.com/qxjit/deep-test

feedback loops

Page 28: Rails in the Large - Neal Ford
Page 29: Rails in the Large - Neal Ford

DistributedDeepTesttime & space

metrics

automation

demonstrationnon-

intuitivity

Page 30: Rails in the Large - Neal Ford

Selenium grid

time & space

Page 31: Rails in the Large - Neal Ford

write smart tests

fight the battle to keep tests fast

invent stuff if you have to

scale development infrastructure just like production infrastructure

lessons learned

Page 32: Rails in the Large - Neal Ford

knowledge transfer

Paper is the least obvious of opening moves.

Page 33: Rails in the Large - Neal Ford

project Mingle on the wall

Page 34: Rails in the Large - Neal Ford

cc_board

http://github.com/qxjit/cc_board/

Page 35: Rails in the Large - Neal Ford

play theme song upon successful checkin

play a song when a build breaks

communication

Page 36: Rails in the Large - Neal Ford

jukebox.rb

http://subversion.hammersforge.com/jukebox.rb/trunk/

currently in alpha

Page 37: Rails in the Large - Neal Ford

pairing stations

adium

no email

Page 38: Rails in the Large - Neal Ford

internal Jabber server chat rooms

devsBAsQAs

shared buddy list

Page 39: Rails in the Large - Neal Ford

automatically set pair name

adium

Mingle card (upon commit)

Page 40: Rails in the Large - Neal Ford

co-location rocks

software is more about communication than technology

use information radiators

have fun

pairing really rocks

lessons learned

communication

feedback loops

Page 41: Rails in the Large - Neal Ford

automate everything

When playing with someone who is not

experienced at the RPS, look out for double runs or, in other words, the

same throw twice.

Page 42: Rails in the Large - Neal Ford

1-click deploy to any environment

using cc.rb as easy deployment tool

Page 43: Rails in the Large - Neal Ford

verification (language keys)

run all unit tests

run all functional tests

commit

rake commit

http://github.com/pgr0ss/rake_commit_tasks

Page 44: Rails in the Large - Neal Ford

canonical pairing station maintenance

cap pairing_stations

Page 45: Rails in the Large - Neal Ford

radmindhttp://rsug.itd.umich.edu/software/radmind/

Page 46: Rails in the Large - Neal Ford

strict rules for advanced language features

Tell your opponent what you are going to throw and then actually throw what you said.

Page 47: Rails in the Large - Neal Ford

background processing

Try playing the throw that would have lost to your opponents last throw

Page 48: Rails in the Large - Neal Ford

evolution of

asynchronous messaging

Page 49: Rails in the Large - Neal Ford

progress bars & async upload

backgrounDrbhttp://backgroundrb.rubyforge.org/

Page 50: Rails in the Large - Neal Ford

3 kinds

Page 51: Rails in the Large - Neal Ford

use backgroundrb for simple message queue backed by database

Page 52: Rails in the Large - Neal Ford

switch to a real messaging queue

(Starling)

Page 53: Rails in the Large - Neal Ford

project time

don’t know what we don’t know

“buy the fanciest one we can” (just in case)

Page 54: Rails in the Large - Neal Ford

project time

technical debt

when youadd it

when youstart using it

Page 55: Rails in the Large - Neal Ford

project time

don’t know what we don’t know

“buy the fanciest one we can” (just in case)

pay $$$ for technical debt…

…that you may never justify

Page 56: Rails in the Large - Neal Ford

trying to predict the future leads to over-

engineering

Page 57: Rails in the Large - Neal Ford

don’t use databases as message queues (for too long anyway)

avoid anticipatory design

gradually add complexity

DBA’s can sometimes get grumpy

lessons learned

Page 58: Rails in the Large - Neal Ford

performance & optimization

When all else fails, go with paper: Statistically, in competition play, it has been observed that scissors is thrown the least often.

Page 59: Rails in the Large - Neal Ford

not that many page views...

...really complex pages!

Page 60: Rails in the Large - Neal Ford

daily web trends

Page 61: Rails in the Large - Neal Ford

monthly web trends

Page 62: Rails in the Large - Neal Ford

why all the rochambeau stuff?

Page 63: Rails in the Large - Neal Ford

view builds are slow =>

separate cc.rb build =>

1 pair assigned as view masters

view builds are fragile =>

Page 64: Rails in the Large - Neal Ford

worst ...job ...ever

Page 65: Rails in the Large - Neal Ford

today’s view master assigned by yesterday’s...

...or play RPS

Page 66: Rails in the Large - Neal Ford

http://www.worldrps.com/

Page 67: Rails in the Large - Neal Ford

ThoughtWorks

?’sThis work is licensed under the Creative Commons

Attribution-Share Alike 3.0 License.

http://creativecommons.org/licenses/by-sa/3.0/us/

NEAL FORD software architect / meme wrangler

ThoughtWorks

[email protected] 3003 Summit Boulevard, Atlanta, GA 30319

www.nealford.com

www.thoughtworks.com

blog: memeagora.blogspot.com

twitter: neal4d

PAUL GROSS software developer / consultant

ThoughtWorks

[email protected] 200 E. Randolph St, 25th Floor, Chicago, IL 60601-6501

[email protected]

www.pgrs.net

www.thoughtworks.com

Page 68: Rails in the Large - Neal Ford

time & space

metrics

feedback loops

automation

communication non-

intuitivity

demonstration