Modern Systems Development

Creating Services at Scale

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it.

Through this work we have come to value:

Individuals and interactions over processes and toolsWorking software over comprehensive documentation

Customer collaboration over contract negotiationResponding to change over following a plan

That is, while there is value in the items onthe right, we value the items on the left more.

• Agility

• Cooperation

• Awareness

• Microservices

“Why does any of this matter?”

Rise of Determinism• Newton’s Principia

• Laws of physics are universal • The world is mechanistic, material and mathematical

• Laplace’s Demon • Careful measurement of current conditions • Understanding of the laws that drive the world • Sufficient computing power to find the “truth”

• Basis for much of science for the next 400 years

The Logistic Equation

f(xn+1)=rxn(1-xn)

• simple formula for estimating population

• r is the rate of population growth

• xn is the current population

First Look

Across Growth Rates

Expanding the View

Unexpected Change

Chaos

Still Deterministic!

Butterfly Effect

xpurple = .2

xgreen = .2000000001

Implications

• Laplace’s Demon may still be theoretically possible • Careful measurement of current conditions • Understanding of the laws that drive the world • Sufficient computing power to find the “truth”

• However it could also be risky to bet on it • Chaotic determinism • Sensitive dependence on measurements

• Be ready for the unexpected • The fastest to adapt usually wins. Speed matters.

Constant Learning

Observe

Orient

Decide

Act

What to do?

+ CONTRIBUTION

TRUST

TRUST

+

“Conflict without trustis politics.

Conflict with trust is a search for the truth.”

Patrick Lencioni

CONTRIBUTION

+

CONTRIBUTION

+

DEVELOPERS

CONTRIBUTION

+

DEVELOPERSOPERATORS USERS

CONTRIBUTION

DEVELOPERS

OPERATORS USERS

CONTRIBUTION

DEVELOPERS

OPERATORS USERS

Barriers to Cooperation

• Don’t know what to do (share knowledge)

• Don’t know how to do it (share culture)

• Can’t gauge progress (measure it)

• Can’t see who is responsible (automate it)

- Theory of Cooperation, John Willis and Damon Edwards

Theory of Cooperation

• Culture (of communication and discipline)

• Automation (superhuman scale and speed)

• Measurement (bringing verification and certainty)

• Sharing (to enable a rapidly growing community to the bar quickly)

- Theory of Cooperation, John Willis and Damon Edwards

Communication

• Single ‘ticket queue’

• ‘Always on’ chat room

• Daily stand ups

• Semi-weekly planning

• Every six month ‘hack week’

Automation and Architecture• CI/CD

• Provisioning

• Containers

• Microservices

• Scale out

• Backwards compatibility

Measure It

• Log Aggregation (Logstash, Loggly, Apache Kafka)

• Metrics (Graphite, Graphana)

• Monitoring (NewRelic, DataDog, Nagios)

• Alerting (PagerDuty)

Summary

• Building modern web services requires new approaches

• Speed. Speed. Speed.

• Constant evaluation from diverse inputs

• Always plan for change

• Culture must intentionally be built to support these goals

Final Thoughts

• Don’t try and do all the things

• Do something now vs. plan for perfect later

• Take time thinking about the ‘ideal’

• Take pragmatic steps everyday that are on the path