Agile execution
description
Transcript of Agile execution
![Page 1: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/1.jpg)
Lessons learned while re-platforming a complex system
Mik Quinlan
March 2014
AGILE EXECUTION
![Page 2: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/2.jpg)
Background – The Inventory System
![Page 3: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/3.jpg)
THE CHALLENGE: NEED TO MIGRATE OFF THE CURRENT INVENTORY SYSTEM…
![Page 4: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/4.jpg)
…TO A FUTURE ARCHITECTURE:
The Inventory Management Platform (IMP)
r
![Page 5: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/5.jpg)
CREATE A PLATFORM: IMP VERSION 1.0
![Page 6: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/6.jpg)
Agile Execution
![Page 7: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/7.jpg)
KEY PRINCIPLES…
• Iterative and incremental• Introspect as much as is possible without affecting progress
• How would you approach executing the following types of software projects?
• Green field development• New feature add / maintenance of a mature product• Complex legacy migration
![Page 8: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/8.jpg)
STRUCTURE: PHASED APPROACH• Huge uncertainty initially
• Sprint 0 doesn’t cut it
Borrowed from DSDM –divide the work up into smaller pieces:• Foundations
• Exploration
• Engineering
• Incremental Deployment
![Page 9: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/9.jpg)
STRUCTURE: CEREMONY AND SYSTEM COMPLEXITY
Be comfortable with uncertainty
Source: Information Age
![Page 10: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/10.jpg)
DE-RISKING: GO TO PRODUCTION EARLY…
• Early understanding of:– Application / technology in OUR
organisation• Prove it works throughout all
environments:– Validate features ASAP– Decreases number features in ‘ready for
release but not yet proven’ state• Allows focus on features• Easier if have good automated tests
Source: Growing Object Oriented Software Guided by Tests (Freeman et al)
![Page 11: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/11.jpg)
WALKING SKELETONS
“A Walking Skeleton is the implementation of the thinnest possible slice of real functionality that we can automatically build, deploy and test end-to-end”
- Alistair Cockburn in Crystal Clear: A Human Powered Methodology for Small Teams (2004)
![Page 12: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/12.jpg)
THE KEY IS IN THE DAY TO DAY EXECUTION…(1)• Psychology of an Engineer
• Team setup and iteration structure
• US – EU distributed team
• 1 week iterations
• Small stories (IMP alpha launch: mode size = 1, median = 3)
• Splitting stories – Research vs Implement
• You CAN make the stories smaller!
• Product and Technical decisions:
• ALWAYS know your Minimum Viable Product (MVP) / Minimum Marketable Product (MMP)
• Keep things simple
• Defer all decisions to the last responsible moment
• Take a lean approach wherever possible
• Product and Technical MUST work closely, each with an appreciation and understanding of the other
![Page 13: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/13.jpg)
THE KEY IS IN THE DAY TO DAY EXECUTION…(2)• Attention to our own Productivity:
• Sharing code – re-usable libraries when it makes sense
• Simplifying project setup via archetypes
• Building specialised tools, e.g.:
• Git branch/merge scripts
• Legacy -> new platform migration tools
• Specific category of story
• Roles vs doing:
• Everybody does QA!
• Mid iteration execution:
• What to do if a story is
• not going to be completed?
• is bigger than anticipated?
![Page 14: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/14.jpg)
THE KEY IS IN THE DAY TO DAY EXECUTION…(3)• Fix bugs immediately!
Source: http://pettermahlen.com/2011/04/
![Page 15: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/15.jpg)
THE KEY IS IN THE DAY TO DAY EXECUTION…(4)• Continuous “Inspect and Adapt”:
• Version 2.0 feed spec in current system
• Feed Quality Service component
• Shore up DR solution for publishing as interim primary publishing mechanism
• Support:
• Ownership of code all the way to production
![Page 16: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/16.jpg)
WHAT DOES IT MEAN TO HAVE SMALL STORIES?• Less to carry over
• PO and Lead must THINK about what is being delivered – less nebulous
• Acceptance Criteria MUST be detailed as well as succinct. E.g. 1 point story:
[DP] – Snapshot Service meets organisational standardsAs mycompany.com, I want to be able to deploy, configure and troubleshoot the snapshot service so that the it can integrate into the organisation’s operational environment.
Acceptance criteria• Logging configuration follows the organisation’s standard wrt access/error/console/syslog• Configurable parameters follows the organisation’s standard (properties file outside of WAR)• Packaging follows the organsiation’s standard (deployable WAR, probably nothing to do here)• Build/archive functionality follows company standard (build and archive uses CM scripts on Jenkins)• There is a health-check that checks that the service and its dependencies are healthy. • There is an index page with the following features:
• a link to the health check• a link to the configuration information• a link to the build version information• a form that allows users to submit JSON ingestion requests
• Relevant statistics are exposed via JMX
![Page 17: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/17.jpg)
THE LARGEST STORY ACCEPTED – 5 POINTS[Quality] Reliable VoltDB connectionsAs mycompany.com, I want services accessing Volt to have reliable connections so that they may continue to process on the event of a cluster node or network failure.
Acceptance criteria:
• Services that lose the connection to VoltDB will make reconnection attempts.
• A successful reconnection attempt means that the service resumes functioning correctly.
• Services can start up without a running VoltDB instance.
• Every lost VoltDB connection and each failed reconnection attempt is logged at ERROR level.
• Lack of VoltDB connectivity should lead to failing health checks.
• The reconnecting algorithm should be configurable using CM properties - so, for instance, if the chosen algorithm means that the service will make a certain number of reconnection attempts and then give up, the number of attempts to make and time interval between attempts should be CM parameters.
• The blackbox tests are updated so Volt does not need to be restarted on re-deployment of a component using a Volt client
Notes
It is possible that we will want to contribute this code back to VoltDB, as it may benefit us if the maintenance burden for the code is shifted to VoltDB. So the code should contain as few dependencies as possible.
Affected services: DI, OIDS, RDEP
![Page 18: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/18.jpg)
TECHNICAL EXECUTION• Break the complex problem down incrementally – v1 is:
• Walking skeletons
• Primary Scenario (aka Happy Path)
• Alternative scenarios
• Sharing solutions to problems across components:
• Dependency POM projects
• Restricted technology stack
• Solve once and share
• Having a cohesive view to the solution of a problem:
• Systems Architecture!
• “Considered haste”
• Think more, work less
![Page 19: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/19.jpg)
CONTINUOUS DELIVERY• Story not BA’d til it’s in Production
• Small stories mean:
• small changes to each release
• rollbacks are minor
• Helps determine what we should monitor when we move to automated Continuous Delivery:
• High risk vs low risk points
• Engineers own the application and this process so we better get it right!
![Page 20: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/20.jpg)
CONTINUOUS DELIVERY AND QUALITY• Two paths:
• Low risk features
• High risk features
• An extensive, trusted test suite means:
• Manual verification for high risk stories only
• Send everything else automatically…
![Page 21: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/21.jpg)
WHAT DID NOT WORK…• Prototype stories. Instead:
• Research -> stops design in planning
• Primary Scenario
• Alternative Scenarios
• Planning was too long – sometimes 2+ hours! Instead:
• Pre-planning sessions with Tech Lead, PO, QA rep
• Team divide up stories to task out
• External Support stories as part of the overall commitment and burndown. Instead:
• Have separate [External Support] type stories with 0 points
• Too many engineers on a story. Instead:
• Have story owners
![Page 22: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/22.jpg)
FUTURE GOALS: AGILE FLUENCY
Source: http://martinfowler.com/articles/agileFluency.html
![Page 23: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/23.jpg)
QUESTIONS?
![Page 24: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/24.jpg)
Advanced Agile Techniques
![Page 25: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/25.jpg)
WHEN THE ENGINEERING ORG IS READY…• What are we trying to do?
Go faster in a controlled manner
• With an eye towards…
Value Learning
• Supported by…
Continuous Delivery
• Delivered by multiple teams…
Scaling execution while minimising bottlenecks
and
Supporting each other
![Page 26: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/26.jpg)
VALUE LEARNING• Know what we’re going to measure – indicators of success
• Isolation of variables for a successful test
• Iterate quickly
![Page 27: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/27.jpg)
CONTINUOUS DELIVERY• Is a cultural change
• Do manually first
• Supported by Architecture:
• Minimum: Service based architecture
• Optional: Micro Services
• Supported by execution strategy:
• Everything is easily scriptable
• E.g. run embedded Tomcat / Jetty through command line
![Page 28: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/28.jpg)
FEATURE TOGGLES• Allows:
• Switch on/off of experimental features
• Delivery of unfinished features
• APIs must be backwards compatible for clients
![Page 29: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/29.jpg)
INTERNAL OPEN SOURCE MODEL
• Google model?
• Amazon model?
• What are YOUR pre-requisites?
![Page 30: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/30.jpg)
Engineering Excellence
![Page 31: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/31.jpg)
UNNECESSARY COMPLEXITY
• Akin to Lean, but approached at a different angle
• Five types:
• Product
• Execution
• Architecture / Design
• Technology
• Code
“It’s just not necessary to do extraordinary things to get extraordinary results”
Warren Buffett
![Page 32: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/32.jpg)
THE PRAGMATIC MANIFESTO
Iterative and incremental dark launch over expensive up front performance testing
Features that are immediately required over features that may be required
Just enough design up front over diving straight in
Fact based product enhancement over theory
![Page 33: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/33.jpg)
SOME PRINCIPLES TO CONSIDER…
Lean is better than Perfect
Simple is better than Complex
Think More, Work Less
![Page 34: Agile execution](https://reader036.fdocuments.in/reader036/viewer/2022062315/568162b1550346895dd33a11/html5/thumbnails/34.jpg)
THANK YOU
p. 34
Blog: www.mikquinlan.com
Follow me: @MikQuinlan
Connect: www.linkedin.com/in/mikquinlan