Waterloo Agile Lean P2P Group

Creating the Technical Plumbing to Support Agile Tools and Techniques from the Muddy Trenches

Leon KehlNovember 22, 2011Waterloo Ontario

Agenda

A Tale of Two Agilites

How Agile are you and your company?

The Philosophy and Practise of Systems Design

Peer to Peer Discussion

Examples from the Trenches

Dialectic Peer to Peer Discussion Revisited

Q/A

How are Agile are you?

How would you rate your Agile knowledge (1-10)?

How would you rate your companies Agile practices (1-10)?

Agile Top 10 Survey

Survey Says: Agile Works in Practice. Sept 2006 – Dr. Dobbs Journal

Agile Top 10 Survey

Survey Says: Agile Works in Practice. Sept 2006 – Dr. Dobbs Journal

Trivial Pursuit

Who knows what happened to the original C3 project at Chrysler?

Trivial Pursuit

The plan was to roll out the system to different payroll 'populations' in stages, but C3 never managed to make another release despite two more years' development. The C3 system only ever paid 10,000 people (target was 85,000)

Chrysler was bought out by Daimler-Benz in 1998. DaimlerChrysler stopped the C3 project on 1 February 2000.

Frank Gerhardt, a manager at the company, announced to the XP conference in 2000 that DaimlerChrysler had de facto banned XP after shutting down C3; however, some time later DaimlerChrysler resumed the use of XP.

The Knowing – Doing Gap

• Each year $60 Billion spent on training

• TQM Training at five companies. Four weren’t using it and last limited use only

• At Toyota “Many plants have put in an andon cord .. to stop the assembly line. A 5-year old can pull the cord. But it takes a lot of effort to drive the right philosophies”

Causes

• Knowing “What to Do is Not Enough

• When Talk Substitutes for Action

• When Memory Is a Substitute for Thinking

• When Fear Prevents Acting on Knowledge

• When Measurement Obstructs Good Judgement

• When Internal Competition Turns Friends into Enemies

Computers in Context: The Philosophy and Practice of System Design

They examine key assumptions about the role of software developers, and their relationship to culture and work in a way which touches everyday practice and which can subtly transform it

Thinking

• Hard Systems Thinking

• Soft Systems Thinking

• Dialectic Systems Thinking

Hard Systems Thinkingaka Waterfall

• A hierarchically organized set of elements

• Everything that can be said in principle is contained in all elements and properties

• Functional Analysis from the Top Down

• Try to create a “true” representation of the world

• “when the map and the terrain disagree the error is in the terrain”

Soft Systems Thinkingaka Agile

• Systems are in our minds, they are perspectives that we change and improve by being confronted with other perspectives

• Design is seen as learning

• Perspectives are communicated, shared and negotiated to create shared interest in change

• Requires substantial experience and professionalism by practitioners

Manifesto for Agile Software Development

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

Dialectic Systems Thinking

• Originates with Socrates, used by Marx, Nietzsche and Freud

• Face a network of dynamically changing contradictions

• Resolving some conflicts requires changing perceptions but also resources, structures, systems and power relationships

• The world rather than people’s perceptions of it, is our primary source for learning

• World suffused with conflicts, dark forces, frightening jungle of different interests, struggles for power

Examples of Contradictions

• Developers are told to write unit tests because they will be more productive, but believe they don’t add value and always do them after development

• Customers will report problems and developers will say they are impossible

• Management wants to become Agile without changing legacy software, developers or development practices

• Development is told they are too slow but can’t make wholesale changes to speed things up

• Code quality is supposed to be a priority but developers can go to anyone for code reviews

• Customers and management complain about bugs, but automated unit tests aren’t a priority

Peer to Peer Support

• Split into pairs

• Explain to your partner a current or recent situation where you are seeing a knowing/doing gap or a contradiction relating to Agile best practices.

• Take about five minutes and then switch.

• As the listener, ask questions try to understand the network of contradictions present in the situation

Peer to Peer Support

• Did the stories you heard support these ideas of a knowing/doing gap and the idea of dialectic thinking?

• Do you have any observations you want to share?

Dialectic Systems Thinking Creating an Intervention

• Contradictions are opportunities for intervention

• The challenge is to understand and change established traditions in the user organization as well as in the project group and in the development organization as a whole

• The interventionist paradigm suggests we actively participate in organizational games

• Remain a problem solver relying on both construction and evolution approaches

Enough PhilosophyLet’s Get Practical

• So you want to change the system?

• What tools and techniques can you use?

• What ideally are you trying to achieve?

Enough PhilosophyLet’s Get Practical

• So you want to change the system?

• What tools and techniques can you use?

• What ideally are you trying to achieve?

• Well it depends. Dialectic Systems thinking would imply each situation is unique and presents its own contradictions to exploit

Agile? At Google

Google is an exceptionally disciplined company, from a software-engineering perspective. They take things like unit testing, design documents and code reviews more seriously than any other company I've even heard about. They work hard to keep their house in order at all times, and there are strict rules and guidelines in place that prevent engineers and teams from doing things their own way. The result: the whole code base looks the same, so switching teams and sharing code are both far easier than they are at other places.

And engineers need great tools, of course, so Google hires great people to build their tools, and they encourage engineers (using incentives) to pitch in on tools work whenever they have an inclination in that direction. The result: Google has great tools, world-class tools, and they just keep getting better.

Case #1

• 20 year old legacy system, ported from Unix paired with another legacy system for another purpose

• Supported by developers with long term service, but weak development skills

• “Unit tests” consisted of a 100 page Word document that got added to when functionality changed

• Product management wanted to support another higher speed hardware device for new markets

• Any testing required minutes to hours of setup and physical devices

Solution #1

• Worked on system to develop understanding of business space, software, skillsets

• Developed proposal to rewrite from the ground up utilizing similar technology but with Agile and working prototype in two months. Chutzpah required

• Then got green light for full development

• Used simulators, automated unit tests, continuous integration. Used younger more “flexible” resources.

• Didn’t attempt to understand all existing code, only what it was intended to do

• Was able to accommodate new device into design

Obstacles #1Considerable pressure to use existing components, algorithms,

resources because that was what was always done:

• Ran as silently as possible, avoided seeking advice from existing “experts” or using “experienced” resources

• Used data from the field, run through previous and new algorithms to demonstrate poor performance and in fact crashes from existing libraries

• Measured performance of standard logging to demonstrate inefficiencies

• Used continuous integration, automated unit tests, automated integration tests, code coverage, automated document generation etc. to minimize potential criticisms and to solidify management approval

Case #2

• System was recently developed, using Agile with automated unit tests, code coverage and nightly builds

• A number of team members left, replacements weren't as concerned about unit tests saw them as overhead

• Build often broken, automated unit tests were allowed to fail without fixing, time to run increased significantly to almost an hour

• Little management support for improvements, focus on adding features, bug fixes

Solution #2

• Got buy-in from receptive team members for continuous integration

• Split unit tests into majority which could run quickly and those that couldn't

• Grabbed a spare PC to act as a integration server

• Fixed all the unit tests and got them passing

• Emails and public shaming kept unit tests passing

Case #3

Older thick client application subsystem, responsible for fetching, caching and displaying large quantities from server

Although other subsystems had automated unit tests for a number of years, no real tests existed

Complicated design, difficult and expensive changes, lots of bugs, chief designer left company, remaining resources had weak knowledge

Desire from management for better, stronger faster but can't find resources or solutions

Solution #3

• Using nothing more than sophisticated than Task Manager was able to demonstrate caching wasn't working for large volume cases where users were reporting problems

• Problems were worse in high latency, lower bandwidth situations. Nobody knew because of lack of instrumentation, despite large test groups

• Pitched a skunkworks project to reimplement fetching and caching subsystem using Agile best practices: unit tests, test driven development etc.

• Demonstrated a working prototype that addressed all outstanding issues

• Not yet implemented yet, but multiyear project to extend existing caching to local disk abandoned during final internal testing due to major performance problems

Case #4 Relatively new .Net application, had unit tests

continuous integration, worked on by large number of developers over the years including contractors Lots of bugs, fixes and changes tended to cause more bugs, expensive to change anything

Use of a limited number of design patterns, complaints that there has to be a better ways. Frustration from developers and business partners. Changes consisted basically of jamming in new features under pressure

Solution #4• Use Code Coverage tool to measure code coverage of unit

tests and showed that coverage was only 3%. Analyzed code repository commit records to show recent developer activity was probably only 1%

• Examined code changes for recent project and existing code. Found poor existing designs, inadequate code reviews, violation of basic OO principles.

Identified one common base class intended to be simple wrapper class was over 11,000 lines. No unit tests and in fact impossible to unit test. Had been worked on by everyone.

• Pitched prototyping rewrite to sympathetic management looking for solutions to improve productivity and quality

• Developed prototype with simplified design, dramatic reduction in size and complexity. Used Agile best practices: Complete unit tests, measured with code coverage tools

Continuous Integration

Builds would be broken for days, then everyone would commit again breaking for days

Lava lamps were introduced because they were cool

Side effect was suddenly failures were visibleto management, reality was exposed

Created the opportunity to redo slow flaky build process created by “experts”

Impact of Change

Dropped build times to 20 minutes

Eliminated build problems

Broken builds became rareoccurrences

Recommended Practices

Agile is people centric so success depends on them. Seek out allies, try not to create enemies!

Minimize dependence and interaction with “experienced” resources that created the broken system or process. Better to remain strangers than become enemies

Don't rely on management to make things happen or enforce team norms. Use dialectic thinking

Skunkworks if necessary

Data is your friend. Make visible to the team and management what you want changed. Turn over the rocks!

Recommended Practices

Appraise the resources carefully. An experienced resource that helped create a broken system or process should be avoided. If they were good they would have fixed it or left. Best is a more junior resource who has knowledge and wants change

Need to assess code quality, design, warts and all in detail to understand weaknesses gaps, possibilities. Don't need to understand everything, just how to exploit that knowledge

Fully embrace Agile best practices, because it makes it harder to argue against change. Focus on universal best practices

In general avoid piecemeal approach to broken legacy systems. Try to focus on larger self contained changes.

Patience and timing are everything and failure leads to success

Peer to Peer Support

• Find your partner again

• Based on what you've just heard what strategies/tactics could you use to make a change in the original situation?

• Take about five minutes and then switch.

• This time both of you dialogue trying to come up with solutions to each person's situation.

Anything to share?Any questions?

In conclusion

The significant problems we face cannot be solved at the same level of thinking we were at when we created them

Things should be as simple as possible, but not simpler

I am neither especially clever nor especially gifted. I am only very, very curious

Albert Einstein