Agile Testing in the Real World - RBCS, Inc · Opinions differ on whether Agile works on outsourced...
Transcript of Agile Testing in the Real World - RBCS, Inc · Opinions differ on whether Agile works on outsourced...
Agile Testing in the Real World Moving Beyond Theory to Achieve Practicality
[NEED PICTURE HERE]
Agile Testing in the Real World
There are lots of books, articles, and webinars about how great Agile is However, those often describe purist approaches that adhere to the Agile Manifesto This webinar isn’t about that pure world, but rather about what happens when Agile theory meets the real world In the real world, companies are coming up with interesting and creative ways to adapt Agile practices This creates various opportunities, challenges, risks, and benefits for testing and quality In this webinar, we examine those…
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 2
What Does Agile Look Like?
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 3
Agile Is the Opposite of What?
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 4
Agile Test Tools
Good stuff, mostly free x-Unit Jenkins Cucumber Ruby Selenium Fitnesse Plus various static code analysis (e.g., Crap4J) and code coverage tools…
Not all is good Jira and Rally are not designed by or for testers Often used as bug and test management tools
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 5
Agile Test Metrics
Tests completed
Risks addressed (in risk-based testing)
Bugs found and fixed
Some organizations don’t measure bugs during sprints (loss of visibility)
Velocity of sprint (user stories completed)
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 6
Agile Testing Challenges
Volume and speed of change Remaining effective in short iterations Increased regression risk Inconsistent or inadequate unit testing Poor, changing, and missing test oracles A shifting test basis Meeting overload Over-commitment in the sprints Sprint team siloing The Agile hype cycle and high expectations For more, check out my recorded webinar on this topic
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 7
Agile Testing Opportunities
Automated unit testing Static code analysis Code coverage Continuous integration Automated functional testing Requirements (user story) reviews Test (acceptance criteria) reviews Reasonable workload Control of technical debt (via “fix bugs first”) For more, check out my recorded webinar on this topic
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 8
Testing in “Agilish” Lifecycles
One RBCS client has a “train” lifecycle Train never stops Releases are never delayed Seven week overlapping cycle (three week releases)
Is this Agile? Purist vs. pragmatist
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 9
Agile Testing for Systems-of-Systems
System testing of each iteration System integration testing by separate team Stabilization sprint at the end to fix bugs Is this Agile? Purist vs. pragmatist
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 10
Testing for Customized Enterprise Software
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 11
Agile on Outsourced Projects
Some Agile founders (e.g., Alistair Cockburn) say collocation is essential
However, many of our clients are doing a lot of outsourced testing and development
Some of these clients are trying to use Agile
Some are succeeding, some are not
An on-site point-of-contact is key to success
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 12
Tester Skill Implications of Agile
In the United States, the trend is towards requiring programming skill in testers
This trend seems to be expanding worldwide
This is especially important on Agile projects
Not only is programming skill needed, but test automation skills are also essential
Another important issue is personality Extroverts generally like Agile because of the emphasis on talking over analysis
Introverts often find Agile harder to adapt to, and some report feeling “bulldozed” by extroverts
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 13
Agile Test Team Organization Options
Fully embedded, testers in each sprint team
Fully independent, testers assigned at the end of each iteration
Matrixed, testers assigned throughout sprint, but part of independent team
Let’s examine each of these options…
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 14
Fully Embedded
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 15
Fully Independent
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 16
Matrixed
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 17
Agile and Quality
Does Agile result in greater quality?
Jury is still out
The use of proven (long before the Agile Manifesto) best practices is promising
However, the tendency not to measure defects during sprints obscures the situation
Some report better quality (e.g., Jones), but metrics are limited
Some of the “proof by assertion” in this area is unhelpful to conducting an objective discussion
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 18
Conclusions
Agile lifecycles are distinct from sequential lifecycles, in ways that affect testing and quality A variety of test tools are available for Agile (and non-Agile) teams Metrics programs may be taking a step backward in Agile projects Agile lifecycles create testing challenges and opportunities There are many different—and successful—implementations of Agile lifecycles Opinions differ on whether Agile works on outsourced projects, but many are trying Agile lifecycles have strong implications for tester skills Carefully choose organizational options for Agile test teams Agile’s effect on quality remains unclear, but anecdotal reports are promising
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 19
Agile Testing in the Real World
www.rbcs-us.com
Copyright (c) RBCS 2008-2013 Page 20
For almost twenty years, RBCS has delivered consulting, outsourcing and training services to clients looking to improve their software and hardware testing and quality. Employing the industry’s most experienced and recognized consultants, RBCS trains and improves test groups, tests products, and provides expert test staff for hundreds of clients worldwide. Ranging from Fortune 20 companies to start-ups, RBCS clients save time and money through improved product development, decreased tech support calls, improved corporate reputation and more. To learn more about RBCS, visit www.rbcs-us.com.
Address: RBCS, Inc. 31520 Beck Road Bulverde, TX 78163-3911 USA Phone: +1 (830) 438-4830 E-mail: [email protected] Web: www.rbcs-us.com
To Contact RBCS