Agile Testing

Jeff Checkner . Joseph Beckenbach .

Who are we?

Jeff Checkner - QA Manager at CheckFree – is now part of Fiserv

Manage a team that is responsible for testing Web Based Bill Payment Applications.

Prior to January 2003 (Tester) Tested in Waterfall Software Development Lifecycle.

January 2003 (Working Manager) Transitioned the test team from Waterfall SDLC to extreme Programming (XP).

January 2005 (Manager) Transitioned the test team from extreme Programming (XP) to Rational Unified Process (RUP).

Joseph Beckenbach - Lead Technical Tester at VersionOne

Extend suites and tools for testing web-based Agile Project Management software

Oct 2003 - Mar 2005 (Tech Tester) embedding within XP team

at biotech software startup

Jun 2005 - Sep 2006 (j.o.a.t.) supporting teams adopting Scrum

at Yahoo! Search Marketing

Dec 2007 - now (Tech Tester) embedded within Scrum team

Presentation Objectives

Challenges of Non-Iterative Quality Assurance

Agile Terminology

What is Agile?

Iterative Planning

Collaboration

Best Practices for Agile Testing

Questions

Challenges of Non-Iterative Quality Assurance

A typical 7 month non-iterative life-cycle may look like this:–Requirements gathering 3-4 weeks–Design 8 weeks–Development 6 weeks–Testing 6 weeks–Certification/Pilot 2-4 weeks

If each step of the life-cycle must wait for the previous to complete, a design bug could get all the way to testing before being caught (here, 14 weeks out).

If Quality Assurance gets code at the end of the life-cycle and severe bugs are found, then QA has less time to test the important items as they are spending most of their time resolving issues that could have been addressed earlier.

QA not involved early enough to gain full understanding of intent of the project. Just given requirements and told to test.

Any changes to the project by the customer is painful. Test plans need to be updated and code needs to be re-tested, typically dates do not move when this happens.

Agile Terminology

Sprint/Iteration - A Sprint (Scrum) or iteration (XP) is a fixed period of time (typically 2-4 week period).

Story – Any piece of work that can be accomplished in 1 Sprint/Iteration.

Ideal hours – The number of hours of uninterrupted work. For example there are 40 hours of calendar hours in 1 week. Not all 40

hours can be devoted to initiative work.

Velocity – The average number of hours a team can devote to initiative work in 1 sprint. (calculated by Total number of ideal hours worked in 1 sprint divided by number of team members)

Customer/Stakeholder - anyone who has a stake in the creation or operation of the system. The team works to satisfy the requirements generated by the customer.

What is Agile?

Agile Development consists of Sprints/Iterations in that time (usually 2-4 weeks). The basic premise is that before any code is written, the team (Design, Development, and QA) takes the requirements and fleshes them out into stories. – No different for QA.

Each Sprint/iteration is an entire software project: including planning, requirements analysis, design, coding, testing, and documentation. An iteration may not add enough functionality to warrant releasing the product to market but the goal is to have an available release (without bugs) at the end of each iteration. At the end of each iteration, the team re-evaluates project priorities. – No different for QA.

What is Agile (2)?

Agile Development brings together the customer, developers, and testers before any code has been written. They collaborate to identify a specific piece of functionality, or “story,” to be worked on. Customers and testers then specify criteria to validate that the story works and create an executable document that can be accessed by anyone on the team. – No different for QA.

Having expected results created upfront by a business/QA person helps to find miscalculations, misinterpretations, and miscommunications right away rather than late in the project. - Improves quality earlier, before the code is written so number of defects should decrease.

Some Agile Tools

Tools that assist in the process:

Stand up meetings – typically happen daily where each person describes what they did yesterday, what they are going to do today, and what roadblocks they foresee.

Velocity Calculation – Tracking work in stories allows team to track amount of work accomplished each sprint and give an insight into Velocity to determine how much work a team can realistically accomplish.

Planning Tools – There are tools available to run iterative development that will allow you to plan out sprints/iterations. Ex. Version One, Rally Dev, index cards, create your own.

Value of Iterative Planning

1. More Accurate estimates – Current allocation of test estimates based on development hours is a cause of inaccurate estimates.

When Testing hours are calculated as a percentage of Development estimates the hours to not take into account the regression activities needed to ensure that the code change does not impact other areas of the application.

2. More Accurate Reporting – Using the planning tool to “Story out” each sprint gives Management and Project Manager the ability to see the progress each resource group is making to the iteration goals.

3. Resource Accountability – By having each resource sign up for the stories for a particular Sprint they have accountability to that work.

4. Increased Visibility - It will give visibility to what activities are being requested of resources that side track them from Story completion.

5. Velocity Calculation – Using a planning tool gives resource managers the ability to calculate the team velocity (actual initiative hours spent during a 2 week Sprint) over time to help in future planning.

6. Traceability – Using the Planning tool allows Stories to be tied back to requirements

Value of Iterating

10

Too often

Time

Func

tion

Iterative

PlanIdeal

Collaboration

In Agile Software Development Lifecycles Collaboration is “King”.

Each tester must think of themselves as not just a tester but a member of the team by:

– Building iterative plans - Knowing what is to be delivered and when it will be delivered.

– Partnering with all members of SDLC:Development – Working together to build a more robust unit test suite by conducting Unit test reviews.

Design – Assist in use case identification / creation.

Product Management – Work with Product team to help lead them into making good decisions when defining requirements.

Best Practices for Agile Testing

Attack major risks early and continuously, or they will attack you

Identify Risks to the project earlyExamples: technology, people, organization

Assist Design in identifying Alternate Scenarios and Business rules Assist in Refining Requirements

Ensure that you deliver value to your Customer Get involved in Design to help and influence Product in their decision

process.

Stay focused on executable software Develop Test Cases earlier Develop and Execute Tests during each iteration.

Best Practices for Agile Testing (2)

Accommodate change early in the project Focus on the goals of the current iteration, which reduces the impact

of changes made by the Customer Expect to see test cases and test artifacts produced for the first

iteration grow incrementally as each iteration completes.

Baseline an executable architecture early on Validate the significant elements by executing tests in each iteration.

Build your system with components Assist development with definition and review of unit tests

Best Practices for Agile Testing (3)

Work together as one team Participate in design meetings that contain Design, Development,

Quality Assurance, Product Manager, and Product Delivery Manager “Walk and Talk” to members of team Work with Development in Automated unit test identification and

review

Feedback Provide feedback early, often, and on anything.

Make quality a way of life, not an after-thought Add value throughout the whole life-cycle, by starting early and testing

the design until the code is implemented. Contribute in helping to solve the business problem.