Archana Joshi Testing in agile is it easier said than done

3rd Asia Pacific Software Testing Conference

Testing in Agile

Easier said than done?

Archana Joshi Wipro Consulting Services

[email protected]

© 2009 Wipro Ltd - Confidential 2 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 2

Who am I?

About Myself - Archana Joshi:

•Manager - Agile practice at Wipro Consulting Services • Certified Scrum Practitioner with 9+ years in software development, management and consulting • Involved as a scrum master, process consultant, agile coach on various teams

About Wipro Consulting Services:

• Wipro Consulting Services (WCS) is a division of Wipro Ltd (NYSE: WIT), a $5.5bn global enterprise. • WCS has 1200+ consultants across the globe providing unique solutions to our clients worldwide. • The Agile initiative started in 2004 and agile is now one of the mainstream processes at Wipro. • WCS has a dedicated agile practice which consults teams and clients on agile implementation.


Program Landscape

Development program for a new web based application in wealth management space

• Technology J2EE / Web 2.0 / Flex • Involves integration with 3rd party components like – SFDC, SiteMinder, LDAP AD, Akamai

Drivers for agile • Respond to changing needs of the business • Improve the relationship among the technology and business partners • Gain competitive edge in dynamic financial market through reducing time to market

Team

• Program divided across 3 agile tracks - based on functionality • Each track was a cross-functional distributed team involving

•Developers (vendor 1 & customer IT) •Testers (vendor2), BA (vendor3) • User Experience personnel (vendor 4) • Scrum Master (vendor1), Product Owner (customer business)

• Shared resources included business, legal, architecture, security and compliance teams

Agile Awareness • Excluding the scrum masters, entire team was new to agile • Team had undergone 2 days class room training on agile


• New Product development

– Integration with other 3rd party applications

– Integration across multiple tracks

• Test Team belonged to different hierarchy within the organization

– Forming cross-functional teams

– Work together as a cohesive team

• Compliance to financial and security regulations

– Testing to consider these requirements too

• Team new to agile way of working

– Documentation needed for everything to start work

– All manual testing, minimal automation

• Team distributed across locations

– Multiple location, time zone challenges

Challenges for testing


Overall Process and Release Cycle

Stabilization 2 week Iterations of Development with Testing

Production Live Interim integration points

2 wks 1 wk

2 wks 2 wks

2 wks 1 wk 2 wks 1 wk

2 wks 2 wks 2 wks 2 wks 2 wks 2 wks 5 wks

2 wks 2 wks 2 wks 2 wks 2 wks 2 wks 5 wks

2 wks 1 wk 2 wks 1 wk 2 wks 2 wks

2 wks

2 wks

Track 1

Track 2

Track 3

Release Cycle

Initiation Stabilization Iteration Release and Deployment

Process


Key Lessons Learnt from the program

#1 Team Involvement - Sooner the better

#2 Divide and rule

#3 Work together as a team

#4 Code will change. Accept it.

#5 Readiness checks are necessary

#6 Do not forget to measure

#7 Establish governance mechanism at start


#1 Team Involvement - Sooner the better

• The test team was involved right from the initiation & planning stage • Test strategy formed during the initiation stage.

• It contained the high level approach on how system shall be tested • It also contained the responsibilities of the testers within the team at different stages of the project

• For estimations and planning along with the business and development teams, testers were also involved • Early involvement helped testers in understanding business perspective and dependencies better


#2 Divide and rule

• Instead of trying to fit all types of tests within an iteration, division based on types was done • Testing within an iteration

• Feedback provided by business, UX team & testers • Unit testing & Integration testing (by developers) • Functional testing (by testers with guidance from BA) • Regression testing (by testers)

• Testing outside an iteration (stabilization)

• Performance testing • Security (ethical hack) testing • Final UAT & Regression testing • Exploratory testing by business and legal teams


#3 Work together as a team

• Though there were multiple vendors spread across different locations and time zones, the cross-functional team worked as a cohesive team • The scrum master had to be on a constantly look out for any team behaviors which can result into group-ism and arrest the behavior at the start • Collaboration tools like net meeting, VCON, TCON were extensively used • Some of the key team members in the team – team leads, sr. developers, testers, BA came together at same physical location for planning and initial iteration of the program


#4 Code will change. Accept it.

• Testers typically test on a code which is fairly stable • When an iteration is 2 weeks long, there are code changes till the last day of the iteration • There were challenges in changing the mindset of the testers that the code undergoing test is going to change frequently during development of the iteration • During iteration

• At end of the day, developers provided links to their code from the development environment which the testers used to provide feedback on. • This was considered as feedback & not defects as the iteration was work in progress • At end of the iteration which ever feedback was not addressed was entered as defects in the Quality Center by the test team • Test team was responsible for collating the feedback from business and UX teams as well

C

h

a

n

g

e


#5 Readiness checks are necessary

• Agile does not mean that the checks or sign-off are no longer to be taken by the test team • Within an iteration

• Review were done by the BA and business on the test cases and test plan to ensure coverage • “Done” ness criteria was established for tracking the iteration completeness. • “Done” ness criteria also included the compliance and security checks

• Outside an iteration • Readiness criteria to move from one environment (integration -> staging -> production) was established by the test team

Demo Env. /

Dev. Env.

Integration

Env

Staging

Env.

Production

Env.

Test Environment during

stabilization Test Environment during iterations


#6 Do not forget to measure

• Monitoring of metrics is important. Especially for a large program • HP Quality Center was used as the test management tool • Following metrics were captured

• Within an iteration • Defects which are not addressed and carried forward in an iteration • Test cases and scripts that were completed and on which environment

• Out side iteration • Defect spread based on priority • Test scripts % execution • Test scripts % passing

747

61

583

4685Total Pending Cases/Scripts

Total PassTotal Fail

Total Percentage Passed

Total Scripts Blocked/ Descoped

Execution Status

Total Test Cases/Scripts Planned

Total Executed


• The teams – business, BA, test, developer belonged to different groups within the organization

• There was a need to establish a strong governance structure which was agreeable to various stakeholders involved

• This was not finalized by the time the program started and hence there were issues seen during the initial iterations

• The PMO was made aware of the process to be followed. They provided the by-in on the process and artifacts that were produced as part of the agile process

• Learning from retrospective of each track was circulated at all levels thus ensuring management support

#7 Establish governance mechanism at start

Level 1

Level 2

Level 3 Key stakeholders from IT, business, legal, security &

compliance, PMO

Scrum Master, IT Program

Manager, IT shared teams

Track – 1: Scrum Master and Team

Track - 2: Scrum Masters and

Team

Track - 3: Scrum Masters and

Team

Weekly monitoring, Program risks, Budget, Costs, Change Management

Weekly monitoring, Metrics, Inter & Intra team dependencies IT risks and mitigation for program

Daily tracking, Project Quality, Resource Capacity


Benefits

• Fewer number of defects found during final UAT

• Hardly any issues seen after the application went live

• Less number of testing resources required compared to the traditional approach

• Better team morale – Team feeling agile is the way to go !

• Increased appreciation between business, developer & test teams for their respective work


In Conclusion….

• Start with simple practices that are easy to adopt

• Implement a piece meal approach

• Advanced practices like extensive Automation, Test Driven Development can come in later

• Allow the team time to adapt to the agile way of working

………..And you will realize instead of being

“Easier Said Than Done”

it will be “Easier Said And Done”


Thank you for your time !

Questions ?

Archana Joshi Testing in agile is it easier said than done

Technology

Transcript of Archana Joshi Testing in agile is it easier said than done