Scrum gathering Paris 2013 - test automation strategy for Scrum Projects

INdT

TestLink

Quality

Bugtracker

Validation

VERIFICATION

Bug

SCRUM

Test Automation Strategy and Practices

for Scrum Projects

Nokia Institute of Technology

Test Automation

Eliane Collins

Paris, France - Sept 2013

Scrum Gathering 2013 - Paris

AGILITY

Company Confidential

Nokia Internal Use Only

2. Study Cases in Test

Automation

1. Introduction Summary

3. Results and Lessons

Learned

4. Test Automation

Strategy

5. Execution

6. Conclusion



3

Introduction



INTRODUCTION

4

INTRODUCTION

• It is an independent and nonprofit

institute committed to conducting

research and development of

technological solutions through

application development, new

technologies and concepts. Manaus

Recife

Brasília

São Paulo

Main Areas:

• Product Creation/Product Validation

• Apps and Services

• Hardware and Manufacturing

• Software User Interface

INdT – Nokia Institute of Technology

- Master degree in Software

Engineering.

- 8 years of experience in Software

Testing for web and mobile

platforms.

- Coordinator of Product Validation

team in INDT.

- Instructor and researcher of

projects in related area.

- International certifications CTAL-

TM, CSD, ITIL and COBIT.

Who I am?



5

INTRODUCTION

Global Survey [LOGIGEAR, 2010], found that 40% of companies surveyed

believe that test automation is not easy, only 15% automate regression testing

and 56% fail to maintain test automation, even using the appropriate tools.

Technical survey by Kasurinen in 2010,

revealed only 26% of test cases in the

organizations automated.

Technical survey by DIAS-NETO in 2006,

indicated Brazilian companies from Manaus

do not execute formal tests and test

automation.



6

INTRODUCTION

Difficulties:

• The Test Automation requires time, training and financial resources

to acquire tools .

• Constant maintenance of the test environment is needed.

• The development team do not

collaborate with test automation.



7

Study Cases in Test Automation



Study Case 1

8

Strategy 1: Testers Automating Unit and System Tests :

Team members: 4

Testers decided to perform the unit testing tasks using

GoogleTest tool;

Functional tests for the desktop installer and configuration

screen were automated using Sikuli tool ;



Strategy 2: Developers Automating Unit Tests and Testers

Automating System Tests

Team members: 7

Developers coding new features, unit testing using TDD

approach and Continuous Integration (Hudson).

Testers automating UI Tests using Selenium.

Study Case 2

9



Strategy 3: Test team divided in Test team to analyze and test team

to automate tests.

Team members: 8

Developers just coding new features and unit tests.

Test team to automate Integration tests, UI Tests using

Selenium, Load Tests and Security using JMeter.

Test team to analyze and execute manual tests.

Study Case 3

10



11

Results



Study Case 1

12

Positive and Negative points from Retrospective meetings:

Bugs found per story: 4




Positive

Negative

Bugs found per story: 2,8

Positive Negative

CI strategy improved testing automation

High coverage of GUI testing automation

Security and load tests executed early

Poor knowledge transferring Competition between testers and

developers High effort to automate tests with

record-and-play approach Test automation has been quit. Separation between testing and

development teams

Study Case 2

13




Positive

Negative

Bugs found per story: 2.4

Positive Negative

High coverage of GUI testing automation

Testers that code automated scripts are closer to development team

Exploratory testing reveal more important defects.

Poor knowledge transferring High effort to automate tests with record-

and-play approach The developers give the responsibility for

quality to test team. Difficult synchronize the system changes

and the automated test suite. High effort to update test scripts. Test automation has been delayed

during some sprints. Separation between test analyst, testers

who automate scripts and development teams

Study Case 3

14



Common problems

Separated teams ;

Separated work tools and environment infraestructure;

No collaboration;

Complex test tools and test environment;

Centralized knowledge;

The priority is time to delivery;

Results

15



Results

All strategies have no co-operation between team project

16

Strategy 2

Strategy 3

Strategy 1

Lyndsay, 2006.

Source < http://www.mftrou.com/workplace-conflict-management-strategy.html>.



17

Test Automation Strategy

for Scrum Projects



Test Automation Strategy for Scrum Projects

Iterative and Incremental:

• The test automation architecture should be developed and

updated in each iteration.

• The test scripts should be added to every story developed.

18




Collaboration between team members:

• The test automation tasks are the responsibility of the whole

project team.

• Developers and testers should collaborate, interact and work

together in the activities of test automation.

• The test scripts should be added to every story developed.

19




Encourage Collaboration:

• Co-location: scrum team must be work together in the same

place.

• Pair Programming: tester and programmers in pair to code

automated test scripts.

• Code and documents review.

• Get out of comfort zone

20




Changing Control:

• The test tools must be under configuration management.

• The artifacts such as logs and access history and changes

should be available.

• Test automation under Continuous Integration.

21




Process Improvement

• The evaluation of test automation related tools, tasks and

scripts must be made at the meeting of the retrospective

• The lessons learned at this meeting are extracted and serve as

improvement for upcoming projects.

22




The tasks can be shared.

23




• Test tools under continuous integration.

• Test code under development tools and CI.

• Everyone in the project can generate builds and reports every

time.



25

Execution and Results



Study Case 4: Developers and Testers Automating Unit Tests,

Integration Tests and System Tests

Team members: 9.

Developers and testers configured tools and programmed in

pair functional test scripts for unit tests, integration tests and

system tests;

Tools: Junit and Fitnesse.


26




Bugs per story: 1.42

Positive Negative CI facilitates testing automation. Security and load tests executed

early. Unit test review increased test

coverage. Intense team collaboration. Knowledge transferring regarding

the tools. Testing responsibilities distributed

to all team . Team motivation to learn new

solutions.

High initial effort to set the automation environment.

Rework when customer requires changes in the system.


27



28

Conclusion



Collaboration supports the Success of Agile Test Automation;

Co-location facilitates collaboration;

Automate Each Layer of Software When It Is Possible and Just for

Acceptance Tests in the Sprint;

Test Automation in the Agile Project Should Be Simple, Reusable

and Maintainable for Project Team;

Use Test Automation for Documentation and Information Feedback ;

Conclusions

29



It is feasible to adapt agile practices and values to test automation.

The automation was a resource to document software, reduce cost,

and allocate tasks in smaller parts.

The agile values allow the test automation maintenance.

Next steps:

Evaluate the test automation collaborative strategy as

framework;

Evaluate the impact of team collaboration in distributed test

teams.

Conclusion

30



Thank you! Merci! Gracias! Obrigada!

31

Contact: [email protected]

Blog: http://testautomationdiary.blogspot.com.br/

mailto:[email protected]

http://testautomationdiary.blogspot.com.br/

Scrum gathering Paris 2013 - test automation strategy for Scrum Projects

Technology

Transcript of Scrum gathering Paris 2013 - test automation strategy for Scrum Projects