Test Automation in Agile

2

Agenda

• Capgemini’s World Quality Report on Automation

• Evolution of business models and IT ecosystem

• QA and Testing in the agile world

• Agile and test automation

• Challenges in Agile automation

• In conclusion …

3

Connect with Capgemini:https://www.capgemini.com/testing-services

Capgemini’s World Quality Report

4

“By 2018, one-third of the top 20 market share leaders in most industries will be significantly disrupted by competitors that use the 3rd Platform to create new services and business models.”

Evolution of Business Models

1st PlatformMainframes . Everything centralized dumb terminalsVoice: Telephony stand alone

2nd PlatformC/S: Distributed computing PC, tablet & smart phoneVoice: Telephony as part of computer HW

3rd PlatformSMAC eraVoice to access & exchange real time video, data, text, transactions first step toward cognitive computing

Mainframe Terminal

LAN/ Internet Client/Server

Source: IDC

5

Evolution of IT ecosystem

Development Process Application Architecture Deployment & Packaging Application Infrastructure

Waterfall

Agile

CI/CD

Dev(T)Ops

Monolithic

N-Tier

Microservices

Physical Servers

Virtual Servers

Containers

Datacenter

Hosted

CloudInfra as a Code

6www.unicomlearning.com/ITW_Pune/

Testing is embedded within the Scrum process

• Test preparation begins with completion of user stories

• Daily test runs are conducted along with the development team and errors communicated to scrum team and stakeholders

• Testing is also performed on partially completed scenarios

• Testing is dynamic, changes can occur at any time

• Coordinate additional testing (performance, UAT etc.,) with non-agile development teams, as needed

There is no separate allocated schedule for testing.All elements have to be tested within the scrum team.

Testing in the Agile process

7

Testing in Agile development lifecycle

Req Design & Coding Test AT Req

Design & Coding Test AT Req Design &

Coding Test AT

Dynamic Analysis

Unit Testing

Dynamic Analysis

Unit Testing

Dynamic Analysis

Unit Testing

Functional Test Functional TestFunctional Test

Reviews & Static Analysis

Reviews & Static Analysis

Reviews & Static Analysis

Performance TestingAutomation Testing

Test Data ManagementTest Environment Management

Testing and Quality MetricsConfiguration Management

Agile Testing Objectives•Test as early and as often possibleContinuously plan the testing effort with every sprintAdapt and Evolve with each delivery and test cycle

•Test enough & PrioritiseTest most important features first

•Pair testing with development Constant development testing interaction . Jointly develop test cases. Leverage the Unit test casesLeading guiding agile principle “Every one test, Customer accepts”

Testing Types and Phases•Validation of User Stories•Acceptance Tests

•Validation of Tasks implementing the User StoriesUnit TestsIntegration TestsFunctional TestsExploratory TestingPerformance TestsRegression of previous Sprint(s)

Agile Testing ApproachSprint-1 Sprint-2 Sprint-3

Defect Acceptance Ratio (DAR)

Defect Removal Efficiency (DRE)

% Deferred Defects Reopen defects Test case Execution (TCE) -

ratio of test cases executed vs. test cases planned

Test Automation Coverage

Key Agile Testing Metrics

8www.unicomlearning.com/ITW_Pune/

Agile Development and Testing Activities

9

TMap NEXT® - How TMap is Applied to the Agile Test Lifecycle

A project contains one or more iterations. Per project a compact project test plan is made. An iteration contains one or more user stories. Per iteration a compact iteration test plan is made. The preparation (P), specification (S), execution (E) and completion (C) phases apply

to each individual user story. The control and infrastructure phases are continuous phases covering the whole project.

Preserve testware

Evaluate process

Iteration Plan

Control

Infrastructure

Planning

Iteration 1

E

P S

C

Story 7

E

P S

C

Story 13

Iteration 2 Iteration n

E

P S

C

Story 4

E

P S

C

Story 11

Develop test strategy Risk analysis Test estimation Prepare test plan Define organization

• Review requirements

• Assign techniques (See Appendix)

• Identify scenarios

Create test scripts Define test data

Execution

Preparation Specification

Story 1

Test, (Re)test Check, Assess

Manage the test project Report metrics Control budget &

Timelines

Execution

Completion

10www.unicomlearning.com/ITW_Pune/

A tester in an agile environment: Is a skilled communicator Is flexible Has domain knowledge Is creative but practical Is solution-oriented Is customer-focused Is a team player

Supports the product owner in the prioritization of product backlog and acceptance criteria Supports the business analyst Evaluates unit tests Pairs with a developer to help program with minimal errors Participates in daily scrum meetings

The most important characteristic is that the tester must be proactive andopen-minded, and, as a part of the team, feels responsible for the delivery of a quality

result.

Testing in the Agile process

11www.unicomlearning.com/ITW_Pune/

• Automation has moved beyond graphical user interface testing using Record and playback mode.

• Faster Time to Market, Increased productivity, Real time Data Processing

• Learn to Program. Programming is first step for automation. Be well versed with scripting languages like Python, Ruby etc.

• Learn Automation tools in the market

• Social, Mobile, Analytics, Cloud (SMAC) – Digital everywhere !!

• Devops Automation - the way forward!

Agile and Automation

12

Agile and Automation

Requirements Architecture & Design Development Testing Release

Shift

Lef

t

Test Driven Development

Automation: 80 - 100%

Unit TestsAutomation: 80 - 100%

Behavior driven development

Automation: 50 – 70%

Service VirtualizationAutomation 40 – 60%

Shift Right

Continuous Integration

Auto: 80 – 100%

Continuous Delivery

Auto:10-60%

UAT Automation

40 – 60%

Continuous Monitoring

Auto: 40 – 60%

Environment virtualization

Auto: 40 – 60%

Non functional tests Automation: 80-100%

Performance TestsAutomation; 100%Regression test

Automation 60 – 80%

Test Management/Defect Management

Automation: 70 - 100%

Sprint Acceptance TestsAutomation 40 – 50%

Exploratory tests Automation: 30 – 40%

Functional TestsAutomation: 40 – 50%

13

Test Automation Approach in Agile

Sprint 1 Sprint 2 Sprint 3 Sprint 4

C1 C1

C2

C1

C3

F1

C2

C1

C2

C4

C3

F1Tester automates in N + 1sprint once stable in N sprint

F1 Stable FeatureC Stable Component C Component

14

Test Driven Development [TDD]

Write a test that fails

Make the code work

Eliminate the

redundancy by

refactoring

The mantra of Test-Driven Development (TDD) is red, green, refactor.

JUnitNUnit

HttpUnit

15

Acceptance Test Driven Development

The information contained in this presentation is proprietary. Copyright ©2011 Capgemini. All rights reserved.

15

Write an acceptance

test that fails

Make the code work

Eliminate the

redundancy by

refactoring

16Copyright © Capgemini 2015. All Rights Reserved

Continuous Integration Automation framework

Continuous IntegrationAutomation

Physical / Virtual Environment

Ready for Deployment

SCMEnvironment Sanity

/ Smoke Test

Unit Testing Environment Pre-Check

Deployment

Test Automation

/Regression Suite

SCM Tools

MS TFS, SVN, IBM RTC, Clear Case, HP ALM

Build Tools

MS TFS, Maven, Jenkins, Cruise Control, Build Forge

Unit Testing Tools

MSTest, JUnit

Testing Automation Tools

MS Coded UI, Café Next QTP, Selenium, BDD +

Cucumber

17

With a ready to use tool kit for automation

Release and Sprint Planning : • Version One, Jira Agile, HP Agile Manager, Microsoft TFS,

Agile Tools - Snapshot

TDD and BDD tools :• Cucumber, Jbehave, Junit, xUnit,

Jmeter,csunit, CAST

Sprint Level – Services Automation:CA Lisa, HP ServiceTest

Functional Automation: • QTP, Café (Capgemini Automation

Framework),IMDA, Selenium

Exploratory and test design• HP Sprinter, BluePrint (MBT)

Metrics:• LIVE (Lifecycle Integration with Virtualized

Engine)• HP Quality Center , Version 1,Jira , TFS

Build & Integration: • Jenkins, CruiseControl, Gnumake

18

Future of Automation – A sneak peek

Technical connect Normal testing as a profession would diminish and testers would be expected to have

scripting knowledge with the ability to automate.

Business connect Automation testers would be expected to have domain and business knowhow.

DevOps

End to end continuous integration automation would be a necessity not an innovation.

SMAC adoption Multiple channel integration and smart device compatibility is a necessity.