Property of Progressive Insurance & Casualty Company 1 Agile Test Automation Created and Modified...

Property of Progressive Insurance & Casualty Company

1

Agile Test Automation

Created and Modified By:

Nishant AwasthiJanuary 13, 2009


2

Agile Test Automation

Treat testware as software Application of agile development

principles to the test automation project.Requirements modeling.Automation architecture modeling.Acceptance testing of automated

scripts.Time Boxed iterations with frequent

delivery of testware.


3

Agile Automation Process… contd.. Continuous integration of testware

Running tests with the latest development build

Continuous integrated testingExecute tests daily in the Development and

QA environment Provide continuous feedback

Lock-step with the development and the QA team for iterative releases of testware


4

QA Cycle

The Automation Cycle

Development Cycle

Unit Test andCode

EmbeddedTesting

AcceptanceTesting

AutomationTracker

Test PlanCreation

Story CardAssignment

AutomationPrioritizationand Analysis

AutomationAssignment

AutomationDevelopment

AutomationAcceptance

SourceControl

Build Proc

AutomationRelease

QA EnvDev Env

TestDirectorLab Machines

SystemTesting

Nightly and Ad Hoc

Nightly and Ad HocCheck-in

Build

Release

To Prod

Manual Testing

Notes DB

AutomationScript Review

RequirementsSynch


5


Development Cycle

Unit Test andCode

EmbeddedTesting

AcceptanceTesting

AutomationTracker

Test PlanCreation






SourceControl

Build Proc

AutomationRelease

QA EnvDev Env


SystemTesting

Nightly and Ad Hoc


Build

Release

To Prod

Manual Testing

Notes DB


RequirementsSynch

Development IPM

Development IterationPlanning Meeting

for the current iteration


6

Development Cycle


Unit Test andCode

EmbeddedTesting

AcceptanceTesting

AutomationTracker

Test PlanCreation






SourceControl

Build Proc

AutomationRelease

QA EnvDev Env


SystemTesting

Nightly and Ad Hoc


Build

ReleaseTo Prod

Manual Testing

Notes DB


RequirementsSynch


7

Development Cycle


Unit Test andCode

EmbeddedTesting

AcceptanceTesting

AutomationTracker

Test PlanCreation






SourceControl

Build Proc

AutomationRelease

QA EnvDev Env


SystemTesting

Nightly and Ad Hoc


Build

ReleaseTo Prod

Manual Testing

Notes DB


RequirementsSynch


8

Automation CeremoniesName Frequency Agenda

Automation Planning Meeting (APM)

Every other weekFollowing DevelopmentIteration planningmeeting.

1) Review automation progress 2) Reprioritize any existing test cases as needed3) Review test cases prioritized for the current iteration

a. Overview of functionalityb. Determine automatabilityc. Determine dependencies (script modification, new

development etc.) d. Change Automation Status as needede. Assign automation developer to analyze the test plan and

create a task in Automation Tracker.

Automation Releasemeeting (ARM)

Every other week Following Development iteration releases.

1) Determine velocity and burn down.2) Determine current workload.3) Synchronize scripts for release.

Automation Acceptance Ad Hoc, as and when required.

1) Synch up with QA analyst and get feedback on the behavior and functionality of the script.

2) Mark the script for elevation in the daily run of automated regression Testware.


9

Development Cycle


Unit Test andCode

EmbeddedTesting

AcceptanceTesting

AutomationTracker

Test PlanCreation






SourceControl

Build Proc

AutomationRelease

QA EnvDev Env


SystemTesting

Nightly and Ad Hoc


Build

ReleaseTo Prod

Manual Testing

Notes DB


RequirementsSynch

APM

Automation Planning Meetingfor the current iteration


10

Automation PrioritizationAttribute Description

Business Risk of Not Automating Test If one script will be testing a higher-risk feature, it should be assigned a higher priority.

Number of Planned Executions If a script will be executed more times, it should be assigned a higher priority.

Efficiency (Automated vs. Manual) If one script will eliminate more manual testing it should have a higher priority.

Volatility of Features Being Tested If one script is testing a more volatile feature, it should be assigned a lower priority, since it will require more revisions due to the volatility.

Dependency of Other Tests If a high priority task depends on the completion of the candidate task, then the candidate task should also be assigned a high priority.

Reuse Potential Scripts which can be reused should have higher priority than one-off scripts.

Open Defects Scripts which test an open defect should be prioritized according to the severity/priority of the defect.

Automatibility If a test cannot be automated it should receive a low priority (at best). If it will take significant effort to automate, especially in relation to the benefit to the automation, it should also receive a low priority.

Usability If a script tests usability as opposed to proper functioning of the application, it should receive a lower priority.


11


Unit Test andCode

EmbeddedTesting

AcceptanceTesting

AutomationTracker

Test PlanCreation






SourceControl

Build Proc

AutomationRelease

QA EnvDev Env


SystemTesting

Nightly and Ad Hoc


Build

ReleaseTo Prod

Manual Testing

Notes DB


RequirementsSynch


12

Model based test script design Model : description of a system’s

behavior Simpler than the actual systemState modeling is a common

practice in embedded systems world Finite State Model : consists of a set

of states, a set of input events and the relations between them.

Generate Sequences of Test Actions from the FSM


13

Model based test script …contd… Application behavior model

Exploratory modeling WalkthroughsPaired exploration

Capturing states of the application

When is this action possible?What is the outcome of this action?

Automated test action sequences


14

Automation Steps1) Test Plan Analysis

Analyze and refactor the manual test plan. Group similar test conditions.

2) Script design and development Modularize your script based on the different states or state transitions your

application Keep a single flow either positive or negative in each action that you record. Return back to the initial state in your script. Record the flow you are automating. Try to add all the objects on a particular page from the active screen. Use smart identification or vary the object recognition properties if required. Playback your script Repeat these steps for the negative flow

3) Parameterization Use the data table of QTP for positive data parameterization. Repeat the parameterization for negative data.

4) Checkpoints Use custom and built in checkpoints

5) Script Elevation Get your script acceptance tested and reviewed Elevate it to Test Director


15


Unit Test andCode

EmbeddedTesting

AutomationTracker

Test PlanCreation





SourceControl

Build Proc

AutomationRelease

QA EnvDev Env


SystemTesting

Nightly and Ad Hoc


Build

ReleaseTo Prod

Manual Testing

Notes DB

RequirementsSynch



AcceptanceTesting

Acceptance

Automation acceptancebetween QA and automationdeveloper


16


Unit Test andCode

EmbeddedTesting

AcceptanceTesting

AutomationTracker

Test PlanCreation




SourceControl

Build Proc

AutomationRelease

QA EnvDev Env


SystemTesting

Nightly and Ad Hoc


Build

ReleaseTo Prod

Manual Testing

Notes DB

RequirementsSynch




ARM

Automation release meetingbetween automationdevelopers


17


Unit Test andCode

EmbeddedTesting

AcceptanceTesting

AutomationTracker

Test PlanCreation




SourceControl

Build Proc

AutomationRelease

QA EnvDev Env


SystemTesting

Nightly and Ad Hoc


Build

ReleaseTo Prod

Manual Testing

Notes DB

RequirementsSynch





18

Roles and ResponsibilitiesQA Analyst Automation Developer Application Developer

1)Mark automation status pending after the test plan is created.2)Prioritize the test plan with respect to need for automation 3)Review and walkthrough the prioritized test plan to the automation developer.4)Notify the automation developer working on the test plan if there are any changes.5)Provide feedback to the automation developer after the automation status is “Ready for Release”

1)Pick up test plans that says “Creation complete” prioritized and assigned. 2)After picking up the test plan change Automation Status to In Process and Update Automation Comments field with name/time stamp.3)Estimate and define task(s) in Automation tracker (task planning system)4)Create automation script. 5)Synch up with QA analyst and get feedback on the behavior and functionality of the script after developing the script.6)Mark the test plan as ready for release after getting it reviewed.7)Release the test script into the daily run of “Automated Regression Testware” and mark automation status as completed.8)Facilitate automation ceremonies.

1)Keep automation team in loop while making UI changes.2)Help automation team with controls and technical details of the app as and when required.

Property of Progressive Insurance & Casualty Company 1 Agile Test Automation Created and Modified...

Documents

Transcript of Property of Progressive Insurance & Casualty Company 1 Agile Test Automation Created and Modified...