Let the adventure begin the 80-20 testing - ingenuity

INDIUM SOFTWAREAn Independent Software Testing Firm

Let the Adventure Begin - The 80-20 Testing

“Business Assurance through Quality Assurance”

© 2014 Indium Software

ENHANCING SOFTWARE QUALITY

2

Agenda

Testing – The Need to be Quick

Challenges – With Scripted Tests

Interesting Usage of Intelligence

Case Study – Req & Risks

Exploratory Approach

Metrics and Value Additions

Conclusion



3

Testing - The Need to be Quick

Today’s need:New technology and resources – the environment is unwilling to take its time for the quality achievementsQuick turnaroundCompact lifecycles

The above said:Are we testing enough?Do we have the time to test enough?Are scripted tests complete?What best can we do to ensure coverage?

Let the adventure begin with Exploratory Testing…



4

Challenges – With Scripted Tests

Constant change and Lack of documented requirements

Time constraints and Insufficient test coverage

Early detection of defects

Code breaks & Performance Bottlenecks

Challenges

• What is the effect?

• Why?

• How it affects?

• When?



5

Interesting Usage of Intelligence

Several types of random tests -

When to use what

Situations where

requirement changes

along with an unsettled

process

CurrentA well

groomed approach yielding

maximum test

coverage and high

defect catch rate

Need

Do not validate

unplanned

A complete process with maximum

utilization of minimum resources

Solution



6

Case Study

Requirement

Changing combination of environments

Release Frequency

Weekly (sometimes even lesser)

Content Change Rate

Daily

Testing Life Cycle

3-4 Days

Testing Scope

Compatibility / Functionality

Types of Testing

Scripted & Exploratory

Application Type

Web Application



7

Risk

Story Board

Discussions

Short development cycles

Unrelated Weekly Changes

Support Quick Updates

Platform Dependent Test Cases



8

Exploratory Approach – Team Setup

Team Setup

Planning

Execution

Reporting

Maintenance

• Skilled resources• Knowledge in

productGeneral

• Handy documents• Training is easy• Converts any

resource to handle the process

Derived



9

Exploratory Approach – Planning

Team Setup

Planning

Execution

Reporting

Maintenance

• Minimum planning• Traditional testing

technique used at random

General

• Know your domain and your user

• Data analysis & Areas to focus on

• RCA / Know your environment

• Identify test cases from the suite

Derived



10

Exploratory Approach – Execution

Team Setup

Planning

Execution

Reporting

Maintenance

• Focused testing• Fixed combinationGeneral

• Standardized charter• Soap opera testing• Negative testing

Derived



11

Exploratory Approach – Reporting

Team Setup

Planning

Execution

Reporting

Maintenance

• No formal documentation• Chances of missed

scenario• Redundant testing

General

• Parallel documentation• Exploratory Procedure• Charted• Reviewed• Supervised

Derived



12

Exploratory Approach – Maintenance

Team Setup

Planning

Execution

Reporting

Maintenance

• Testing is considered done

• Script updatesGeneral

• Trackers• Fine tuning scripted

testsDerived



13

Metrics

o This metrics highlights defects found in Scripted vs. Exploratory

o Introduction of derived exploratory testing contributed to 20% more defects in the High and Medium category

o Derived exploratory testing helped to uncover affected component under the UI, Display and behind the scenes functionality

1

2

3

4

5

6

0

10

20

30

40

50

60

Scripted Testing

Generalized Exploratory

Derived Exploratory

IterationsD

efec

ts



14

Metrics

o This chart shows the defects in Test case

o Projects with consecutive iterations need lots of effort in script maintenance, which helps in hands-on testing

o The derived exploratory testing,

helped make documentation more stable therefore reducing time spent in test script fixes

1 2 3 4 5 60

5

10

15

20

25

30

35

40

45

35

40

23

15

8 7

Test Case Defects

Iterations

Defe

cts



15

Metrics

o This pie chat represents the distribution of defects based on the testing type

o Defects with the derived approach is 20% more

o This 20% raises confidence in the application by 80%

o Proves derived approach is a best practice for faster iterations and frequent builds

70%

7%

22%

Defect Distribution

Scprited Testing

Generalized Exploratory

Derived Exploratory



16

Value Additions

Key Levers With Derived Exploratory Testing Value Additions

Test Script maintenance

Reduced by 50% Advanced strategy for bug detection More coverage area Easier to maintain platform dependent scripts Reduced timelines in test script maintenance

TC Defects Reduced by 80% Time saved in test cases fixes

Defect Detection Increased by 25% Defect pattern identified easily Early detection of bugs

Exploratory Issues Increased by 20% Logged high severity issues Increased confidence in product for the business

team

Reporting Better by 40% Proactive monitoring Effective documentation for future reference



17

Conclusion

All paths may or may not lead to the same destination…

… but have all paths been tested?

Try our Derived Exploratory Approach!



18

Thank You

USA: [email protected]

EMEA & APAC: [email protected]@indiumsoft.com

INDIA: [email protected]

SunnyvaleSuite 210, 1250 Oakmead ParkwaySunnyvale, CA – 94085.Phone: +1(408) 501-8844Fax: +1(408) 501-8808

Atlanta Crown Office Suites1870 The Exchange, Suite 100 Atlanta, GA 30339Phone: +1 (678) 608-0809, +1(770) 989-7302

New DelhiF-1/5 Okhla Industrial Area Phase-I, New Delhi – 110 020.Phone: +91-11-6613 0400

BengaluruSRS Plaza, No. 246, 2nd & 3rd Floor13th Cross, Indira Nagar 2nd StageBengaluru – 560 038.Phone: +91-80-4914 7777

ChennaiVDS House41, 2nd Floor, Cathedral RoadChennai – 600 086.Phone: +91-44-2811 6330Fax: +91-44-4210 4033

mailto:[email protected]




https://www.facebook.com/pages/Indium-Software/290116636922

http://www.linkedin.com/company/82503

https://twitter.com/indiumsoft

Let the adventure begin the 80-20 testing - ingenuity

Technology

Transcript of Let the adventure begin the 80-20 testing - ingenuity