Improving Defect Yield - a three step approach
-
Upload
stag-software-pvt-ltd -
Category
Technology
-
view
456 -
download
3
description
Transcript of Improving Defect Yield - a three step approach
Improving defect yield - A three step approachPowered by HBT
T AshokFounder & CEOSTAG Software Private Limited
in.linkedin.com/in/AshokSTAG Ash_Thiru
© 2013 STAG Software Private Limited. All rights reserved.
Defect Yield
2
Defect yield is the #defects (of good quality) uncovered by the test cases
Yield = Outcome/Effort
Defect yield = #Defects / # TestCases
© 2013 STAG Software Private Limited. All rights reserved.
Expectation & Challenges
3
ExpectationIncreasing the yield - a natural expectation.
ChallengesDefect yield drops/is-low : - when system matures - when system of testing is “weak” - when testing is transitioned to a different team
© 2013 STAG Software Private Limited. All rights reserved.
Why Lower Yield ?
4
Are we wasting effort by testing stable parts?
Are we missing doing something else?
© 2013 STAG Software Private Limited. All rights reserved.
The 3-step approach to yield improvement
5
Potency assessment1
Potential defect type re-targeting2
Potency improvement3
What types of defects are being targeted by the test cases?Are the test cases complete/adequate?
Are there any other types of defects to be targeted?
Add new test cases to cover the new types of defectsEnhance existing test cases to ensure that they are complete
© 2013 STAG Software Private Limited. All rights reserved.
Hypothesis Based Testing - HBT
6
System Under Test
Cleanliness Criteria
Potential Defect Types
Test CasesRequirementstraceability“what to test”
Faulttraceability“test for what”
should satisfy impeded by
© 2013 STAG Software Private Limited. All rights reserved. 7
Quality Levels & PDTs - Powered by HBT
Input cleanliness
Input interface cleanliness
Structural integrity
Behaviour correctness
Environment cleanliness
Attributes met
Flow correctness
Clean Deployment
End user value
L1
L2
L3
L4
L5
L6
L7
L8
L9
That inputs are handled wellPDTs related to Input data correctness
That the functional behaviour is correctPDTs related to Functionality
That the internal structure is robustPDTs related to Structural aspects
That the user interface is cleanPDTs related to UI
That end-to-end flows work correctlyPDTs related to Flow behaviour, Interactions
That it does not mess up the environmentPDTs related to Resource leaks, Compatibility...
That the stated attributes are metPDTs related to Performance, Load, Volume......
That it deploys well in the real environmentPDTs related to Compatibility, Migration
That user expectations are metPDTs related to User flows, User experience
© 2013 STAG Software Private Limited. All rights reserved.
Potency Framework (For Assessment & Re-target)
8
QL9 End user value End user value testPDT 57-59
QL8 Clean deployment Installation test PDT 54-55
Migration testPDT 56Migration test
QL7 Attributes met LSPS testPDT 44-51
Reliability testPDT 52Reliability testPDT 52
Security testPDT 53
QL6 Environment cleanliness Good citizen testPDT 39-41
Compatibility testPDT 42-43Compatibility test
QL5 Flow correctness Interaction testPDT 35-38
QL4 Behaviour correctness Functionality testPDT 24-31
Access control testPDT 32-34Access control test
QL3 Structural integrity Structural testPDT 14-23
QL2 Input interface cleanliness API validation testPDT 5-7
GUI validation testPDT 8-13GUI validation test
QL1 Input cleanliness Input validation testPDT 1-4Input validation test
QL - Quality LevelPDT - Potential Defect Type
Pow
ered
by
HB
T
© 2013 STAG Software Private Limited. All rights reserved.
An illustration : A component of a large system
9
Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1BeforeBeforeBeforeBeforeBefore ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)
#PDTs
#TCs#TCs#TCs #Defects '+PDTs '+TCs'+TCs'+TCs #Defects
Total '+ve '-ve Total '+ve '-veL8 0 0L7 1 1 1 0 1 2 1 1L6 0 0L5 0 0L4 7 1116 976 140 8 7 45 26 19 17L3 1 135 130 5 0L2 2 29 26 3 0L1 3 23 9 14 0Total 14 1304 1142 162 8 8 47 27 20 17
Increase byIncrease byIncrease byIncrease byIncrease by 57% 4% 2% 12% 213%
TC YieldTC Yield 0.01 Defects/TCDefects/TC TC YieldTC Yield 0.36 Defects/TCDefects/TC
Potency assessment PDT re-targeting21
3 Potency improvement
© 2013 STAG Software Private Limited. All rights reserved.
An illustration - The entire system
10
Potency assessment PDT re-targeting21
3 Potency improvement
Customer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : Summary
BeforeBeforeBeforeBeforeBefore ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)
#PDTs #TCs#TCs#TCs #Defects '+PDTs '+TCs'+TCs'+TCs #Defects
Total '+ve '-ve Total '+ve '-ve
C1 14 1304 1142 162 8 8 47 27 20 17
C2 20 543 353 190 6 8 176 114 62 7
C2 6 741 494 247 19 7 17 12 5 0
Total 40 2588 1989 599 33 23 240 153 87 24
Increase byIncrease byIncrease byIncrease byIncrease by 58% 9% 8% 15% 73%
TC YieldTC Yield 0.01 Defects/TCDefects/TC TC YieldTC Yield 0.10 Defects/TCDefects/TC
10x yield improvement
© 2013 STAG Software Private Limited. All rights reserved.
HBT OverviewSIX staged purposeful activities, powered by EIGHT disciplines of thinking
11
D1D2
D4
D3
D5D6
D7
D8
Core Concepts
S1
S2
S3S4
S5
S6SIX Stages of DOINGS1: Understand expectationsS2: Understand contextS3: Formulate hypothesisS4: Devise proofS5: Tooling supportS6: Assess & Analyse
EIGHT Disciplines of ThinkingD1: Business value understandingD2: Defect hypothesisD3: Strategy & PlanningD4: Test designD5: ToolingD6: VisibilityD7: Execution & ReportingD8: Analysis & Management
Uses 32 Core ConceptsFor Problem solving - Techniques, Principles, Guideline
Click here to know more about HBT.http://stagsoftware.com/blog?p=570
© 2013 STAG Software Private Limited. All rights reserved.
HBT is the intellectual property of STAG Software Private Limited.STEMTM is the trademark of STAG Software Private Limited.
www.stagsoftware.com
@stagsoft
blog.stagsoftware.com
Connect with us...
Improving defect yield - A three step approachPowered by HBT
Potency assessment1
Potential defect type re-targeting2
Potency improvement3
Thank you.