Post on 21-Jan-2018
Detailed Manual Estimation Approach for Pre-sale phase
Vlada Benyukh, Volodymyr Prymakov
Speaker info
Vlada Benyukh, UkraineQA Consultant/QA Lead at Ciklum, TCoE
• 7 years in QA
• Manual QA Engineer, QA Lead
• Certified: SAFe, ICAgile, ISTQB
/in/vlada-benyukh
Speaker info
/in/vladimirprimakov/
Volodymyr Prymakov, UkraineSenior QA Manager at Ciklum, TCoE
Head of Performance QA Unit at TCoE
•14 years in QA
•45 projects experience
•Certified: ISTQB Advanced Test Analyst & Manager,
ICAgile CP, SAFE
Agenda
1. What is an Estimate? Why it is important?
2. Which basis and factors should be considered in estimation process?
3. What main and additional activities should be included in QA Estimation
approach?
4. How Regression testing should be calculated?
5. How QA Estimation can be adjusted by risks multipliers?
6. Demo of real-life example
Why we need to estimate?
How to estimate these tasks?
Assumptions
We assume that:
1. Project is planned to work based on Scrum
2. Development team size: up to 30 people
3. QA Engineer Competency is Middle Level
4. Non-functional requirements are estimated separately
Test Basis
Such test basis should be considered as below:
1. Functional requirements in scope (WBS) are defined
2. Test environments and their priorities are determined while pre-sale phase
3. Project duration (number of sprints) is considered before estimation
process
What factors?
1. Domain specific knowledge and technologies used
2. Integration-related problems/risks (within internal and external systems)
3. Complexity of the application (e.g. calculations verifications)
4. Compatibility testing (e.g. file compatibility, plugin computability etc)
What main activities?
1. Detailed Requirements Review*
2. Test Design
3. Test Execution
4. Bug investigation and submission. Bug Retest
5. Environments/Platforms Multipliers
How to estimate time?
Time calculation for main test activities = (test execution + bugs submission and retest) *environments/platforms multiplier+ test design+ detailed requirements review*
How to calculate env multiplier?
Environments/platforms factors to select:
1. Identify customer region and target audience where the application will be
used
2. Select top used environments/platforms based on the customer’s
region/audience
Example to calculate env multiplier?
Web-browser e.g. Test Type Coefficient
The highest priority web-browser Chrome 57Full Test (Positive, negative flows) 1
High priority web-browser IE 11, FF54 Only positive flows tests 0,5
Medium priority web-browser Edge14 Explorartory testing 0.25
Low priority web-browser Chrome 56Smoke test (at least 1 positive flow test) 0.1
EXAMPLE of multiplier calculation: 1 + 0.5 + 0.5 + 0.25 + 0.1 = 2.35
Example of estimation basis
What typical preparation activities?
1. Detailed evaluation of the product/project. Full Master Test Plan creation
2. Learning and getting acquainted with the product's documentation and/or
functionality. Preliminary requirements review and feedback
3. Checklist/Suite creation (with empty test cases)
4. Test Environment and Infrastructure Preparation & Access
Example of preparation activities?
What typical repetitive activities?
1. Sprint Grooming & Planning, Retrospective
2. Daily Communication Meetings
3. Demo Meetings
4. Global Release Readiness Analysis and Reporting
5. Regular Product Deploy and Configuration, Test Environment Configuration,
Test Data Preparation
* If estimated by PM, it should NOT be estimated here and it should be set to 0h
Example of typical repetitive activities
How to calculate Regression?
1. Number of sprints: (e.g. 12)
2. Number of regression testing cycles: (e.g. 6 or 3)
3. Pure New functionality testing time ⇒ (without time working with bugs and
test environment multiplier)= (e.g. 300h)
4. Percentage of regression-tested functionality: (e.g. 60%)
5. Test environments Multiplier: (e.g. 2)
How to calculate Regression?
Pure new functionality testing time (if 0 bugs)
How to calculate Regression?
Regression cycles: 6; regressed functionality: 60%
How to calculate Regression?
Regression testing time * Test environments Multiplier: 2
How to calculate Regression?
If number of regression cycles is 3
How to calculate Regression?
Calculation:1) Time for last regression cycle = Pure New functionality testing time (0 bugs) * % of regressed functionality * Test environments Multiplier
2) Regression increment = Time for last regression cycle * number of regression cycles
3) Time for regression testing = Regression Increment * (1+2+… number of regression cycles)
Example of Regression calculations
What basis for Smoke Testing?
1. Number of smoke testing cycles in sprint
2. Number of sprints/iterations expected
3. Approximate time for smoke
4. Test environments Multiplier
Example of Smoke calculations
What Risks should be included?
1. Changing/Not Stable Requirements, Increased scope (10%)
2. Documentation and requirements clarification problems (7%)
3. Risk of functionality integration/interdependence and regression
problems (10%)
4. Buggy application risk, because of developers competence (5%)
5. Technology risks (5%)
6. Risk of not mature processes and time spent for improving them (5%)
7. Infrastructure & environment Security & Accessibility Risks (5%)
8. Cross project communication necessity/dependency risk (5%)*
Example of Risks basis
Example of Summary calculations
Demo of real-life example
Demo of real-life example
Statistics
1. General amount of estimated projects: 15
2. General amount of the project where QA estimation should be adjusted, because of scope and deadlines changes: 5
3. Duration of the projects: 1 -12 months
4. Team size: 5-30
Questions