Istqb chapter 5
-
Upload
nstprabakaran -
Category
Documents
-
view
606 -
download
3
Transcript of Istqb chapter 5
![Page 1: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/1.jpg)
Test Management
Software Testing
1 Principles 2 Lifecycle
4 Dynamic testtechniques
3 Static testing
5 Management 6 Tools
![Page 2: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/2.jpg)
Contents
Organisation
Configuration Management
Test estimation, monitoring and control
Incident management
Standards for testing
ISEB Foundation Certificate Course
Test Management
1 2
4 5
3
6
![Page 3: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/3.jpg)
3
Importance of Independence
Time
No. faults
Release toEnd Users
![Page 4: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/4.jpg)
4
Organisational structures for testing
� Developer responsibility (only)� Development team responsibility (buddy
system)� Tester(s) on the development team� Dedicated team of testers (not developers)� Internal test consultants (advice, review,
support, not perform the testing)� Outside organisation (3rd party testers)
![Page 5: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/5.jpg)
5
Testing by developers
� Pro’s:- know the code bestknow the code best- will find problems that the testers will misswill find problems that the testers will miss- they can find and fix faults cheaplythey can find and fix faults cheaply
� Con’s- difficult to destroy own workdifficult to destroy own work- tendency to 'see' expected results, not actual resultstendency to 'see' expected results, not actual results- subjective assessmentsubjective assessment
![Page 6: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/6.jpg)
6
Testing by development team (buddy)
� Pro’s:- some independencesome independence- technical depthtechnical depth- on friendly terms with “buddy” - less threateningon friendly terms with “buddy” - less threatening
� Con’s- pressure of own development workpressure of own development work- technical view, not business viewtechnical view, not business view- lack of testing skilllack of testing skill
![Page 7: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/7.jpg)
7
Tester on development team
� Pro’s:- independent view of the softwareindependent view of the software- dedicated to testing, no development responsibilitydedicated to testing, no development responsibility- part of the team, working to same goal: qualitypart of the team, working to same goal: quality
� Con’s- lack of respectlack of respect- lonely, thankless tasklonely, thankless task- corruptible (peer pressure)corruptible (peer pressure)
- a single view / opiniona single view / opinion
![Page 8: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/8.jpg)
8
Independent test team
� Pro’s:- dedicated team just to do testingdedicated team just to do testing- specialist testing expertisespecialist testing expertise- testing is more objective & more consistenttesting is more objective & more consistent
� Con’s- ““over the wall” syndromeover the wall” syndrome- may be antagonistic / confrontationalmay be antagonistic / confrontational- over-reliance on testers, insufficient testing by over-reliance on testers, insufficient testing by
developersdevelopers
![Page 9: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/9.jpg)
9
Internal test consultants
� Pro’s:- highly specialist testing expertise, providing support highly specialist testing expertise, providing support
and help to improve testing done by alland help to improve testing done by all
- better planning, estimation & control from a broad better planning, estimation & control from a broad view of testing in the organisationview of testing in the organisation
� Con’s- someone still has to do the testingsomeone still has to do the testing- level of expertise enough?level of expertise enough?- needs good “people” skills - communicationneeds good “people” skills - communication- influence, not authorityinfluence, not authority
![Page 10: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/10.jpg)
10
Outside organisation (3rd party)
� Pro’s:- highly specialist testing expertise (if outsourced to a highly specialist testing expertise (if outsourced to a
good organisation)good organisation)
- independent of internal politicsindependent of internal politics� Con’s
- lack of company and product knowledgelack of company and product knowledge
- expertise gained goes outside the companyexpertise gained goes outside the company- expensive?expensive?
![Page 11: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/11.jpg)
11
Usual choices
� Component testing:- done by programmers (or buddy)done by programmers (or buddy)
� Integration testing in the small:- poorly defined activitypoorly defined activity
� System testing:- often done by independent test teamoften done by independent test team
� Acceptance testing:- done by users (with technical help)done by users (with technical help)
- demonstration for confidencedemonstration for confidence
![Page 12: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/12.jpg)
12
Resourcing issues
� independence is important- not a replacement for familiaritynot a replacement for familiarity
� different levels of independence- pro's and con's at all levelspro's and con's at all levels
� test techniques offer another dimension to independence (independence of thought)
� test strategy should use a good mix- "declaration of independence”"declaration of independence”
� balance of skills needed
![Page 13: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/13.jpg)
13
Skills needed in testing
� Technique specialists� Automators� Database experts� Business skills & understanding� Usability expert� Test environment expert� Test managers
![Page 14: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/14.jpg)
Contents
Organisation
Configuration Management
Test estimation, monitoring and control
Incident management
Standards for testing
Test Management
1 2
4 5
3
6
![Page 15: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/15.jpg)
15
Problems resulting from poor configuration management
� can’t reproduce a fault reported by a customer� can’t roll back to previous subsystem� one change overwrites another� emergency fault fix needs testing but tests
have been updated to new software version� which code changes belong to which version?� faults which were fixed re-appear� tests worked perfectly - on old version� “Shouldn’t that feature be in this version?”
![Page 16: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/16.jpg)
16
A definition of Configuration Management
� “The process of identifying and defining the configuration items in a system,
� controlling the release and change of these items throughout the system life cycle,
� recording and reporting the status of configuration items and change requests,
� and verifying the completeness and correctness of configuration items.”- ANSI/IEEE Std 729-1983, Software Engineering ANSI/IEEE Std 729-1983, Software Engineering
TerminologyTerminology
![Page 17: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/17.jpg)
17
Configuration Management
� An engineering management procedure that includes- configuration identificationconfiguration identification- configuration controlconfiguration control- configuration status accountingconfiguration status accounting- configuration auditconfiguration audit
• Encyclopedia of Software Engineering, 1994Encyclopedia of Software Engineering, 1994
![Page 18: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/18.jpg)
18
Products for CM in testing
� test plans� test designs� test cases:
- test inputtest input- test datatest data
- test scriptstest scripts- expected resultsexpected results
� actual results� test tools
CM is criticalfor controlled
testing
CM is criticalfor controlled
testing
What would not be underconfiguration management?
Live data!
![Page 19: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/19.jpg)
Contents
Organisation
Configuration Management
Test estimation, monitoring and control
Incident management
Standards for testing
Test Management
1 2
4 5
3
6
![Page 20: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/20.jpg)
20
Estimating testing is no different
� Estimating any job involves the following- identify tasksidentify tasks- how long for each taskhow long for each task
- who should perform the taskwho should perform the task- when should the task start and finishwhen should the task start and finish- what resources, what skillswhat resources, what skills- predictable dependenciespredictable dependencies
• task precedence (build test before running it)task precedence (build test before running it)
• technical precedence (add & display before edit)technical precedence (add & display before edit)
![Page 21: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/21.jpg)
21
Estimating testing is different
� Additional destabilising dependencies- testing is not an independent activitytesting is not an independent activity- delivery schedules for testable items misseddelivery schedules for testable items missed
- test environments are criticaltest environments are critical� Test Iterations (Cycles)
- testing should find faultstesting should find faults- faults need to be fixedfaults need to be fixed- after fixed, need to retestafter fixed, need to retest- how many times does this happen?how many times does this happen?
![Page 22: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/22.jpg)
22
Test cycles / iterations
Debug D RD R
3-4 iterations is typical
TestTheory:
Test
Practice:
Des Ex VerBldIden
Retest
Retest
![Page 23: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/23.jpg)
23
Estimating iterations
� past history� number of faults expected
- can predict from previous test effectiveness and can predict from previous test effectiveness and previous faults found (in test, review, Inspection)previous faults found (in test, review, Inspection)
- % faults found in each iteration (nested faults)% faults found in each iteration (nested faults)- % fixed [in]correctly% fixed [in]correctly
� time to report faults� time waiting for fixes� how much in each iteration?
![Page 24: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/24.jpg)
24
Case history
Incident Reports (IRs)
0
20
40
60
80
100
120
140
160
180
200
04-Jun 24-Jul 12-Sep 01-Nov 21-Dec 09-Feb
Opened IRs
Closed IRs
Source: Tim Trew, Philips, June 1999
![Page 25: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/25.jpg)
25
Control
� Management actions and decisions- affect the process, tasks and peopleaffect the process, tasks and people- to meet original or modified planto meet original or modified plan- to achieve objectivesto achieve objectives
� Examples- tighten entry / exit criteriatighten entry / exit criteria- reallocation of resourcesreallocation of resources
Feedback is essential to see the effect of actions and decisions
Feedback is essential to see the effect of actions and decisions
![Page 26: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/26.jpg)
26
Entry and exit criteria
Test Phase 1
Test Phase 2
"tested"is it ready for my
testing?
Phase 2 Phase 1
Entry criteria Exit criteria
Acceptancecriteria
Completioncriteria
![Page 27: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/27.jpg)
27
Entry/exit criteria examples
poor
better
� clean compiled� programmer claims it is working OK� lots of tests have been run� tests have been reviewed / Inspected� no faults found in current tests� all faults found fixed and retested� specified coverage achieved� all tests run after last fault fix, no new faults
![Page 28: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/28.jpg)
28
What actions can you take?
� What can you affect?- resource allocationresource allocation- number of test iterationsnumber of test iterations- tests included in an tests included in an
iterationiteration
- entry / exit criteria entry / exit criteria appliedapplied
- release daterelease date
� What can you not affect: - number of faults already number of faults already
therethere� What can you affect
indirectly?- rework effortrework effort- which faults to be fixed which faults to be fixed
[first][first]- quality of fixes (entry quality of fixes (entry
criteria to retest)criteria to retest)
![Page 29: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/29.jpg)
Contents
Organisation
Configuration Management
Test estimation, monitoring and control
Incident management
Standards for testing
Test Management
1 2
4 5
3
6
![Page 30: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/30.jpg)
30
Incident management
� Incident: any event that occurs during testing that requires subsequent investigation or correction.- actual results do not match expected resultsactual results do not match expected results- possible causes:possible causes:
• software faultsoftware fault
• test was not performed correctlytest was not performed correctly
• expected results incorrectexpected results incorrect
- can be raised for documentation as well as codecan be raised for documentation as well as code
![Page 31: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/31.jpg)
31
Incidents
� May be used to monitor and improve testing� Should be logged (after hand-over)� Should be tracked through stages, e.g.:
- initial recordinginitial recording- analysis (s/w fault, test fault, enhancement, etc.)analysis (s/w fault, test fault, enhancement, etc.)- assignment to fix (if fault)assignment to fix (if fault)- fixed not testedfixed not tested- fixed and tested OKfixed and tested OK- closed closed
![Page 32: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/32.jpg)
32
Use of incident metrics
Is this testing approach “wearing out”?
What happenedin that week?
We’re betterthan last year
How many faultscan we expect?
![Page 33: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/33.jpg)
33
Report as quickly as possible?
report
5test can’t reproduce - “not a fault” - still there
can’t reproduce, back to test to report again
insufficient information - fix is incorrect
dev 5
reproduce
20
fix5
re-test fault fixed
10dev
can’treproduce
incidentreporttest
10
![Page 34: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/34.jpg)
34
What information about incidents?
� Test ID� Test environment� Software under test ID� Actual & expected results� Severity, scope, priority� Name of tester� Any other relevant information (e.g. how to
reproduce it)
![Page 35: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/35.jpg)
35
Severity versus priority
� Severity- impact of a failure caused by this faultimpact of a failure caused by this fault
� Priority- urgency to fix a faulturgency to fix a fault
� Examples- minor cosmetic typominor cosmetic typo- crash if this feature is usedcrash if this feature is used
company name,board member:
priority, not severe
Experimental,not needed yet:
severe, not priority
![Page 36: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/36.jpg)
36
Tester Tasks Developer Tasks
Incident Lifecycle
1 steps to reproduce a fault
2 test fault or system fault
3 external factors that influence the symptoms
4 root cause of the problem
5 how to repair (without introducing new problems)
6 changes debugged and properly component tested
7 is the fault fixed?
Source: Rex Black “Managing the Testing Process”, MS Press, 1999
![Page 37: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/37.jpg)
Contents
Organisation
Configuration Management
Test estimation, monitoring and control
Incident management
Standards for testing
Test Management
1 2
4 5
3
6
![Page 38: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/38.jpg)
38
Standards for testing
� QA standards (e.g. ISO 9000)- testing should be performedtesting should be performed
� industry-specific standards (e.g. railway, pharmaceutical, medical)- what level of testing should be performedwhat level of testing should be performed
� testing standards (e.g. BS 7925-1&2)- how to perform testinghow to perform testing
![Page 39: Istqb chapter 5](https://reader031.fdocuments.in/reader031/viewer/2022013108/559b635f1a28ab1f258b45d4/html5/thumbnails/39.jpg)
Summary: Key Points
Independence can be achieved by different organisational structures
Configuration Management is critical for testing
Tests must be estimated, monitored and controlled
Incidents need to be managed
Standards for testing: quality, industry, testing
Test Management
1 2
4 5
3
6