DSR Testing (Part 2)
-
Upload
steve-upton -
Category
Engineering
-
view
123 -
download
1
Transcript of DSR Testing (Part 2)
![Page 1: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/1.jpg)
Software Testing@Steve_Upton
![Page 2: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/2.jpg)
![Page 3: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/3.jpg)
![Page 4: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/4.jpg)
![Page 5: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/5.jpg)
![Page 6: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/6.jpg)
![Page 7: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/7.jpg)
![Page 8: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/8.jpg)
Systems testing
![Page 9: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/9.jpg)
Systems testing
Test the system as a whole
Step up from integration testing
Aim to test close to the real world
Many sub types
Generally done by dedicated testers
![Page 10: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/10.jpg)
Performance + Load testing
![Page 11: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/11.jpg)
Performance + Load testing
Performance: test performance of systems (duh)
Load: test behaviour under load
Also: stress, volume, scalability, recovery testing
![Page 12: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/12.jpg)
Performance + Load testing
Performance: test performance of systems (duh)
Load: test behaviour under load
Also: stress, volume, scalability, recovery testing
Relies on tooling
![Page 13: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/13.jpg)
![Page 14: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/14.jpg)
config:
target: 'https://artillery.io'
phases:
- duration: 60
arrivalRate: 20
defaults:
headers:
x-my-service-auth: '987401838271002188298567'
scenarios:
- flow:
- get:
url: "/docs"
![Page 15: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/15.jpg)
config:
target: 'https://artillery.io'
phases:
- duration: 60
arrivalRate: 20
defaults:
headers:
x-my-service-auth: '987401838271002188298567'
scenarios:
- flow:
- get:
url: "/docs"
Scenarios launched: 5
Scenarios completed: 5
Requests completed: 58
RPS sent: 0.86
Request latency:
min: 102.4
max: 3067.5
median: 325.5
p95: 2118.5
p99: 3020
Scenario duration:
min: 56745.4
max: 67339.1
median: 59275.6
p95: NaN
p99: NaN
Codes:
200: 58
![Page 16: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/16.jpg)
![Page 17: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/17.jpg)
![Page 18: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/18.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Automated Tools
Manual
![Page 19: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/19.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Performance/Load testing
Automated Tools
Manual
![Page 20: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/20.jpg)
Usability testing
![Page 21: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/21.jpg)
Usability testing
Can people use our product?
“Consumability”
Really difficult to do with team members
Accessibility testing
![Page 22: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/22.jpg)
![Page 23: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/23.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Automated Tools
Manual
![Page 24: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/24.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Usability testing
Automated Tools
Manual
![Page 25: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/25.jpg)
(User) Acceptance Testing
![Page 26: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/26.jpg)
(User) Acceptance Testing
Gates (Waterfall)
Client sign off (UAT)
Operational acceptance
Regulation/contract acceptance
![Page 27: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/27.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Automated Tools
Manual
![Page 28: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/28.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Acceptance testing
Automated Tools
Manual
![Page 29: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/29.jpg)
Exploratory testing
![Page 30: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/30.jpg)
Exploratory testing
Generally unstructured, time-boxed exploration
Minimum planning, maximum execution
Can be hugely productive/informative, but unpredictable
Experience based
![Page 31: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/31.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Automated Tools
Manual
![Page 32: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/32.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Exploratory testing
Automated Tools
Manual
![Page 33: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/33.jpg)
Security testing
![Page 34: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/34.jpg)
Don’t get hacked!
Security testing
![Page 35: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/35.jpg)
Don’t get hacked!
Data protection regulations
Security testing
![Page 36: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/36.jpg)
Don’t get hacked!
Data protection regulations
Vulnerability scanning
Security testing
![Page 37: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/37.jpg)
Don’t get hacked!
Data protection regulations
Vulnerability scanning
Security reviews/audits
Security testing
![Page 38: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/38.jpg)
Don’t get hacked!
Data protection regulations
Vulnerability scanning
Security reviews/audits
Penetration testing
Security testing
![Page 39: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/39.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Automated Tools
Manual
![Page 40: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/40.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Security testing
Automated Tools
Manual
![Page 41: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/41.jpg)
Alpha/Beta testing
![Page 42: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/42.jpg)
Alpha/Beta testingWhat’s the difference between Alpha and Beta?
![Page 43: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/43.jpg)
Alpha/Beta testingWhat’s the difference between Alpha and Beta?
¯\_(ツ)_/¯
![Page 44: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/44.jpg)
Alpha/Beta testingWhat’s the difference between Alpha and Beta?
¯\_(ツ)_/¯
![Page 45: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/45.jpg)
Alpha/Beta testingWhat’s the difference between Alpha and Beta?
¯\_(ツ)_/¯
Generally, Beta is more mature
![Page 46: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/46.jpg)
Alpha/Beta testingWhat’s the difference between Alpha and Beta?
¯\_(ツ)_/¯
Generally, Beta is more mature
Goals include:
- Testing in close to ‘real’ environment
- Gain feedback from users (UAT)
- Prepare for release
![Page 47: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/47.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Automated Tools
Manual
![Page 48: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/48.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Alpha/Beta testing
Automated Tools
Manual
![Page 49: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/49.jpg)
Defect Management
![Page 50: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/50.jpg)
Defect reporting
![Page 51: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/51.jpg)
![Page 52: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/52.jpg)
![Page 53: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/53.jpg)
![Page 54: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/54.jpg)
![Page 55: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/55.jpg)
Q: What is our goal when reporting a defect?
A: Provide enough information to make the defect actionable
![Page 56: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/56.jpg)
Q: What is our goal when reporting a defect?
A: Provide enough information to make the defect actionable
![Page 57: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/57.jpg)
IEEE 1044-2009Defect ID - Unique identifier for the defect.
Description - Description of what is missing, wrong, or unnecessary.
Status - Current state within defect report life cycle.
Asset - The software asset (product, component, module, etc.) containing the defect.
Artifact - The specific software work product containing the defect.
Version detected - Identification of the software version in which the defect was detected.
Version corrected - Identification of the software version in which the defect was corrected.
Priority - Ranking for processing assigned by the organization responsible for the evaluation, resolution, and closure of the defect relative to other reported defects.
Severity - The highest failure impact that the defect could (or did) cause, as determined by (from the perspective of) the organization responsible for software engineering.
Probability - Probability of recurring failure caused by this defect.
Effect - The class of requirement that is impacted by a failure caused by a defect.
Type - A categorization based on the class of code within which the defect is found or the work product within which the defect is found.
Mode - A categorization based on whether the defect is due to incorrect implementation or representation, the addition of something that is not needed, or an omission.
Insertion activity - The activity during which the defect was injected/inserted (i.e., during which the artifact containing the defect originated).
Detection activity - The activity during which the defect was detected (i.e., inspection or testing).
Failure reference(s) - Identifier of the failure(s) caused by the defect.
Change reference - Identifier of the corrective change request initiated to correct the defect.
Disposition - Final disposition of defect report upon closure.
![Page 58: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/58.jpg)
What goes in a defect report?
![Page 59: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/59.jpg)
What goes in a defect report?
Expected/actual results
Steps to reproduce
Supporting materials (screenshots, logs etc.)
Impact/severity
Input from tester (thoughts, intuitions)
![Page 60: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/60.jpg)
How do you write a defect report?
Clear
Accurate
Objective
![Page 61: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/61.jpg)
Defect Management Systems
![Page 62: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/62.jpg)
Defect Management Systems
![Page 63: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/63.jpg)
![Page 64: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/64.jpg)
![Page 65: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/65.jpg)
Do you need a Defect Management System?
![Page 66: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/66.jpg)
Do you need a Defect Management System?
Useful for tracking
![Page 67: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/67.jpg)
Do you need a Defect Management System?
Useful for tracking
Useful for auditing and traceability
![Page 68: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/68.jpg)
Do you need a Defect Management System?
Useful for tracking
Useful for auditing and traceability
Less relevant for Agile processes
![Page 69: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/69.jpg)
Do you need a Defect Management System?
“Defect tracking systems don’t promote
communication between programmers and testers.”Lisa Crispin, Agile Testing
Useful for tracking
Useful for auditing and traceability
Less relevant for Agile processes
Communication is key!
![Page 70: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/70.jpg)
Defect prioritisation
![Page 71: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/71.jpg)
Impact
Pro
babi
lity
![Page 72: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/72.jpg)
Impact
Pro
babi
lity
Options menu uses wrong font
![Page 73: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/73.jpg)
Impact
Pro
babi
lity
Misaligned text on main screen
Options menu uses wrong font
![Page 74: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/74.jpg)
Impact
Pro
babi
lity
Misaligned text on main screen
Options menu uses wrong font
Uninstalling app crashes phone
![Page 75: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/75.jpg)
Impact
Pro
babi
lity
Misaligned text on main screen
Options menu uses wrong font
App crashes on startup
Uninstalling app crashes phone
![Page 76: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/76.jpg)
Test metrics
![Page 77: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/77.jpg)
What are metrics?
![Page 78: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/78.jpg)
What are metrics?
A way of understanding our effectiveness
A way to identify problems
A guide to success
![Page 79: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/79.jpg)
What are metrics?
A way of understanding our effectiveness
A way to identify problems
A guide to success
“A pit of wasted effort … numbers for the sake of
numbers … actively harmful”
Lisa Crispin, Agile Testing
![Page 80: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/80.jpg)
Code coverage
![Page 81: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/81.jpg)
Code coverageHow much of the code has been tested (covered)?
Several ways of measuring
Generally summed over a test suite
![Page 82: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/82.jpg)
Code coverage - TypesFunction coverage
How many functions/methods have been called?
Statement coverage
How many executable statements have been tested?
Branch coverage
Has every branch in program flow been tested?
![Page 83: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/83.jpg)
def greet(name):
print 'Hello', name
def comment_on_age(age):
print 'In ten years, you will be', (age + 10)
![Page 84: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/84.jpg)
def func(a, b):
if (a > 10 and b == 5):
a = a * b
if (a == 5 or b == 5):
a += 10
a += b
![Page 85: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/85.jpg)
def func(a, b):
if (a > 10 and b == 5):
a = a * b
if (a == 5 or b == 5):
a += 10
a += b
a>10 and b==5
a==5 or b==5
a += 10
a += b
a = a * b
![Page 86: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/86.jpg)
def calculate_interest(balance):
interest = 0.1
if (balance > 1000):
interest = 0.05
if (balance > 50000):
interest += 0.1
else:
interest += 0.05
balance = balance + (1 * interest)
![Page 87: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/87.jpg)
def calculate_interest(balance):
interest = 0.1
if (balance > 1000):
interest = 0.05
if (balance > 50000):
interest += 0.1
else:
interest += 0.05
balance = balance + (1 * interest)
balance > 1000
balance > 50000interest += 0.1
interest += 0.1
interest += 0.05
balance = balance +
(1*interest)
![Page 88: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/88.jpg)
def calculate_speed(speed, drag):
if (speed > 100):
if (drag > 50):
return speed - (drag * 2)
else:
speed -= drag
else:
if (drag > 50):
speed -= (drag * 0.5)
return speed - (drag * 0.3)
![Page 89: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/89.jpg)
def calculate_speed(speed, drag):
if (speed > 100):
if (drag > 50):
return speed - (drag * 2)
else:
speed -= drag
else:
if (drag > 50):
speed -= (drag * 0.5)
return speed - (drag * 0.3)
speed > 100
drag > 50 drag > 50
return speed - (drag * 2)
speed -= drag
speed -= (drag * 0.5)
return speed - (drag * 0.3)
![Page 90: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/90.jpg)
![Page 91: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/91.jpg)
![Page 92: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/92.jpg)
coverage.py
pip install coverage
Specify what source files to monitor
coverage run my_program.py
Output a report
coverage report -m
$ coverage report -m
Name Stmts Miss Cover Missing
----------------------------------------------------
my_program.py 20 4 80% 33-35, 39
my_other_module.py 56 6 89% 17-23
----------------------------------------------------
TOTAL 76 10 87%
![Page 93: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/93.jpg)
coverage.py
pip install coverage
Specify what source files to monitor
coverage run my_program.py
Output a report
coverage report -m
![Page 94: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/94.jpg)
pytest-cov
Plugin for pytest
pip install pytest-cov
Specify what source files to monitor
--cov=game_of_life
$ pytest test_game_of_life.py --cov=game_of_life
=============== test session starts ==============
plugins: cov-2.4.0
collected 1 items
test_game_of_life.py .
----- coverage: platform linux2, python 2.7.12-final-0 ----
Name Stmts Miss Cover
-------------------------------------
game_of_life.py 39 26 33%
================ 1 passed in 0.03 seconds ================
![Page 95: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/95.jpg)
pytest-cov
$ pytest test_game_of_life.py --cov=game_of_life--cov-report=html
Plugin for pytest
pip install pytest-cov
Specify what source files to monitor
--cov=game_of_life
![Page 96: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/96.jpg)
Is code coverage a useful metric?
![Page 97: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/97.jpg)
Is code coverage a useful metric?
Bad (even broken) tests can increase coverage
False sense of security?
Using coverage as a shipping gate leads to bad practices
Can give helpful cues about weak spots in your tests
Expect good coverage, don’t require it
![Page 98: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/98.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Automated Tools
Manual
![Page 99: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/99.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Code Coverage
Automated Tools
Manual
![Page 100: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/100.jpg)
Defect metrics
Number of defects in each state
Rate of opening vs closing
Average age of a defect
Per engineer/component counts...
![Page 101: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/101.jpg)
Defects closed
Defect metrics
Time
Defects opened
![Page 102: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/102.jpg)
Defect metrics
Need info DeferredFixed As designed
Defect resolution
![Page 103: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/103.jpg)
Are defect counts a good metric?
Useful for understanding velocity
Can be used to identify problems
Targets are easy to misuse
![Page 104: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/104.jpg)
“Test metrics can be useful if used properly or harmful if used poorly. Understand the metrics and who they might be useful to.”
![Page 105: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/105.jpg)
“Test metrics can be useful if used properly or harmful if used poorly. Understand the metrics and who they might be useful to.”
Steve Upton
![Page 106: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/106.jpg)
Jon Tilt@jontilt
![Page 107: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/107.jpg)
“Accurate and actionable information is more useful than precise and detailed information”
Jon Tilt
![Page 108: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/108.jpg)
http://confidencemap.mybluemix.net/index.html
![Page 109: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/109.jpg)
![Page 110: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/110.jpg)
![Page 111: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/111.jpg)
![Page 112: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/112.jpg)
![Page 113: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/113.jpg)
![Page 114: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/114.jpg)
![Page 115: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/115.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Automated Tools
Manual
![Page 116: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/116.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Automated Tools
Manual
![Page 117: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/117.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Automated Tools
Manual
![Page 118: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/118.jpg)
Testing for Data Science
![Page 119: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/119.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Automated Tools
Manual
![Page 120: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/120.jpg)
Business Facing
Technology Facing
Sup
porti
ng th
e te
amC
ritique the product
Auto/Manual
Unit testing
Automated Tools
Manual
![Page 121: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/121.jpg)
![Page 122: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/122.jpg)
If a model looks too good to be true, then generally it is
![Page 123: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/123.jpg)
Chapter 15: Model Performance Evaluation
Cross-validation
Error Rate
![Page 125: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/125.jpg)
Books
![Page 126: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/126.jpg)
Books
![Page 127: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/127.jpg)
Essential readinghttps://dannorth.net/introducing-bdd/
![Page 128: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/128.jpg)
Good readinghttp://www.exampler.com/testing-com/writings/coverage.pdf
http://www.satisfice.com/articles/what_is_et.shtml
![Page 129: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/129.jpg)
Referenceshttp://sunnyday.mit.edu/accidents/Ariane5accidentreport.html
http://www.intel.de/content/dam/www/public/us/en/documents/specification-updates/desktop-6th-gen-core-family-spec-update.pdf
http://lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/
http://www.exampler.com/old-blog/2003/08/21.1.html#agile-testing-project-1
http://www.gao.gov/assets/220/215614.pdf
http://www.mysmu.edu/faculty/davidlo/papers/saner15-coverage.pdf
https://github.com/cucumber/cucumber-ruby/commit/a9398c13d5a53b71e582050de92ae49e3524412c
![Page 130: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/130.jpg)
References cont.https://shkspr.mobi/blog/2014/03/introducing-corkr-at-nhtg14/
https://shkspr.mobi/blog/copyright-terence-eden/
http://orlando.opensauce.it/corkr/
![Page 131: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/131.jpg)
Code exampleshttps://technobeans.com/2012/04/16/5-ways-of-fibonacci-in-python/
https://wiki.python.org/moin/SimplePrograms
![Page 132: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/132.jpg)
Bad thingshttp://stackoverflow.com/a/22291032/1214161
http://testingbasicinterviewquestions.blogspot.de/2015/03/best-difference-between-alpha-and-beta.html
![Page 133: DSR Testing (Part 2)](https://reader033.fdocuments.in/reader033/viewer/2022051300/58a2ab411a28ab0d0a8b7199/html5/thumbnails/133.jpg)
Image creditsUS Launch Report