What test coverage mean to us
-
Upload
josephyaoruozhou -
Category
Technology
-
view
517 -
download
0
description
Transcript of What test coverage mean to us
![Page 1: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/1.jpg)
What Test Coverage Mean to us?
Joseph Yao
Tuesday, July 9, 13
![Page 2: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/2.jpg)
Who I am?
n Agile Coach at Odd-e
n Father, Husband...
n Software Craftsman
n Basketball, Board Game...
Tuesday, July 9, 13
![Page 3: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/3.jpg)
Agenda
n Bias and Truth of Test Coverage
n Classical Test Coverage
n Mutation Test Coverage
n TDD and Test Coverage
Tuesday, July 9, 13
![Page 4: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/4.jpg)
Bias of Test Coverage
Tuesday, July 9, 13
![Page 5: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/5.jpg)
Truth of Test Coverage
I expect a high level of coverage. Sometimes managers require one. There's a subtle difference.-- Brian Marick
http://martinfowler.com/bliki/TestCoverage.html
Tuesday, July 9, 13
![Page 6: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/6.jpg)
Truth of Test Coverage
Tuesday, July 9, 13
![Page 7: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/7.jpg)
Classical Test Coverage
Tuesday, July 9, 13
![Page 8: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/8.jpg)
Function Coverageint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
Satisfied Test: foo(2, 2)Tuesday, July 9, 13
![Page 9: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/9.jpg)
Function Coverageint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
Tuesday, July 9, 13
![Page 10: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/10.jpg)
Statement Coverageint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
Satisfied Test: foo(2, 2)Tuesday, July 9, 13
![Page 11: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/11.jpg)
Statement Coverageint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
Unsatisfied Test: foo(2, -1)Tuesday, July 9, 13
![Page 12: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/12.jpg)
Decision Coverageint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
T/F
Satisfied Test: foo(2, 2), foo(-1, 2)Tuesday, July 9, 13
![Page 13: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/13.jpg)
Decision Coverageint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
T/F
Unsatisfied Test: foo(2, 2)Tuesday, July 9, 13
![Page 14: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/14.jpg)
Condition Coverageint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
T/F
Satisfied Test: foo(2, 2), foo(2, -1), foo(-1, -1)
T/F
Tuesday, July 9, 13
![Page 15: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/15.jpg)
Condition Coverageint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
T/F
Unsatisfied Test: foo(2, 2), foo(-1, 2)
T/F
Tuesday, July 9, 13
![Page 16: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/16.jpg)
Classical Test Coverage
Tuesday, July 9, 13
![Page 17: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/17.jpg)
Classical Test Coverage
Tuesday, July 9, 13
![Page 18: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/18.jpg)
Expectation of Automated Test
Tuesday, July 9, 13
![Page 19: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/19.jpg)
Expectation of Automated Test
Any Possible “Bad” Code
Change can be Detected
Tuesday, July 9, 13
![Page 20: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/20.jpg)
Mutation Test Coverage
Tuesday, July 9, 13
![Page 21: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/21.jpg)
Why Mutation Test
Tuesday, July 9, 13
![Page 22: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/22.jpg)
Redundant Code?int foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } sideEffect(z); return z;}
Tuesday, July 9, 13
![Page 23: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/23.jpg)
Redundant Code?int foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } sideEffect(z); return z;}
int foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } sideEffect(z); return z;}
Tuesday, July 9, 13
![Page 24: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/24.jpg)
Redundant Code?int foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } sideEffect(z); return z;}
int foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } sideEffect(z); return z;}
n Missing test to prove the sideEffect call is needed
n If code is redundant, it can’t be learnt via any classical test coverage
Tuesday, July 9, 13
![Page 25: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/25.jpg)
Conditional Boundary Mutatorint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
Condition Coverage Satisfied Test:
assertEquals(2, foo(2, 2))
assertEquals(0, foo(2, -1))
assertEquals(0, foo(-1, 2))
Tuesday, July 9, 13
![Page 26: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/26.jpg)
Conditional Boundary Mutatorint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
int foo (int x, int y){ int z = 0; if ((x>0) && (y>=0)) { z = x; } return z;}
Condition Coverage Satisfied Test:
assertEquals(2, foo(2, 2))
assertEquals(0, foo(2, -1))
assertEquals(0, foo(-1, 2))
Tuesday, July 9, 13
![Page 27: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/27.jpg)
Conditional Boundary Mutatorint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
int foo (int x, int y){ int z = 0; if ((x>0) && (y>=0)) { z = x; } return z;}
Condition Coverage Satisfied Test:
assertEquals(2, foo(2, 2))
assertEquals(0, foo(2, -1))
assertEquals(0, foo(-1, 2))
Test to cover this Mutator:
assertEquals(2, foo(2, 2))
assertEquals(0, foo(2, 0))
assertEquals(0, foo(-1, 2))
Tuesday, July 9, 13
![Page 28: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/28.jpg)
Conditional Boundary Mutatorint foo (int x, int y){ int z = 0; if ((x>0) && (y>0)) { z = x; } return z;}
int foo (int x, int y){ int z = 0; if ((x>0) && (y>=0)) { z = x; } return z;}
Condition Coverage Satisfied Test:
assertEquals(2, foo(2, 2))
assertEquals(0, foo(2, -1))
assertEquals(0, foo(-1, 2))
n Test not prevent “bad” code change to happen
Test to cover this Mutator:
assertEquals(2, foo(2, 2))
assertEquals(0, foo(2, 0))
assertEquals(0, foo(-1, 2))
Tuesday, July 9, 13
![Page 29: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/29.jpg)
Other Mutatorsn Negate Conditionals Mutator
n Math Mutator
n Increments Mutator
n Invert Negatives Mutator
n Inline Constant Mutator
n Return Values Mutator
n Void Method Calls Mutator
n Non Void Method Calls Mutator
n Constructor Calls Mutator
Tuesday, July 9, 13
![Page 30: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/30.jpg)
Test Driven Development
Tuesday, July 9, 13
![Page 31: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/31.jpg)
TDD and Test Coverage
n You can most likely ignore Classical Test Coverage
n You can’t ignore Mutation Test Coverage
n If not TDD, then you can learn a lot from both
Classical and Mutation Test Coverage
Tuesday, July 9, 13
![Page 32: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/32.jpg)
PokerHands Kata
Tuesday, July 9, 13
![Page 33: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/33.jpg)
True Mutation
Tuesday, July 9, 13
![Page 34: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/34.jpg)
Problematic Test
Tuesday, July 9, 13
![Page 35: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/35.jpg)
False Mutation
Tuesday, July 9, 13
![Page 36: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/36.jpg)
Reference
n Martin Fowler’s blog http://martinfowler.com/bliki/TestCoverage.html
n Mutation Test Wikipedia http://en.wikipedia.org/wiki/Mutation_testing
n Mutation Test Analysis Report http://crestweb.cs.ucl.ac.uk/resources/mutation_testing_repository/TR-09-06.pdf
Tuesday, July 9, 13
![Page 37: What test coverage mean to us](https://reader034.fdocuments.in/reader034/viewer/2022051609/5463ad29b4af9f493f8b4608/html5/thumbnails/37.jpg)
Q & A
新浪微博:姚若舟微信:yaoruozhou
土豆(Kata视频):姚若舟Github: JosephYao
Tuesday, July 9, 13