Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro...
Transcript of Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro...
![Page 1: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/1.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Testing
Kurt Schmidt
Dept. of Computer Science, Drexel University
December 5, 2018
![Page 2: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/2.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
A QA engineer walks into a bar:A QA engineer runs into a bar:A QA engineer skips into a bar:
She orders 1 beerShe orders -1 beersHe orders 99,999 beersIt orders sfojbeerkiwHe orders ’0 ; select id, password from users ; dropbeers ; commit’
Opening day a customer walks in, asks for the bathroom.The bar catches fire, everybody dies.
![Page 3: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/3.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Intro
![Page 4: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/4.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Testing and Debugging
Testing “...a determined, systematic attempt to break aprogram that you think is working.”
Running a program with the intent offinding bugs“...testing can demonstrate the presenceof bugs, but not their absence.”
Debugging Finding the cause of an error, and fixing it.
![Page 5: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/5.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Goals of Testing
Discover bugs, not to show that program worksReduce the risk of failure to an acceptable levelDesigning a test before writing code is a great way toreduce bugs
![Page 6: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/6.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Complete Testing
Complete testing coverage is generally not possibleOr, not practical
E.g., consider a program that takes an input of 10characters
280 distinct inputsAt 1µs/test, would take more than twice the age of theuniverse
![Page 7: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/7.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Testing Caveats
People naturally assume what they do is correctPeople by nature overlook minor deficiencies in theirown workEasy to overlook or ignore bad resultsEasy to choose only test cases that show the program“works”It’s useful to get another’s help
![Page 8: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/8.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Software Testing Myths
Really good programmers don’t have bugsTesting implies an admission of failureTesting is a punishment for our errorsTesting can be avoided if we
ConcentrateUse OO methodsUse a good programming language
![Page 9: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/9.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Software Testing Reality
Humans make mistakesEspecially when creating complex artifacts
Even good programs have 1-3 bugs per 100 lines ofcodePeople who claim they write bug-free code likelyhaven’t coded much
![Page 10: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/10.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Defensive Programming
Anticipate potential problemsDesign, code the system so problems are accountedfor, or detected as early as possible
Defensive design – Minimise confusion due to complexityDefensive coding – Take steps to localise problems
![Page 11: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/11.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Defensive Design
Simplicity of designEncapsulationDesign with error in mindPrototype, walk-throughMake all assumptions and conditions explicitly
![Page 12: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/12.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Encapsulation
Minimise coupling (dependencies) between objects1
Provide a sufficient interfaceHide all data behind the interfaceHave enough functionality so that the client needn’taccess the data directly
1Loosely; i.e., a collection of related data, not necessarily an instanceof a class
![Page 13: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/13.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Designing With Error in Mind
Error handling is often added as an afterthoughtShould be part of the interface
Decided before/as you code
Ask “What if?” often
![Page 14: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/14.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Design Reviews
Show the design to another programmer, or leadDiscussions and critiques are an excellent way to learnIt is much easier to see another’s mistakes, andassumptions
Another pair of eyeballs is always helpful
![Page 15: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/15.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Pre- and Post-Conditions
Pre-conditions Assumptions that are made by a function ormethod upon entry, of necessity or efficiency
Units or state of argumentsState of the objectState of globals
Post-condition Anything a function guarantees upon exit (ifpre-conditions were met
Return valueAny side-effects
![Page 16: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/16.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Class Invariants
Conditions on an object which are always trueFor a Vector V of length n, elements V0 through Vn−1are validA Fraction might always be in lowest terms
Conditions public (interface) functions can assumeupon entryMust guarantee are true upon exit
Note, this does not necessarily apply to helper functions
![Page 17: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/17.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Evolutionary Programming
Compile and test your code oftenImplementation is bottom-up
Write small piecesTest themGather into bigger pieces
Evolve (and test) your program by writing modules,using stubs
![Page 18: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/18.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Testing
![Page 19: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/19.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
General Testing Guidelines
A necessary part of a test case is the expectedoutput/behaviorTest border conditionsTest error conditionsCheck that the program doesn’t do what it shouldn’t(harder to impossible)Never be the last to test your own codeTesting is an art; your skills will develop
![Page 20: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/20.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Functional vs. Structural Testing
Functional (black box) testing:Implementation details are invisibleCode is tested against its specs – look at advertisedbehavior
Inputs, outputsSide effectsError codes and exceptions
Cases can be written before the code is writtenStructural (clear box) testing:
Tester must be familiar with the codeExercise all control paths, loop boundariesTools such as gcov might be helpful
![Page 21: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/21.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Integration, Unit, System, and AcceptanceTesting
These labels often get in the way of talking abouttesting1
I don’t want to confuse this discussion this term“Black-box” does not imply “unit” testing“Clear-box” does not imply “integration” testing
1Mike Kelly, searchsoftwarequality.techtarget.com/
![Page 22: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/22.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
The assert statement
Used during development, testing and debugging totest programmer’s assumptions
Not to be used in released code
#include <assert.h>...assert( x!=0 ) ;
If condition failsPrint conditionPrint a line numberDump the program
To turn it off:
$ gcc -DNDEBUG ...
![Page 23: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/23.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Debugging
![Page 24: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/24.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Types of Bugs
SyntacticDesignLogic
InterfaceMemory
![Page 25: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/25.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
How to Find a Problem
ThinkI’ll spend a few minutes with print/debug statementsI’ll move to a debugger before too longIf you reach an impasse, sleep on itIf you reach an impasse, show it to someone else
![Page 26: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/26.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Why Use a Debugger?
Debuggers are very powerfulDesk checking code can be tedious and error-pronePrint/Debug statements might require re-compilation
Error-proneLeaves a mess to clean up
![Page 27: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/27.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Basic Common Debugger Functions
BreakpointsSet at line or functionCan be conditional
Run programExecute line at a timeRun to next breakpointMove execution pointer aroundSet command-line argumentsSee stack trace
DataEvaluate any variable, expressionModify values in memory
![Page 28: Testing Kurt Schmidt Intro Testing - Drexel CCI · 2018-12-05 · Testing Kurt Schmidt Intro Testing Assertions Debugging Integration, Unit, System, and Acceptance Testing These labels](https://reader033.fdocuments.in/reader033/viewer/2022053019/5f27c6e14a3ce811d62a8a53/html5/thumbnails/28.jpg)
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Think Before Repairing Errors
Be careful you fix the problemDon’t simply address the symptom
Remember, don’t fix bad code, rewrite it!