Unit Testing Methods White-box Testing Black-box Testing Extensions

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Unit Testing Methods White-box Testing Black-box Testing Extensions

  • Slide 1
  • Unit Testing Methods White-box Testing Black-box Testing Extensions
  • Slide 2
  • Software Testing Methods Testing Goals Reveal failures/faults/errors Locate failures/faults/errors Show system correctness Within the limits of optimistic inaccuracy Improve confidence that the system performs as specified (verification) Improve confidence that the system performs as desired (validation) Program testing can be used to show the presence of bugs, but never to show their absence [Dijkstra]
  • Slide 3
  • Unit Testing Techniques Unit Testing checks that an individual program unit (subprogram, object class, package, module) behaves correctly. Static Testing - testing a unit without executing the unit code Dynamic Testing - testing a unit by executing a program unit using test data
  • Slide 4
  • Black-Box Testing Black box testing Purpose: tests the overall functional behavior Drawback: may not reveal cause of the defect Method: Treat the module as a classic input/output module; when you provide it with certain inputs it should provide certain outputs requirements events input output
  • Slide 5
  • Black-box Testing (nave approach) Getting started: Review documentation for expected output Example: A Fibonacci sequence starts like this: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89 Unit Test Procedure: Ask the user to provide a number from 0 to 11 inclusive Call the Fib function Compare result value to the corresponding value in the above list You go try it this way
  • Slide 6
  • Specification-Based Testing Black-box Testing is more formally known as specification- based testing Specification-based testing Test cases designed, selected, and run based on specifications Use specifications to derive test cases Requirements Design Function signature Based on some kind of input domain The approach is not nave, but considers the range of data inputs Typical values Boundary values Special cases Invalid input values (negative testing)
  • Slide 7
  • Equivalence Partitioning Equivalence partitioning is an approach to black box testing that divides the input domain of a program into classes of data from which test cases can be derived. Math review: What is an equivalence class? Equivalence classes are based on equivalence relations: For all x, y, z in set T, the following hold: -Reflexivity: x ~ x -Symmetric:x ~ yiffy ~x -Transitivity: if x ~ y and y ~ z, then x ~ z Examples: = has the same astrological sign Non-examples: > is taller than How does this help? -Define equivalence as these data are expected to produce the same result -We only need to test with 1 representative data element from that class
  • Slide 8
  • Equivalence partitioning Example: Suppose a program computes the value of the function. What are the equivalence classes? X < = -2valid -2 < X < 1invalid X >= 1valid Test cases would be selected from each equivalence class. Cons: Lacks causality: 2 inputs may expect to produce the same output for entirely different reasons No deterministic way to define the classes; often difficult for non-mathematical operations or multi-dimensional inputs Pros: Often one can intuitively identify representative data, so there really is no reason not to try this. Can greatly reduce the number of test cases you need to run
  • Slide 9 = 11, 1.1,100, 2 31 -1">
  • Boundary Value Analysis Boundary Value Analysis - black box technique where test cases are designed to test the boundary of an input domain. Studies have shown that more errors occur on the "boundary" of an input domain than in the "center". Commonly referred to as a type of edge case Boundary value analysis complements and can be used in conjunction with equivalence partitioning. Example revisited: After using equivalence partitioning to generate equivalence classes, boundary value analysis would dictate that the boundary values of the three ranges be included in the test data. That is, we might choose the following test cases (for a 32 bit system): X = 11, 1.1,100, 2 31 -1
  • Slide 10
  • Example: Possible Boundaries Basis: Array length Empty,1, 2, small, or large number of elements Basis: Position of minimum score Smallest element first, second, in middle, in last Basis: Number of minima Unique minimum, a few minima, all minima Input domain: float[] Basis: array length one smallemptylarge Input domain: float[] Basis: position of minimum somewhere in middle first last Input domain: float[] Basis: number of minima All data equal1 minimum 2 minima two second
  • Slide 11
  • Structural (White-Box) Testing White box testing Structural testing Test cases designed, selected, and run based on structure of the source code Scale: tests the nitty-gritty (line-by-line) Drawbacks: need access to the source Use source code to derive test cases Build a graph model of the system Control flow Data flow Choose test cases that guarantee different types of coverage Node coverage Edge coverage Loop coverage Condition coverage Path coverage... our goal is to ensure that all statements and conditions have been executed at least once...
  • Slide 12
  • Exhaustive Testing loop < 20 X There are 10 possible paths! If we execute one test per millisecond, it would take 3,170 years to test this program!! 14
  • Slide 13
  • Selective Testing loop < 20 X Selected path
  • Slide 14
  • Example 1 float findAverage(float[] scores) { 2 float total = 0, min = MAX_FLOAT, min2 = MAX_FLOAT; 3 for (inti = 0 ; i
  • Other Coverage Criteria Loop coverage Select test cases such that every loop boundary and interior is tested Boundary: 0 iterations Interior: 1 iteration and> 1 iterations This was missing from our tests, and could have uncovered bugs! Watch out for nested loops Less precise than edge coverage Condition coverage Select test cases such that all conditions are tested if (min != MAX_FLOAT && min2 != MAX_FLOAT) More precise than edge coverage Our edge coverage test doesnt say why 12 14 is traversed
  • Slide 18
  • Other Coverage Criteria Path coverage Select test cases such that every pathis traversed Loops are a problem Consider example on earlier slide what is scores.length? Suppose it was 5, how many test cases would you need? 0, 1, average, max iterations Why are paths important? Because previous instructions may cause side effects to runtime state that require you to verify later instructions in light of those side effects Most thoroughbut is it feasible? Not really, so we try instead to compute the set of linearly independent paths within the graph Called the basis paths; more to come in Metrics (McCabe)
  • Slide 19
  • Unit Testing: OO Perspective Can apply per method But this is not considered a best practice as methods represent conceptual behaviors and as such may not be used in isolation. Unit test an entire scenario implemented as a collection of objects exchanging messages Think UML sequence diagram Use-case driven unit tests often happen at the component level OO white box testing involves verifying the object traverses a defined set of states In response to stimuli (messages) - reactive systems Verify object is used in the manner designed State machines a common modeling approach for this Polymorphism creates unique challenges Encapsulation is violated Different languages support different semantics
  • Slide 20
  • Unit Testing: Process Perspective Unit testing is popular (again) Promoted by XP/Agile methodologies Provides continuous feedback (code a little, test a little) Testing is good so we will do it all the time Has surpassed inspections in popularity Data still says inspections are better Unit tests struggle to provide white box capability Process Terminology Test class - set of input data that is identified for testing Test case - specific data chosen for testing a test class. Test suite - set of test cases that serve a particular goal Exhaustive testing - test all logical paths in a module
  • Slide 21
  • TDD: An Alternative View Test-Driven Development (TDD) Write your tests first! A promoted practice for XP code a little, test a little becomes test a little, code a little Derives from your user story acceptance tests
  • Slide 22
  • Summary: Unit Testing White-box Structural evaluation Coverage difficult set a target! Black-box Treats implementation of function as unknown Test for valid outputs given a range of inputs Science based on domain/range of the inputs Unit-level testing process Unit testing now considered a very agile way of coding. TDD a great way to ensure you verify& validate as you go. Many developers struggle to write complete unit tests. Many developers struggle to maintain their unit tests.