3-White Box Testing

8/12/2019 3-White Box Testing

1/47

No part of this presentation may be reproduced without prior written permissionfrom authors, copyright owner and publisher of the book

White Box Testing(Chapter 3)


2/47

3/2/2014 White Box Testing Slide : 2

Objectives

To review the V Model and set the context for differenttypes of testsTo focus on white box testing

To distinguish between different types of white boxtestingTo place special emphasis on Code reviews / inspections

Methods of code coverage and relationship toquality Code complexity


3/47


Bending the Waterfall

High-LevelDes ign

Low-LevelDes ign

Development

Testing

Unit Testing

ComponentTesting

IntegrationTesting

Sys temTesting

AcceptanceTesting

SoftwareRequi rements

OverallBus inessRequirements


4/47


OverallBus inessRequirements

SoftwareRequi rements

High-LevelDes ign

Low-LevelDes ign

Development Unit TestExecut ion

ComponentTestExecut ion

IntegrationTest Execution

Sys temTest Execution

AcceptanceTest Execution

AcceptanceTest Design

Sys temTest Design

IntegrationTest Design

ComponentTest Design

Unit TestDes ign

V Model


5/47


Principles of the V ModelDo an early test design.Involve the right people in the design of the right typeof tests.Make the tests reflect closely the steps on the left-hand side of the V.Unit testing White box testing (focus of this chapter) Black box testing (focus of the next chapter)


6/47


White Box Testing

Requires access to codeLooks at the program code and performs testing bymapping program code to functionality


7/473/2/2014 White Box Testing Slide : 7

Why White Box Testing?The program code truly represents what the programactually does and not just what it is intended to do!It minimizes delay between defect injection and defectdetection (i.e., does not postpone detection to be

done by someone else).Can catch obvious programming errors that do notnecessarily map to common user scenarios (e.g.,divide by zero).



Types of White Box Testing



Static TestingInvolves only the source code and not the executables orbinariesDoes not involve executing the program on a machine butrather humans going through it or the usage of specializedtools

Some of the things tested by static testing: Whether the code works according to the functional

requirement Whether the code has been written in accordance with

the design developed earlier in the project life cycle

Whether the code follows all applicable standards Whether the code for any functionality has been missed

out Whether the code handles errors properly



Static Testing by Humans

Humans can find errors that computers cant.Multiple humans can provide multiple perspectives.

A human evaluation can compare the code against thespecifications more thoroughly.It can detect multiple defects at one go.It reduces downstream, inline pressure.



Static Testing TypesDifferent types Desk checking of the code Code walkthrough Code review

Code inspectionQA vs QC argumentIncreasing the involvement of peopleMore variety of perspectivesIncreasing formalismIncreasing the likelihood of identifying more complexdefects



Desk Checking

Author informally checks the code against thespecifications and corrects defects found.No structured method or formalism is required toensure completeness.No log or checklist is maintained.It relies only on the thoroughness of the author.It may suffice for obvious programming errors butmay not be effective for incomplete / misunderstoodrequirements.



Advantages and Disadvantages ofDesk Checking

Advantages The programmer knows the code and

programming language well and hence is bestsuited to read the program

Less scheduling and logistics overheads Reduces delay in defect detection and correctionDisadvantages

Person dependent, not scalable or reproducible Tunnel visioning of developers Developers prefer writing new code and dont like

testing



Code Walkthrough

Group oriented (as against desk checking)Brings in multiple perspectivesMultiple specific roles (author, moderator, inspector,etc.), as discussed in Fagan Inspection



Formal Inspection / FaganInspection

Group-oriented activityHighly formal and structuredHas specific roles

Requires thorough preparation



Roles in a formal inspection Author

Author of the work product Makes available the required material to the reviewers Fixes defects that are reported

Moderator Controls the meeting(s)

Inspectors (reviewers) Prepare by reading the required documents Take part in the meeting(s) and report defects

Scribe Takes down notes during the meeting Assigned in advance by turns Can participate to review to the extent possible Documents the minutes and circulates them to participants



Process in a Fagan Inspection

Typical documents circulated: Program code Design / program specifications

SRS (if needed) Any applicable standards (e.g., coding standards) Any necessary checklists (e.g., code review

checklist)



Meetings in a Fagan InspectionPreliminary meeting (optional) Author explains his / her perspective Makes available the necessary documents Highlights concern areas, if any, for which review comments

are sought

Defect Logging Meeting All come prepared! Moderator goes through the code sequentially Each reviewer comes up with comments Comments / defects categorized as defect / observation,

major / minor, systemic / mis -execution Scribe documents all the findings and circulates them

Follow-up meeting (optional) Author fixes defects If required, a follow-up meeting is called to verify

completeness of fixes


19/47



Combinations of various methods

Prioritizing various programs High priority / important codes subject to formal

inspection, medium ones for walkthrough and lowones for desk checking

Deciding between 100% coverage and partialcoverage



Static Analysis ToolsHelp identify / are used:

Whether there are unreachable codes (usage of gotostatements sometimes creates this situation; there could beother reasons too)Variables declared but not usedMismatch in definition and assignment of values to variablesIllegal or error-prone type-casting of variablesUse of non-portable or architecture-dependent programmingconstructsMemory allocated but not having corresponding statements for

freeing up memoryFor calculation of cyclomatic complexity

As an extension of compilers (lint, compiler flag drivenchecking)



Use of a Code Review Checklist

Never leave home without it! The code review checklist aids in organizationallearning by indicating what to look for.It should be kept current as we learn.

A discussion of a sample checklist is given onPage 54.


23/47


Structural TestingDone by running the executable on the machineEntails running the product against some predefinedtest cases and comparing the results against theexpected results

Designing test cases and test data to exercisecertain portions of codeTypes of structural testing Unit / code functional testing

Code coverage testing Code complexity testing


24/47


Unit / Code Functional Testing

Initial quick checks by developerRemoves obvious errors Done before more expensive checks

Building debug versionsRunning through an IDEDebugging or testing? ( Who cares?! )


25/47


26/47


Types of Code Coverage

Statement coveragePath coverageCondition coverage

Function coverage


27/47


Programming Constructs

Sequential instructionsTwo-way decision statements (if then else)Multi-way decision statements (switchstatements)Loops, like while/do, for, repeat/until, etc


28/47


Testing sequential instructions

Generate test data to make the program enter thesequential block, to make it go through the entireblock (Is this valid?) Asynchronous exceptions Multiple entry points, in non-structured

programmingStatement coverage as a metric: # of statements exercised / Total # of statements


29/47


Testing IF THEN ELSEHave data to test the THEN partHave data to test the ELSE partRelevance of statement coverage?

Total = 0; /* set total to zero */if (code == M) {

stmt1;stmt2;Stmt3;stmt4;Stmt5;stmt6;

Stmt7;}else percent = value/Total*100; /* divide by zero */


30/47


Path Coverage

Statement Coverage may not indicate truecoverage Path Coverage provides better representation Previous example: Having data go through the

THEN part and ELSE part gives only 50%coverage, irrespective of number of statements ineach path

More detailed example of path coverage in next slidePath coverage = # of paths exercised / total # ofpaths in the program


31/47



32/47


Condition Coverage

Further refinement of path coverageMakes sure each constituent condition in a booleanexpression is coveredProtects against compiler optimizationsCondition coverage = # of conditions covered /number of conditions in the program


33/47


Function Coverage

Finds how many functions are covered by test casesHigher level of abstraction; hence possible to achieve100% coverageMore logical than the other types of coverage

Can be derived from RTM Easy to prioritize as they are based on requirementsLeads more naturally to black box testsCan also be a natural predecessor to performance

testing


34/47


Other Uses of Code CoverageMethods

Performance analysis and optimizationResource usage analysisChecking of critical sections

Identifying memory leaksDynamically generated code (e.g., dynamicgeneration of URLs and security checking)


35/47


Code Complexity Testing

Which of the paths are independent? (If paths are notindependent, we may be able to minimize the numberof tests.)Is there an upper bound on the number of tests to beexecuted to ensure that all the statements have beenexecuted at least once?Cyclomati c com plex i ty provides answers to someof these questions.


36/47


Intuitive Meaning of CyclomaticComplexity

Provides an indication of how many independentpaths are there in a program What gives birth to new paths?How can one count the new paths that are born?


37/47


Steps in Determining CyclomaticComplexity

Start with a conventional flow chart.Identify the decision points. Convert complex predicates to simplepredicates (see next slide).


38/47



39/47


Steps in Determining CyclomaticComplexity

Start with a conventional flow chart.Identify the decision points.

Convert complex predicates to simple predicates (see nextslide).

If there are loops, convert the loop conditions to simplepredicates.Combine all the sequential statements into a single node in theflow graph.

When a set of sequential statements are followed by a simplepredicate, combine all the sequential statements and thepredicate check into one node and have two edges emanatingout of this node. Nodes with two emanating edges are calledpredicate nodes.

Make sure that all edges terminate in some node, adding a nodeto represent all the statements at the end of the program.


40/47


Conv ent ional Flow Char tFlow Diagram fo r

Calcula t ing Com plexi ty


41/47


Formal and Intuitive Interpretationof Cyclomatic Complexity

Formal definitions CC = # of predicate nodes + 1 CC = E N + 2 (E = edges, N = nodes)

Intuitive definitions When a condition is introduced, adds a new path See the next two slides


42/47


End

Case (i): A hypothetical program with no decision node

# of ind ependent paths = 1 # of no des, N = 2; # of edg es, E = 1; Cyc lom atic com plex i ty = E N + 2 = 1 # of pred icate no des, P = 0; Cyclo m at ic com plexi ty = P + 1 = 1


43/47


End

Case (ii): Adding one decision node

# of independent paths = 2 # of no des, N = 4;

# of edg es, E = 4; Cyc lom atic com plex i ty = E N + 2 = 2 # of pr edicate no des, P = 1; Cyclo m at ic com plexi ty = P + 1 = 2


44/47


Independent Paths and Basis Sets

An independent path is a path in the flow graph thathas not been traversed before in other paths.

A set of independent paths that cover all the edges iscalled a basis set.Find the basis set and write the test cases to executeall the paths in the basis set.


45/47


Meaning and Interpretation ofCyclomatic Complexity

Complexity What it means

1-10 Well-written code, testability is high,cost / effort to maintain is low

10-20 Moderately complex, testability ismedium, cost / effort to maintain is

medium

20-40 Very complex, testability is low, cost /effort to maintain is high

>40 Not testable, any amount of money /effort to maintain may not be enough


46/47


Challenges in White Box TestingDevelopers not finding time for testing They develop blind spots for their own defectsFully covered code may not correspond to realisticscenarios


47/47

Objectives (Recap)

To review the V Model and set the context for differenttypes of testsTo focus on white box testingTo distinguish between different types of white boxtestingTo place special emphasis on Code reviews / inspections Methods of code coverage and relationship to

quality Code complexity

3-White Box Testing

Documents

Transcript of 3-White Box Testing