An overview to Software Testing

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of An overview to Software Testing

  • Software Quality Assurance A general walkthroughWhat is it?What are its objectives?What do QAs do?

  • ContentEssenceTerminologyClassificationUnit, System BlackBox, WhiteBox

  • DefinitionGlen MyersTesting is the process of executing a program with the intent of finding errors

  • Objective explainedPaul JorgensenTesting is obviously concerned with errors, faults, failures and incidents. A test is the act of exercising software with test cases with an objective ofFinding failureDemonstrate correct execution

  • A Testing Life CycleRequirementSpecsDesignCodingTestingFault ResolutionFaultIsolationFaultClassificationErrorFaultFaultFaultErrorErrorincidentFix

  • Verification versus ValidationVerification is concerned with phase containment of errorsValidation is concerned about the final product to be error free

  • Relationship program behaviors

  • Classification of TestThere are two levels of classificationOne distinguishes at granularity levelUnit levelSystem levelIntegration levelOther classification (mostly for unit level) is based on methodologiesBlack box (Functional) TestingWhite box (Structural) Testing

  • Relationship Testing wrt BehaviorProgram Behaviors

  • Cont2, 5Specified behavior that are not tested1, 4Specified behavior that are tested3, 7Test cases corresponding to unspecified behavior

  • Cont2, 6Programmed behavior that are not tested1, 3Programmed behavior that are tested4, 7Test cases corresponding to un-programmed behaviors

  • InferencesIf there are specified behaviors for which there are no test cases, the testing is incompleteIf there are test cases that correspond to unspecified behaviorsEither such test cases are unwarrantedSpecification is deficient (also implies that testers should participate in specification and design reviews)

  • Test methodologiesFunctional (Black box) inspects specified behaviorStructural (White box) inspects programmed behavior

  • When to use whatFew set of guidelines availableA logical approach could bePrepare functional test cases as part of specification. However they could be used only after unit and/or system is available.Preparation of Structural test cases could be part of implementation/code phase.Unit, Integration and System testing are performed in order.

  • Unit testing essenceApplicable to modular designUnit testing inspects individual modulesLocate error in smaller regionIn an integrated system, it may not be easier to determine which module has caused faultReduces debugging efforts

  • Test cases and Test suitesTest case is a triplet [I, S, O] whereI is input dataS is state of system at which data will be inputO is the expected outputTest suite is set of all test casesTest cases are not randomly selected. Instead even they need to be designed.

  • Need for designing test casesAlmost every non-trivial system has an extremely large input data domain thereby making exhaustive testing impracticalIf randomly selected then test case may loose significance since it may expose an already detected error by some other test case

  • Time for an exercise Give me all possible test cases for this object:

  • Black box testingEquivalence class partitioningBoundary value analysisComparison testing

  • Equivalence Class PartitioningInput values to a program are partitioned into equivalence classes. Partitioning is done such that:program behaves in similar ways to every input value belonging to an equivalence class.

  • Why define equivalence classes?Test the code with just one representative value from each equivalence class: as good as testing using any other values from the equivalence classes.

  • Equivalence Class PartitioningHow do you determine the equivalence classes?examine the input data. few general guidelines for determining the equivalence classes can be given

  • Equivalence Class PartitioningIf the input data to the program is specified by a range of values:e.g. numbers between 1 to 5000. one valid and two invalid equivalence classes are defined. 15000

  • Equivalence Class PartitioningIf input is an enumerated set of values: e.g. {a,b,c}one equivalence class for valid input values another equivalence class for invalid input values should be defined.

  • ExampleA program reads an input value in the range of 1 and 5000:computes the square root of the input number SQRT

  • Example (cont.)There are three equivalence classes: the set of negative integers, set of integers in the range of 1 and 5000, integers larger than 5000. 15000

  • Example (cont.)The test suite must include:representatives from each of the three equivalence classes:a possible test suite can be: {-5,500,6000}.15000

  • Boundary Value AnalysisSome typical programming errors occur: at boundaries of equivalence classes might be purely due to psychological factors. Programmers often fail to see:special processing required at the boundaries of equivalence classes.

  • Boundary Value AnalysisProgrammers may improperly use < instead of
  • ExampleFor a function that computes the square root of an integer in the range of 1 and 5000:test cases must include the values: {0,1,5000,5001}.


  • Acceptance testingFormal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system. Alpha testingSimulated or actual operational testing by potential users/customers or an independent test team at the developers site, but outside the development organization. Alpha testing is often employed for off-the-shelf software as a form of internal acceptance testing. Back-to-back testingTesting in which two or more variants of a component or system are executed with the same inputs, the outputs compared, and analyzed in cases of discrepancies. Beta testingOperational testing by potential and/or existing users/customers at an external site not otherwise involved with the developers, to determine whether or not a component or system satisfies the user/customer needs and fits within the business processes. Beta testing is often employed as a form of external acceptance testing for off-the-shelf software in order to acquire feedback from the market.

  • ContinuedBlack-box testingTesting, either functional or non-functional, without reference to the internal structure of the component or system.Boundary valueAn input value or output value which is on the edge of an equivalence partition or at the smallest incremental distance on either side of an edge, for example the minimum or maximum value of a range.Boundary value analysisA black box test design technique in which test cases are designed based on boundary values. Branch testingA white box test design technique in which test cases are designed to execute branches.DefectA flaw in a component or system that can cause the component or system to fail to perform its required function, e.g. an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system.

  • ContinuedFunctional testingTesting based on an analysis of the specification of the functionality of a component or system.Integration testingTesting performed to expose defects in the interfaces and in the interactions between integrated components or systems. Load testingA test type concerned with measuring the behavior of a component or system with increasing load, e.g. number of parallel users and/or numbers of transactions to determine what load can be handled by the component or system. Monkey testingTesting by means of a random selection from a large range of inputs and by randomly pushing buttons, ignorant on how the product is being used.Recoverability testingThe process of testing to determine the recoverability of a software product.Regression testingTesting of a previously tested program following modification to ensure that defects have not been introduced or uncovered in unchanged areas of the software, as a result of the changes made. It is performed when the software or its environment is changed.

  • ContinuedSeverityThe degree of impact that a defect has on the development or operation of a component or system.Smoke testA subset of all defined/planned test cases that cover the main functionality of a component or system, to ascertaining that the most crucial functions of a program work, but not bothering with finer details. A daily build and smoke test is among industry best practices.Test automationThe use of software to perform or support test activities, e.g. test management, test design, test execution and results checking.Test case specificationA document specifying a set of test cases (objective, inputs, test actions, expected results, and execution preconditions) for a test item. Test design specificationA document specifying the test conditions (coverage items) for a test item, the detailed test approach and identifying the associated high level test cases.Test environmentAn environment containing hardware, instrumentation, simulators, software tools, and other support elements needed to conduct a test. Test harnessA test environment comprised of stubs and drivers needed to execute a test.Test logA chronological record of relevant details about the execution of tests.

  • Questions?