CPIS 357 Software Quality & Testing

I.Rehab Bahaaddin Ashary

Faculty of Computing and Information Technology

Information Systems DepartmentFall 2010

Chapter 1 :Basic Concepts and Preliminaries

Lecture Objectives• Central Issue in Testing• Testing Activities• Test Levels• Sources of Information for Test Case Selection• White- Box and Black – Box Testing• Test Planning and Design• Monitoring and Measuring Test Execution• Test Tools and Automation• Test Team Organization and Management

Central Issue in Testing

• Divide the input domain D into D1 and D2• Select a subset D1 of D to test program P• It is possible that D1 exercise only a part P1 of P• Selection of the subset of the input domain must be done in a systematic and careful manner so that the deduction is as accurate and complete as possible. For example. The idea of coverage is considered while selecting the test case.

Figure 1: A subset of the input domain exercising a subset of the program behavior

Testing Activities

• Identify the objective to be tested. (Purpose designing one or more test case)

• Select inputs.(Requirements specification, source code)

• Compute the expected outcome.(High Level understanding both objective and specification)

• Set up the execution environment of the program. (Local/Remote)

• Execute the program.(Different location at different time)

• Analyze the test results.(Pass/Fail/Inconclusive)

Figure 2: Different activities in process testing

Test Levels

• Unit testingIndividual program units, such as procedure, methods, Function and Classes in isolation

• Integration testing Modules are assembled to construct larger subsystem and tested by software developer and test engineer

• System testing Includes wide spectrum of testing such as functionality, Security, Stability, Performance, reliability and load

• Acceptance testing•Customer’s expectations from the system Two types of acceptance testing

•UAT: System satisfies the contractual acceptance criteria•BAT: System will eventually

• Measure the quality of the product rather than searching for the defect.

Figure 3: Development and testing phases in the V model

Test Levels

• New test cases are not designed• Test are selected, prioritized and executed• To ensure that nothing is broken in the new version of the software

Figure 4: Regression testing at different software testing levels

Source of Information for Test Case Selection

•Requirement and Functional Specifications Capturing user needs. Informal or formal manner Examples: plaintext, flowchart, figures, equations, use case, entity relationship diagram or class diagram

• Source Code Internal behavior of the system. By software engineer (for the design details) and programmer add more details Example: Sort Function

• Input and Output DomainExample: factorial (0) =1; factorial (1) =1 ; factorial (n) =n* factorial (n-1)• Operational Profile

Is a quantitative characterization of how the system will be used and guide test engineers in selecting test case input.

• Fault Model Error Guessing Fault Seeding (Fault Injection) Mutation Analysis (Fault Simulation)

White-box and Black-box Testing

White Box Testing ( Structural Test)

Black Box Testing(Functional Test)

• Examines source code with focus on:

Control Flow Data Flow

• Control flow refers to flow of control from one instruction to another.• Data flow refers to propagation of values from one variable or constant to another variable.

• It is applied to individual units of a program.

•Software developers perform structural testing on the individual program units they write

• Examines the program that is accessible from outside

• Applies the input to a program and observe the externally visible outcome.

•It is applied to both an entire program as well as to individual program units.• It is performed at the external interface level of a system.

• It is conducted by a separate software quality assurance group.

Test Planning and Design• The purpose is to get ready and organized for test execution.

• A test plan provides a: Framework

• A set of ideas, facts or circumstances within which the tests will be conducted.

Scope• The test activities and the outline of the domain.

Details of resource needed Effort required Schedule of activities Budget

• During the test design phase the system requirements are critically studied, system features to be tested and the objective and the detailed behavior of test cases are defined.

• Test objectives are identified from different sources.• Each test case is designed as a combination of modular test components called test steps.

•Test steps are combined together to create more complex tests.

Monitoring and Measuring Test Execution

• Metrics for monitoring test execution (concern on test).

• Metrics for monitoring defects (concern on the result from the test).

• Needed to be tracked and analyzed in a periodic basic (daily or weekly).

• Metrics is meaningful if enable management to make decision making which reduce delay, improve quality and lower the cost.• The evaluation done through the following: Evaluate the effectiveness, Evaluate productivity, Evaluate the quality, Evaluate the product.

• Test case effectiveness metricsMeasure the “defect revealing ability” of the test suiteUse the metric to improve the test design process

• Test-effort effectiveness metricsNumber of defects found by the customers that were not found by the test engineers After product Developed.

Test Tools and Automation

strengths Prerequisites List• Increased productivity of the testers.

• Better coverage of regression testing.

• Reduced durations of the testing phases.

• Reduced cost of software maintenance.

•Increased effectiveness of test cases.

• The test cases to be automated are well defined.

• Test tools and an infrastructure are in place.

•The test automation professionals have prior successful experience in automation.

• Adequate budget have been allocation for the procurement of software tools.

Weaknesses• Long learning Curves.• High time and cost for test preparation.• Test tool limitation.

Test Team Organization and Management

• Unit Testing Programmer• System Integration Test engineer• System test group must be separated from development group and they put the plan for the UAT, BAT• Acceptance Client special group (SW quality assurance engineer, business associate, customer support engineer)

•Hiring and retaining test engineers is a challenging task.•Interview is the primary mechanism for evaluating applicants.

Figure 5: Structure of test groups