Important Points to Note -...

Important Points to Note • All Participating colleges are requested to mute your telephone lines during the webinar session.

• Participants are requested to make note of questions if any and post it to the instructor using the

chat option during the webinar session .

• Question & Answer(Q&A) is scheduled at the end of the session and each college is given a

chance to ask 2-3 questions. Answers to the questions posted by colleges through chat option

will also be answered during the Q&A.

• Instructor may initiate the relevant Polls at regular intervals during the session. Colleges are

requested to gain inputs from all the participants and provide your responses to these polls

initiated by instructor.

• Session Feedback will be initiated at the end of the webinar. Colleges are requested to get the

collective opinion from the participants and provide your responses for the feedback

questionnaire.

Software Testing

Session Rationale and ObjectiveSession Name: Software Testing

Session Rationale: The purpose of learning this Session is to make the learner understand that various types of

testing which is an important phase in software development process. Also, testing will help the team to produce

defect free and quality code.

Session Objective:

At the end of this Session, you will be able to:At the end of this Session, you will be able to:

• Develop test plans to carry out Unit testing

• Perform unit and integration testing

• Understand the code review guidelines and checklist and apply them to produce

quality code

Session PlanSession Name: Testing

ContentLearning Aid /

Methodology

Trainer

Approach

Learner

Activity

Learning Outcome

(Skill, Competency)

Objective of Testing

and Test plans

generation

Presentation, Scenario

Explains,

Discusses,

facilitates

Participates,

Absorbs, RespondsAbility to generate Test plans

5

generation facilitates

Recording / Logging

a DefectPresentation, Hands-

on

Explains,

Discusses,

facilitates

Participates,

Absorbs, RespondsAbility to log defects during Unit testing

Different types of

Testing Techniques Presentation

Explains,

Discusses

Participates,

Absorbs, RespondsAbility to understand various testing types

Testing – What Is it?

• A process of demonstrating that errors are not present?

OR

• A way of establishing confidence that a program • A way of establishing confidence that a program does what it is supposed to do?

OR

• A means of achieving an error-free program by finding all errors?

Testing is…

• A process of executing a program with the

intent of finding errors

• A “DESTRUCTIVE”, yet creative process

Why Testing?

• Verifies that all requirements are implemented correctly (both for positive and negative conditions)

• Identifies defects before software deployment

• Helps improve quality and reliability• Helps improve quality and reliability

• Makes software predictable in behavior

• Reduces incompatibility and interoperability issues

• Helps marketability and retention of customers

Cost of Ineffective Testing

• Time– Late Releases– Projects need to be reworked or abandoned

• Money– Budget over-runs– Budget over-runs– Defects are 100 to 1000 times more costly to find and

repair after deployment

• Quality– Developers unsure of product quality– Products released with undiscovered or unresolved defects

Testing Lifecycle

Iteration nDevelopment Lifecycle

Project Planning

Requirements Capture

Implementation

Build 0 Build 1 Build 2

Analysis & Design

Test Lifecycle

Plan Test

Design Test

Setup Test Bed

Execute TestEvaluate Test

Defect Tracking/Enhancement Request

Testing Lifecycle

Requirements Capture

Analysis

Test planning & Scenario Design

Test Case

The Scenarios Design and the Test Case Development could normally start in parallel with the Development Cycle.Test Execution Synchs up with the Development Cycle during the functional Testing Phases

Test Case Development

Test Execution

Test Result Analysis

Test Cycle Closure

Defect Fixing Cycle

Defects

New Version

Testing Lifecycle

• Test requirements identification– Validate for testability

• Test Planning and Scenario design– Develop Test Objectives– Develop Test Objectives

– Identify Test Items

– Resources and Schedules

• Test Case Development– Test Case Specification

– Pre-requisites, post-requisites and Acceptance criteria

Testing Lifecycle (contd.)

• Prepare Test Bed– Test Data Preparation– Test Environment Setup

• Test Execution– Run Tests and Validate Results– Run Tests and Validate Results– Bug Reporting– Bug fixes and retesting

• Test Result Analysis– Defect Analysis– Determining Test Set Coverage and Effectiveness

Start Testing Early

• There are two basic forms of testing

– Verification Testing

– Validation Testing

• Verification: “Are we building the right thing?”

• Validation: “Are we building it right?”

Testing Stages – the V-Model

Requirement Analysis

Functional Specifications

User Acceptance Testing

System Testing

High-Level Design

Detailed Design

Code

Integration Testing

Unit Testing

Verification ValidationReviews

Testing Techniques

• Static Testing - Verification

• Dynamic Testing - Validation

– White Box Testing– White Box Testing

– Black Box Testing

Testing Techniques (contd.)

• Static Testing/Reviews– Scope

• Requirements Feasibility

• Design and Technical Architecture

• Program specifications and Code• Program specifications and Code

• Test Plans, Test Cases and Data

• User Documents

– Modes• Reviews

• Inspections

• Code Walkthroughs


• Static Testing– Advantages

• Capture defects early, so saves rework cost

• Checklist-based approach

• Focuses on coverage• Focuses on coverage

• Group perspective

• High probability of finding defects

– Disadvantages• Cannot test data dependencies

• High skill levels required


• Dynamic Testing

– White-box/Glass-box Testing

• Unit Testing• Unit Testing

• Integration/Interface Testing

– Black-box Testing

• System Testing

• Acceptance testing

White-Box/Glass-box Testing

• Structural Testing process

• Program Logic-driven Testing

• Design-based Testing• Design-based Testing

• Examines the internal structure of program

White-Box Testing – Advantages

• High Code coverage – Exhaustive (thorough)

path testing

• Program logic is tested• Program logic is tested

• Internal boundaries are tested

• Testing is algorithm specific

Black-box Testing – Advantages

• User’s perspective

• Focus on features, not implementation

• Big-picture approach• Big-picture approach

Levels of Testing

• Unit Testing

• Integration Testing

• Function Testing• Function Testing

• Performance Testing /System Testing

• Acceptance Testing

• Installation Testing

Unit Testing

Testing Stages – the V-model




System Testing

High-Level Design

Detailed Design

Code

Integration Testing

Unit Testing


Unit Testing (1 of 2)• Each individual unit of code is tested to ensure that it performs its intended

functionality

• Unit tests are done on their respective modules by Software Engineer who has written code

• Unit tests are created using some techniques which ensure that all logical paths of the code unit are tested and maximum number of errors can be detected by executing these tests

• The Unit Test Plan (UTP) prepared towards the end of the design phase is used for Unit Testing the software or code

Unit Testing (2 of 2)

• Any defects found during unit testing are logged in the Defect Tracking System (DTS) and they are tracked till the defects are removed from the code

• Test Case: A set of inputs, execution preconditions, and expected outcomes developed for a particular objective, such as to exercise a outcomes developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement

• Precondition for a test case: A precondition for a test case is any activity that is required to be completed in order for the stage to be set for the test case execution. A precondition is one without which the test procedure cannot be executed

Documenting Test Cases (1 of 2)

• Very often test plans contain hundreds of test cases and so it is essential to keep documentation of test cases clear and in simplest terms as possible

Typical Test Plan Template

SlNo

Test case name

Test Procedure Pre-condition (Input

Condition)

Expected Result

Reference to Detailed Design / Spec Document

• A test case name should be of the following format.

<Module Name>_<Function Name>_<Test Procedure>,

where

– Module Name is the name of the module the test case tests

– Function Name is the name of the function or functionality the test case tests

– Test Procedure is a term or word which briefly represents what the test case is trying to do

Documenting Test Cases (2 of 2)

• Test Procedure (Condition to be tested): Explains briefly but clearly, what the test case is doing.

• Input Condition: Input Condition contains details of any specific preconditions

• Expected Result: sometimes known as Pass/Fail Criteria

Identifying Test Cases

• Boundary Value Analysis

• Equivalence Partitioning

• Logic Coverage

• Random Generation

M1 may check the footer notes, does not seem very relevantMeenakshi_S04, 3/29/2009

Boundary Value Analysis (1 of 5)

• A boundary value is one which indicates the border (or the limit) of a value

• Test cases that explore boundary values have the highest payoff in terms of detecting bugs, as the most common errors occur at the boundary levelthe boundary level

• Boundary value test cases are those which test the function or field using values that – Lie on the boundary

– Just above the boundary

– Just Below the boundary

• For example if an input condition specifies that

the range of values of the input variable items

must be from 0 to 100, the boundary values

would be -1, 0,1,99,100 and 101


would be -1, 0,1,99,100 and 101

/************************************************** **********/* Function: fnFindGrade * Description: Given the percentage score of studen t, * assigns the grade of the student.* Criteria for Grades:* A - 80 to 100* B+ - 73 to 79* B – 65 to 72* C – 55 to 64


* B – 65 to 72* C – 55 to 64* D – 0 to 54* Z – for invalid grades (Score <0 or score >1 00)* Input Parameters: * int iPercentScore - Percentage scored by the stude nt* Char acGrade[] – Array containing the grade assign ed* Returns:None*************************************************** **********/

void fnFindGrade (int iPercentScore, char acGrade[]) {if (iPercentScore >=80 && iPercentScore <=100) {

strcpy(acGrade,”A”);}else if (iPercentScore >=73 && iPercentScore <=79) {

strcpy(acGrade,”B+”);}else if (iPercentScore >=65 && iPercentScore <=72) {

strcpy ( acGrade ,”B”);


strcpy ( acGrade ,”B”);}else if (iPercentScore >=55 && iPercentScore <=64) {

strcpy(acGrade,”C”);}else if (iPercentScore >=0 && iPercentScore <=54) {

strcpy(acGrade,”D”);}else {

strcpy(acGrade,”Z”);}

}

Boundary Value Analysis (5 of 5)Sl No Test case name Test Procedure Pre-condition Expected Result


1fnFindGrade_MinusOne

Call fnFindGrade with iPercentScore = -1

None “Z” should be assigned to grade (Negative Test case)

fnFindGrade

2 fnFindGrade_0 Call fnFindGrade with iPercentScore = 0

None Grade “D” should be assigned

fnFindGrade



fnFindGrade


None Grade “A” should be assigned

fnFindGrade


None Grade “A” should be assigned

fnFindGrade


None “Z” should be assigned to grade (Negative test case)

fnFindGrade

Equivalence Partitioning (1 of 3)

• This consists of dividing all possible inputs into a set of classes, where either all inputs that fall into a given class are valid or all are invalid. Then selecting a few test cases from each class is sufficient

• For example if an input condition specifies that the range of values of the • For example if an input condition specifies that the range of values of the input variable items must be from 1 to 999, one valid equivalence class would be 1<=items<=999 and 2 invalid equivalence classes would be items<1 and items>999

• The values chosen for Equivalence partition should not be a boundary value

Equivalence Partitioning(2 of 3)Sl No Test case name Test Procedure Pre-

conditionExpected Result Reference to Detailed Design /

Spec Document

1 fnFindGrade_D20 Call fnFindGrade with iPercentScore = 20


fnFindGrade

2 fnFindGrade_D48 Call fnFindGrade with iPercentScore = 48


fnFindGrade iPercentScore = 48 assigned

3 fnFindGrade_C59 Call fnFindGrade with iPercentScore = 59

None Grade “C” should be assigned

fnFindGrade

4 fnFindGrade_B71 Call fnFindGrade with iPercentScore = 71

None Grade “B” should be assigned

fnFindGrade

Equivalence Partitioning(3 of 3)Sl No Test case name Test Procedure Pre-

conditionExpected Result Reference to Detailed Design /

Spec Document

5 fnFindGrade_A90

Call fnFindGradewith iPercentScore = 90

None Grade “A” should be assigned fnFindGrade

6 fnFindGrade_InvalidMinus

Call fnFindGradewith iPercentScore =

None “Z” should be assigned to grade fnFindGrade_InvalidMinus

30 -30 (Negative test case) fnFindGrade

7 fnFindGrade_Invalid300

Call fnFindGradewith iPercentScore=300

None “Z” should be assigned to grade (Negative test case) fnFindGrade

Logic Coverage (1 of 3)• This technique aims to generate enough test cases so that an appropriately

defined coverage criterion is met

• Criterion: Every statement in the program must be executed at least once, every branch in the program must be executed at least once, or every path in the program must be executed at least once

• Example:• Example:– The User Interface for searching the address book should be very friendly and should

use the following search criteria

– Can search the address book based on one or more of the following fields:• E-mail Id of the person

• Name of the person

• Employee Number of the person

– Can accept one or more fields for search criteria

– Accepts partial values for E-mail and Name

Logic Coverage (2 of 3)

Logic Coverage (3 of 3)Sl No Test case name Test Procedure Pre-condition Expected Result


1 AddrBook_All_BlankAll the fields are kept blank and click on 'search'

NoneAddress Book must display an error message and prompt the user to enter atleast one field (Negative test case)

Address Book Module

2 AddrBook_EmpNo_OkType in an Employee Number and then click on 'search'

NoneAddress Book must fetch only one entry of the person with that employee number

Address Book Module

Type in an invalid Employee Number Address Book must fetch zero records and

3 AddrBook_EmpNo_FailType in an invalid Employee Number and then click on 'search'

NoneAddress Book must fetch zero records and display that the record is not found (Negative test case)

Address Book Module

4 AddrBook_Email_FullType in a full email-id (aniket_sharma) and then click on 'search'

NoneAddress Book must fetch only one entry of the person with the corresponding email-id

Address Book Module

5 AddrBook_Email_PartialType in a partial but valid email-id (aniket) and then click on 'search'

NoneAddress Book must fetch one or more records where email-id begins with same letters

Address Book Module

6 AddrBook_Email_FailType in an invalid email-id (kkgghh) and then click on 'search'

NoneAddress Book must fetch zero records and display that the record is not found (Negative test case)

Address Book Module

Random Generation

• Data is generated randomly either using a tool or

manually. This is the simplest method but not

the most efficient

Scenario

Problem Description: Online Balance Enquiry

The User interface design for balance enquiry is given. PIN number can take at least 4 digits

starting from 1000; Balance amount is displayed if valid PIN number is entered by the user.

Otherwise, it displays appropriate error message.

Identify and document test cases for a given user interface design using the techniques

learnt.

Otherwise, it displays appropriate error message.

PIN Number:

Balance Amount: ………………………………………………………

Submit

Recording / Logging a Defect

• Any defect found in code or document must be recorded

• Recording of defects will ensure that the defects detected

are not lost and also ensures that they are tracked till the are not lost and also ensures that they are tracked till the

defects are removed completely

• Defect data can also be used to do an analysis, which will

help improve the process of writing code and testing

Defect Tracking System• Most software companies have a dedicated system, for logging and

tracking defects

• Most defect tracking systems can also do detailed analysis of the defects to help a project take corrective action in due course of time.

• In the absence of a Defect Tracking System, an MS Excel sheet (spreadsheet) is maintained to log and track defects

• When testing or code review is done and defects are found, defects are logged

A sample defect tracking Excel sheet (1 of 2)

• A defect log captures some of the following information:

– Defect Number or Id: The number or Id which identifies the defect

– Submitted by: Person who found the defect

– Description: A detailed description of the defect

– Detected Stage: The stage at which defect was detected

• Example: Unit Testing, Code Review etc.

A sample defect tracking Excel sheet (2 of 2)

• Example: Unit Testing, Code Review etc.

– Assigned to: The developer who has to remove this defect

– Type of defect: Type of defect tells about the nature of the defect

• Example: Coding Standards related, Logical Error

– Injected Stage: The stage of software life cycle where this defect might have been introduced

• High Level Design, Detailed Design, Coding etc

– Action Taken: To be fixed or fixed

– Notes: Additional notes such as a guideline for fixing the bug could be included

CrosswordSolve the puzzle from SDLC and Testing sessions:

Integration Testing





System Testing

High-Level Design

Detailed Design

Code

Unit Testing

Integration Testing


Integration Testing

• Combining and testing multiple components

together

• Integration of modules, programs and • Integration of modules, programs and

functions

• Tests Internal Program interfaces

• Tests External interfaces for modules

Integration TestingBanking System

CRM HR Financial External interface

External System

Java Based Based on C, C++ VB based

Online banking Loans Account OpeningInter module interface

Intra module interface Unit Module

Approach for testing interfaces

Intra-Module Inter-Module External

Test Techniques White Box Black Box Black Box

Test Environment Setup Development environmentIsolated development environment

Simulate live environment

Test ApproachCheck for parameter / procedure called.Database updation

Identify pre-process and approach to Interface.Identify the post-process to validate the interface

Identify external / internal interface and validate against external system

Techniques based on order of Integration

• Big-bang Integration (non-incremental)

• Random Incremental Integration

• Top-Down Integration• Top-Down Integration

• Bottom-up Integration

Top-down vs. Bottom-up

• Top-down Integration– Advantage

• Key Interface defects trapped earlier

– Disadvantage• Core functionality tested late in the cycle• Core functionality tested late in the cycle

• Bottom-up Integration– Advantage

• Core functionality tested early

– Disadvantage• Key Interface defects trapped late in the cycle

System Testing





System Testing

High-Level Design

Detailed Design

Code

Integration Testing

Unit Testing


System Testing – What is it?

• A Black-box Testing technique

• Specifications-based Testing

• Typically independent team testing

• Simulated environment testing

• Live/Simulated user data• Live/Simulated user data

• Tests the whole system

• Functional and non-functional requirements tested

• Business transaction-driven testing

• Compatibility errors uncovered

• Performance limitations uncovered

System Testing – Different Types

• Functional testing (Sanity / Regression )

• Performance and Scalability testing

• Load/Stress testing

• Volume Testing

• Usability testing• Usability testing

• Configuration testing

• Installability testing

• Disaster and Recovery testing

• Reliability / Availability testing

• Statistical Testing based on Usage Models

Functional Testing – Sanity Test

• Very basic minimal number of tests to verify the product for the feature / protocol compliance

• Could be a subset of regression test used to validate only high level functionality

• Typically an initial testing effort to determine if a new • Typically an initial testing effort to determine if a new software version is performing well enough to accept it for a major testing effort

• Sanity testing will be performed in scenarios when cursory testing is sufficient to prove that the application is functioning properly

Why Sanity Testing?

• Regression testing is more time taking and more effort is required whereas Sanity testing is quicker focusing on important functionality of software

• Saves time as only selected high level functionality areas are only tested. Bugs found this way would take

• Saves time as only selected high level functionality areas are only tested. Bugs found this way would take almost triple the time when found through regression testing

• Efficient in scenarios wherein releases comes in pretty often with very few changes

Functional Testing – Regression

• Re-execution of one or more tests in subsequent build of the application/product to ensure– Revisiting and testing all prior bug-fixes in response to a

new fix/enhancement

Re-testing all programs that might be affected by the – Re-testing all programs that might be affected by the fix/enhancement

– Hidden Bugs are uncovered

• The baseline for Regression Testing grows with every build

Regression Testing

• Should Cover– Business Process depending on criticality

– User friendliness

– Cross functional dependencies– Cross functional dependencies

• Activities– System appreciation

– Preparation of Test case repository

– Automate test cases

– Execute Regression test suite

Without Regression Testing – Hidden Bugs

PASS PASS? PASS?Test Requirement 1

UnknownUnknown

CBA

Software Builds

FixFAIL FAIL PASS

PASS BUG? BUG?

Requirement 1

Test Requirement 2

Test Requirement 3

NewProblem

HiddenBug

Fix

With Regression Testing – No Hidden Bugs

PASS PASS PASSTest Requirement 1

CBA

Software Builds

FixFAIL FAIL PASS

PASS BUG PASS

Test Requirement 2

Test Requirement 3

NewProblem

Fix

Fix

Acceptance Testing

• Demonstrates that the system meets mutually agreed Acceptance criteria

– Critical Requirements

– Minimum Performance level– Minimum Performance level

– Maximum Defect Detection Rate

• Typically, a sub-set of System testing

Alpha / Beta Testing

• Forms of Acceptance testing

• Testing in the production environment

• Alpha testing is performed by end users within • Alpha testing is performed by end users within

a company but outside development group

• Beta testing is performed by a sub-set of

actual customers outside the company

Performance Testing

• Number of concurrent users accessing at any point in given time

• System’s performance under high volume of data

• Stress testing for systems, which are being scaled up to • Stress testing for systems, which are being scaled up to larger environments or implemented for the first time

• Operational intensive transactions. (Most frequently used transactions)

• Volume intensive transactions (for both volume and stress testing)

Installation Testing

• Basic installation

• Installation of various configurations

• Installation on various platforms• Installation on various platforms

• Regression testing of basic functionality

When do you stop testing?

• When time runs out

• Specific test case design methods

• Target test coverage attained

• Certain number of errors found*

• Error detection rate drops below a threshold*• Error detection rate drops below a threshold*

• Predicted density of errors left drops below a threshold

Sequence test cases in order of priority

* Infosys Process Capability Baseline

Test Artifacts

• Test Plan

• Test Environment/Bed

• Test Case

• Test Data• Test Data

• Test Tools

• Test Scripts

• Test Log

• Bug Reports

Traits of a good Tester

• Destructive creativity

• Detective skills

• Appreciating the users’ perspective • Appreciating the users’ perspective

• Adapt to and Understand Requirements change

• A skeptical, but not hostile attitude

• An eagerness to embrace new technologies

Test Engineers

• Role - Execute tests, record test scripts, maintain statistics and metrics, check test data setup, test environment setup, execute re-tests

• Knowledge - Understanding of the importance of testing, awareness of tools, how to progress against a plan

• Knowledge - Understanding of the importance of testing, awareness of tools, how to progress against a plan

• Skills - Observation, accuracy, methodical, co-ordination, problem solver

• Experience - Following instructions, problem reporting and solving, and relevant testing tools

Test Automation

• Good candidates:– Complex and time-consuming tests

– Tests requiring a great deal of precision

– Tests involving many simple, repetitive tests– Tests involving many simple, repetitive tests

– Tests involving many data combinations

• Lower priority candidates:– One-time only tests

– Testing peripheral devices

– Subjective assessment tests (look and feel based)

Testing Tools

Types of Testing Tools

• The software testing tools broadly of the following type:

– Functional / regression–– Load /Stress: Performance testing under high load

/ stress

– Web Testing: testing the Web Sites

– Data Generation: Generation of test data

Benefits

• Reduces manpower and time

• More coverage within the same testing time.

• Testing products which are very difficult to • Testing products which are very difficult to

test manually. Example, Load Tests

• Can look inside the software for memory

leaks, redundant code etc.

Test Tools (Client Server)Category Tool Vendor

Source Code Testing

BoundsChecker Numega

Pure Coverage Rational

Purify Rational

Jprobe Sitraka Software

ATTOLCoverage ATTOL Software

QA Run Compuware

Functional Testing

QA Run Compuware

Win Runner Mercury

Team Test Rational

QA Center Compuware

Performance Testing

OptimizeIT VM Gear

Load Runner Mercury

QA Load Compuware

Test Tools (Web Applications)

Category Tool Vendor

Functional Testing Rational Robot Rational

Silk Test Segue Software

Performance Testing Silk Performer Segue Software

Web Load Radview

Link & HTML Testing NetMechanic Monte Sano

Linkbot Tetranet

Site Inspector

Doctor HTML Imageware

Hands-on

Problem Description: Online Balance Enquiry

Implement the Balance Enquiry module in C and Perform Unit Testing and Log Defects.

Implement the Balance Enquiry module by writing a function. The data structure can

be 2-D array for storing set of pin numbers and the corresponding balance amount. be 2-D array for storing set of pin numbers and the corresponding balance amount.

Accept a pin number from the user, if it is found in the array, then display the

corresponding balance amount. If not, display suitable error message. After writing

the source code, test it and log the defects in defect tracking system.

Thank You

Important Points to Note -...

Documents

Transcript of Important Points to Note -...