ASSIGNMENT ON METRICS AND EVALUATIONSASSIGNMENT ON METRICS AND EVALUATIONS

SUBMITTED BY:SUBMITTED BY:

Pavithra.S.Pavithra.S. Preethi SubramanianPreethi Subramanian Punithavathi.P.Punithavathi.P. VidhyaLakshmi.R.VidhyaLakshmi.R. YaminiPriya.K.YaminiPriya.K.

SOFTWARE TESTING

WHY METRICS AND MEASUREMENT?WHY METRICS AND MEASUREMENT?

Metrics derive Info from raw data with a view to help in Decision makingMetrics derive Info from raw data with a view to help in Decision making

in areas likein areas like

Relationship between data pointsRelationship between data points

Any cause and effect correlation between observed data pointsAny cause and effect correlation between observed data points

Any pointers to how data can be used in FutureAny pointers to how data can be used in Future

Project Performance can be tracked by:Project Performance can be tracked by:

Measuring the Right Parameters Measuring the Right Parameters

Performing right analysis on data measuredPerforming right analysis on data measured

Presenting the results of analysis in an appropriate mannerPresenting the results of analysis in an appropriate manner

STEPS IN METRICSSTEPS IN METRICS

Identify What to measureIdentify What to measure

Transform Measurements to metricsTransform Measurements to metrics

Decide operational requirementsDecide operational requirements

Perform metrics analysisPerform metrics analysis

Take actions and follow upTake actions and follow up

Refine measurements and metricsRefine measurements and metrics

WHY METRICS IN TESTINGWHY METRICS IN TESTING

It is essential to measure the progress of testing and product qualityIt is essential to measure the progress of testing and product quality

Test case execution productivity:Test case execution productivity: Days needed to complete testing=total test cases yet to be executedDays needed to complete testing=total test cases yet to be executed

Estimate of total days:Estimate of total days:

Days needed for release=Max(Days needed for testing &fixing defectsDays needed for release=Max(Days needed for testing &fixing defects

Release date:Release date:

Total days needed for fixing defects= (Outstanding defects yet to be fixedTotal days needed for fixing defects= (Outstanding defects yet to be fixed ++ Defects that can be found in future test)Defects that can be found in future test) // Defect fixing capabilityDefect fixing capability

Metrics test help in identifying Metrics test help in identifying

When making the releaseWhen making the release

What is the release:What is the release:

Based on defect density across modules & analysis scope of Based on defect density across modules & analysis scope of

product can be decided to release the product in timeproduct can be decided to release the product in time

Whether product is being released with known quality:Whether product is being released with known quality:

Deciding whether product is released with known quality orDeciding whether product is released with known quality or

will function on predictable way in field.will function on predictable way in field.

TYPES OF METRICSTYPES OF METRICS

Project metrics:Project metrics:

Indicates how project is planned and executed Indicates how project is planned and executed

Progress metrics:Progress metrics:

Tracks how different activities are progressingTracks how different activities are progressing

Productivity metrics:Productivity metrics:

Takes into account various productivity numbers used for Takes into account various productivity numbers used for

planning and testing activitiesplanning and testing activities

PROJECT METRICSPROJECT METRICS

Projects start with information gatheringProjects start with information gathering

After info collection Planning And tracking Is doneAfter info collection Planning And tracking Is done

In planning scope is estimated that is mapped as size estimate which In planning scope is estimated that is mapped as size estimate which

in turn is mapped as Effort estimatein turn is mapped as Effort estimate

This is This is Baselined effort.Baselined effort.

Two factors to be tracked in each phaseTwo factors to be tracked in each phase

EffortEffort

ScheduleSchedule

If something goes wrong with the scope or project progresses If something goes wrong with the scope or project progresses

the process is re-evaluated. This is called the process is re-evaluated. This is called Revised effortRevised effort

The basic measurements used here are:The basic measurements used here are:

The different activities,the initial baseline and schedule used for The different activities,the initial baseline and schedule used for

each process.each process.

The actual effort and time taken for various activities.The actual effort and time taken for various activities.

The Revused estimate for effort and schedule .The Revused estimate for effort and schedule .

Effort Variance:Effort Variance:

Plotting baselined ,revised and actual efforts ,phase-wisePlotting baselined ,revised and actual efforts ,phase-wise

Substantial difference between Baselined and revise effort point to Substantial difference between Baselined and revise effort point to

incorrect initial estimation.incorrect initial estimation.

Variance%=[(Actual effort – Revised estimate)/Revised estimate]*100Variance%=[(Actual effort – Revised estimate)/Revised estimate]*100

Schedule Variance:Schedule Variance:

Deviation of actual schedule from Estimated scheduleDeviation of actual schedule from Estimated schedule

Calculated at the end of every milestoneCalculated at the end of every milestone

Several phases could be active at the same time based on the Several phases could be active at the same time based on the SDLC usedSDLC used

Effort Distribution across phasesEffort Distribution across phases

Variance calculations help in finding out whether commitments are metVariance calculations help in finding out whether commitments are met

The distribution effort across each phases can be estimated at time of The distribution effort across each phases can be estimated at time of planning planning

PROGRESS METRICSPROGRESS METRICS

Projects need to be tracked from two angles:Projects need to be tracked from two angles:

How well project is doing with respect to effort and scheduleHow well project is doing with respect to effort and schedule

How well product is meeting with the quality requirements for How well product is meeting with the quality requirements for releaserelease

Defects metrics are classified as:Defects metrics are classified as:

Test defect metricsTest defect metrics

Development defect metricsDevelopment defect metrics

Number of defects found in product are main indicators of qualityNumber of defects found in product are main indicators of quality

TEST DEFINE METRICS:TEST DEFINE METRICS:

Defect priority Defect priority Provides management perspective for order of defects Provides management perspective for order of defects

Defect Severity Defect Severity Provides a perspective of impact of the defectProvides a perspective of impact of the defect

Defect find rate:Defect find rate:

Purpose of testing is to find defects early in the cyclePurpose of testing is to find defects early in the cycle

After certain period of testing and fixing plot the results in a Bell Curve.After certain period of testing and fixing plot the results in a Bell Curve.

Defect fix rate:Defect fix rate:

Purpose of development is to fix defects as soon as they arrivePurpose of development is to fix defects as soon as they arrive

Last minute fixing results in slip of deadlines , product quality and also put Last minute fixing results in slip of deadlines , product quality and also put development at pressuredevelopment at pressure

Outstanding defects rate:Outstanding defects rate:

No of outstanding defects =Total defects found – Total defects FixedNo of outstanding defects =Total defects found – Total defects Fixed

Outstanding defects in a well executed project is nearly Zero.Outstanding defects in a well executed project is nearly Zero.

Priority outstanding rate:Priority outstanding rate:

Sometimes defects coming out of testing may be critical and enormous.Sometimes defects coming out of testing may be critical and enormous.

Correspond to extreme and critical classification of defectsCorrespond to extreme and critical classification of defects

Some defects of high priority may require change in design or architecture while Some defects of high priority may require change in design or architecture while some defects may take only little time but enormous effortsome defects may take only little time but enormous effort

Defect trend:Defect trend:

Consolidating fix rate, find rate, priority outstanding and outstanding into one chartConsolidating fix rate, find rate, priority outstanding and outstanding into one chart

Defects classification trends:Defects classification trends:

How many are Extreme defectsHow many are Extreme defects

How many are CriticalHow many are Critical

How many are importantHow many are important

Weighted defects trend:Weighted defects trend:

Helps in quick analysis of defectsHelps in quick analysis of defects

Weighted defect =[Extreme*5+Critical*4+Important*3+Minor*2+Cosmetic]Weighted defect =[Extreme*5+Critical*4+Important*3+Minor*2+Cosmetic]

Defect cause distribution:Defect cause distribution:

Why defects occur and what are root causes of the defectWhy defects occur and what are root causes of the defect

What areas must be focused for getting more defects in testing.What areas must be focused for getting more defects in testing.

Development defects metricsDevelopment defects metrics

Component wise defect distribution:Component wise defect distribution:

Important to map defects into different componentsImportant to map defects into different components

Knowing the component that produce more defects help in fixing them.Knowing the component that produce more defects help in fixing them.

Defect density and defect removal rate:Defect density and defect removal rate:

Defect density correlates source code and defectsDefect density correlates source code and defects

Defects per KLOC=[Total defects in the product]/[total executable line Defects per KLOC=[Total defects in the product]/[total executable line

of codes in KLOC]of codes in KLOC] Defect removal rate= (Defects found by verification activitiesDefect removal rate= (Defects found by verification activities

++

Defects found in unit testing) /(Defects found in test team)Defects found in unit testing) /(Defects found in test team)

* *

100 100

Age analysis of outstanding defects:Age analysis of outstanding defects:

Age is the time defects have been waiting to be fixed.Age is the time defects have been waiting to be fixed.

The time needed to fix a defect is proportional to ageThe time needed to fix a defect is proportional to age

Introduced reopened defects trend:Introduced reopened defects trend:

Fixing a defect sometimes may lead to the malfunctioning of a Fixing a defect sometimes may lead to the malfunctioning of a

currently working proces.This is called Introduced Defect.currently working proces.This is called Introduced Defect.

PRODUCTIVITY METRICSPRODUCTIVITY METRICS

Purpose:Purpose:

Estimating for new releaseEstimating for new release

Finding out how well a team is progressingFinding out how well a team is progressing

Estimating the number of defectsEstimating the number of defects

Estimating release date and qualityEstimating release date and quality

Estimating cost involved in ReleaseEstimating cost involved in Release

Defects per 100 Hours of Testing:Defects per 100 Hours of Testing:

(Total defects found for a period / Total hours spent to get defects)*100(Total defects found for a period / Total hours spent to get defects)*100

Test cases Executed per 100 Hours of testing:Test cases Executed per 100 Hours of testing:

(Total test cases executed for a period / Total Hours spent in last (Total test cases executed for a period / Total Hours spent in last execution)*100execution)*100

Test cases Developed per 100 Hours of testing:Test cases Developed per 100 Hours of testing:

(Total test cases developed for a period / Total hours spent in test case (Total test cases developed for a period / Total hours spent in test case Development)*100Development)*100

Defects per 100 test cases:Defects per 100 test cases:

(Total defects found for a period / Total test cases executed for the (Total defects found for a period / Total test cases executed for the same period)*100same period)*100

Defects per 100 failed test cases:Defects per 100 failed test cases:

(Total defects found for a period / Total test case failed due to those (Total defects found for a period / Total test case failed due to those defects)*100defects)*100

Closed defect distribution:Closed defect distribution:

Making sure the defects are fixed and the customer gets quality Making sure the defects are fixed and the customer gets quality

assuranceassurance

RELEASE METRICSRELEASE METRICS

Deciding the release of productsDeciding the release of products

Varies from Product to productVaries from Product to product

Provides guidelines in making the DecisionProvides guidelines in making the Decision

CourtesyCourtesy

““Software testing “ by William StallingsSoftware testing “ by William Stallings

22ndnd Edition Edition