WHAT PREDICTION MODEL SHOULD BE ？

Institute of Software Chinese Academy of SciencesQing Wang

2

Why we use prediction model ？ Understand the past behavious of

process Control the present behavious of process Predict the future behavious of process

3

How to understand the past behavious ？

Based on the historical data which has been collected from relative processes

Apply appropriate measure and analysis techniques to analyze the data quantitatively.

Try to find some nature characteristics Is stable ？ Nature bounds ？ Does some quantitative relationship among

process attributes ？ Can be stratified /segmented or composed

further ？

4

How to control the present behavious ？

Based on the nature bounds (baselines) to establish the desired objectives of project

Compare the present data to say if it is compliant with the nature bounds.

Take proper action to correct the deviation

Monitor and evaluate the effect of correct action

5

How to predict the future behavious ？

Based on the quantitative relationship which established depend on the understanding of past behavious.

Predict the interim outcome and final outcome of project to know if the project is going well toward the project objectives.

6

What is the essential ？ Differ from estimation model

Deterministic, uncertainty, …… Use past and present data to predict future Some dependent X factors, predict

independent Y outcomes The measurement for Xs and Y make sense Data collection timely and frequently Dynamical Statistical, probability, simulation-based

7

Must be careful when establish prediction model ？

Use real world and relevant data Align the necessary project objectives SMART principle

Specific Measurable Attainable Relevant Time-bound

Cost-benefit balance Feedback and refinement

Practical Case – A CMMI ML4 organization

Issue: the organization found there are more than 90% projects delay is due to test and fix bugs need much more time than estimated

Question: How to predict the effort and schedule of test process?

Method: Identify the factors which impact the effort and

schedule of test process Establish a prediction model use these factors Measure and control these factors Predict the effort and schedule of test based on

the factors when they were got

The process

Each user case has been implemented by requirement analyze, design and coding

reviews for each user case for each development activity were conducted respectively and defect data was recorded

Each iteration has a unique testing and data was recorded The subsystem of each iteration was delivered to customer as a

intermediate result, they must be on schedule and have good quality

System testing was conducted when all iterations were completed (each release)

Process improvement objectives Goal

G1: manage the schedule of each iteration and reduce the defect as soon as possible

G2: manage the system test to detect and fix defects as many as possible

G1: in each iteration

All defect data related requirement analysis, design and coding were collected before testing

Can we find some quantitative relationship between these data and the effort of testing process?

There are two primary activities in testing process

Detect defects Fix defects

Get data from 16 history projects

13

Defect injection analysis - 1

Defect injected in design

Defect injected in requirement analysis

Defect injection analysis - 2

Defect injected in codingThe factor can be control , it can be used to establish prediction model

Y is the effort of fix defects.

If the predicted effort beyond estimate, they should assign more engineers to assure on schedule deliver of each iteration.

G2: for system testing

16

Use controllable factor to predict

Similar prediction model for system testing can be established.

The possible impact factors could be: The defects injected in requirement, design

and coding The defects removed respectively The defects removed in integrated testing

of each iteration ……

17

Summary

All data can be get The factors which used to establish

prediction model are controllable and independent

Statistical techniques are used to do analysis

Prediction object is relevant to the organization process improvement objectives

The prediction model can be refined along with the using and more data was got

18 Thanks ！