Measuring ChangeEvaluation Phase

EME 6636 Change ManagementInstructor: Dr. DarabiYang Xu

Fig.1: Integrated SAAPS with the HPT model

Align Structure Staffing, development,

measurement, rewards, communication, organization structure

Optimize results: Change integration Workforce measure Workforce measures

questionnaire Lessons learned

Fig.2: Kate Nelson & Stacy Aaron, Change Guide LLC

Keep track the process of change,

and identify the effectiveness of

process

Identify the validity

(outcome) of change

Identify the long-term impact of change, and

gain supports for future

The purpose of measuring change

Kirkpatrick’s 4-level evaluation

Level Measures

1. Reaction and

Planned Action

Participant’s reaction to the program and

outlines specific plans for implementation

2. Learning Skills, knowledge, or attitude changes

3. Application Changes in behavior on the job and specific

application of the training material

4. Business Impact Business impact of the program

5-level framework for evaluation

Level Measures

1. Reaction and

Planned Action

Participant’s reaction to the program and

outlines specific plans for implementation

2. Learning Skills, knowledge, or attitude changes

3. Application Changes in behavior on the job and specific

application of the training material

4. Business Impact Business impact of the program

5. ROI Monetary value of the results and costs

for the program, usually expressed as a

percentage

The

2003 2004 2005 2006

30

50

70

120

The comparision between cost and benefits

• Control groups• Trend line analysis• Forecasting• Customer input• Participants’ and supervisors’

estimates of intervention’s effects• Expert estimation

• Evaluation purpose• Evaluation instruments• Evaluation timing• Evaluation levels

10 Steps to establish a results-based monitoring and evaluation system

Jody Zall Kusek, Ray C. Rist, The world bank

Plan Implement Maintain

• Objectives

• Links activities and their

resources to objectives

• Performance indicagtors

and targets

• Collect data

• Reports to managers and

alert them problems

• Analyze results

• Assess specific causal

contributions

• Examine implementing

process

• Explores unintended

results

• Recommendation

Monitoring Evaluation

Readiness Assessment

• Need for assessment

• Capability for assessment

Agree on outcomes to monitor and evaluate

1. Identify the key stakeholders

2. Identify the stakeholders’

concerns

3. Transfer problems into

measurable expected

outcomes

Selecting key indicators to monitor outcomes

Clear

Monitorable

Adequate

Economic

Relevant

Setting baselines and gathering data in indicators

Planning for improvement—selecting results targets

By Jan. By Feb.

Results targets

The salary level will increase 5%.

The salary level will increase 15%.

The systematic model

Plan

Monitoring for results

1. Data sources2. Data collection

methods3. Data collection

frequency4. Personnel’s

Responsibility

Key criteria for collecting date

Reliability

Validity Timeliness

Reporting findings

Title in hereTo whom

Title in here

Which format

Title in hereFor what 1.Demonstrate accountability

2.Convince and gain support

3.Document

4. Promote understanding

Purpose of reporting

The systematic model

Plan Implement Maintain

Barriers in measuring change

1. Time consuming

2. Expensive

3. Complicated process

4. Negative evaluation results