Using Measurement to Drive the Impact and Effectiveness of Your Leadership Development Programs...

© 2013 The Forum Corporation.

Using Measurement to

Increase the Impact &

Effectiveness of Your Leadership

Development Programs

© 2012 The Forum Corporation.

2 © 2013 Forum Corporation

Your Presenters

Tom Atkinson, Ph.D.

Learning & Development Consultant

The Forum Corporation

Jeffrey Berk

Chief Operating Officer

KnowledgeAdvisors


Why Measure Leadership Initiatives?

Improve

Prove

Reinforce


You’ll walk away with:

1. Insights on the challenges and benefits of measuring

leadership development

2. The four big differences between the best

and the rest when it comes to measuring leadership development

3. Frameworks for developing a measurement plan

4. Case studies of what high-performing organizations are

doing to measure their leadership initiatives

Four Takeaways for Today


Quiz

How effective is your organization in using measurement to increase

the impact of your leadership development programs?

– We collect and use data to engage all stakeholders

– We collect and use data within the L&D function

– We collect data efficiently but don’t do much with it

– We collect data, but it’s a cumbersome process

– We have no formal measurement process


Measuring and Demonstrating Impact:

What Top Performing Organizations Do

They treat measurement as part of the solution —

just as a dashboard is part of a car and a lab test is part of a

medical exam.

They involve all key stakeholders — especially business

sponsors — in devising the measurement plan.

They scale the measurement process to the size of the

initiative and the need for precision (and err on the side of simpler

and less costly).

They follow a disciplined process for collecting and

reviewing data to drive action — not to “see if anything

happened” but to “make it happen.”


Gaining buy-in to

the initiative’s

goals, metrics, and

implementation

plan

Laying the

foundation for

behavior change

by teaching new

concepts and

skills

Supporting

leaders in

applying and

embedding new

attitudes and

behaviors on the

job

ALIGN EQUIP SUSTAIN

Measurement Helps Accelerate

Leadership Development

Executing, measuring and evaluating with discipline


Actions:

Conduct a planning

meeting with

stakeholders

ALIGN EQUIP SUSTAIN

Elements of a Measurement Plan

Questions:

What are expected

outcomes?

Who is accountable for

what?

Actions:

Survey participants

and managers during

training

Questions:

Is the learning process

effective & efficient?

Are leaders prepared

to apply learning?

Actions:

Survey participants

and managers after

training; organizational

follow-up

Questions:

What is the evidence of

effectiveness?

What tools, training or

support are needed?


Creating Alignment

What You

Measure

Examples of Specific

Metrics

L&D Accountability Line Accountability

1. Reaction

Increased awareness,

positive attitude toward

topics, motivation to

change

■ Course evaluation ratings

of learning satisfaction

■ Design quality

■ Develop materials

■ Select/ train instructor

■ Project management

■ Clarity of objectives

■ Subject matter experts and data

access

■ Visible support for training

2. Learning

Acquisition of concepts

and skills

■ Pre- and post-course

understanding of material

■ Measurement tools

■ Debrief learning with managers

■ Learning plans and meetings

3. Job Impact

Application of concepts

and skills on the job—

Behaviour change

■ 360-feedback scores

■ Employee satisfaction with

leadership behaviour

■ Observed employee

competencies

■ On-the-job application

activities

■ Application activity

debriefs

■ Model desired behavior

■ Post-training reinforcement

meetings

■ Observe/coach on specific

behaviours

■ Align learning priorities for

leaders with performance

management goals

4. Business Results

Achievement of

positive outcomes for

the company and/or

individual

■ Profitability and growth

■ Employee satisfaction

■ Turnover

■ Costs of recruiting

■ Design process to create

a management-led

measurement plan

■ Design & implement a regular

measurement plan

■ Manage processes, provide

technology, recognise & reward

behaviour change


Creating Alignment

What You

Measure


Metrics


1. Reaction




change



■ Design quality






access


2. Learning


and skills






3. Job Impact



Behaviour change



leadership behaviour


competencies


activities


debriefs

■ Model desired behaviour


meetings


behaviours



management goals

4. Business Results

Achievement of


the company and/or

individual



■ Turnover



a management-led

measurement plan


measurement plan



behaviour change


Creating Alignment

What You

Measure


Metrics


1. Reaction




change



■ Design quality






access


2. Learning


and skills






3. Job Impact



Behavior change



leadership behavior


competencies


activities


debriefs



meetings


behaviors



management goals

4. Business Results

Achievement of


the company and/or

individual



■ Turnover



a management-led

measurement plan


measurement plan



behaviour change


Creating Alignment

What You

Measure


Metrics


1. Reaction




change



■ Design quality






access


2. Learning


and skills






3. Job Impact



Behavior change



leadership behavior


competencies


activities


debriefs



meetings


behaviors



management goals

4. Business Results

Achievement of


the company and/or

individual



■ Turnover



a management-led

measurement plan


measurement plan


technology, recognize & reward

behavior change


KnowledgeAdvisors

Jeffrey Berk

Chief Operating Officer

KnowledgeAdvisors


Leadership program Key Tools

June 26, 2013 14

1. Identify Leadership KPIs

2. Create Leadership ‘Smart Sheet’

3. Design the Leadership Dashboard

4. Provide a Leadership Program Executive Report

5. Deploy On-the-Job Reinforcement

6. Conduct Business Impact Exercise


Identify Leadership KPIs

15

Outcome Metrics - Business Outcomes

Leader Retention

Employee Engagement

Employee Retention

Leader Performance Ratings

Leader Financial Management

Effectiveness Metrics – Program

Performance

Delivery Quality

Knowledge Gain

Application to Job

Business Alignment

Value

Efficiency Metrics – Cost and Activity

# of Participants

% of Employees Trained

Total L&D Investment

L&D Investment per Participant

Courses Utilized

Classes Conducted

Instructors Utilized

Locations Used


Create Leadership ‘Smart Sheet’

16

Not only

evaluate

The training

but also the

process

such as on-

boarding or

leadership

development

or sales

effectiveness

.


Smart Sheet Forecasting

Connecting training to specific outcomes


Design Leadership Dashboard

18

A visual, concise data

representation

Shows the efficiency,

effectiveness and outcome

metrics

Data from evaluations, tests,

assessments imports, LMS /

HRIS feeds, uploads, etc.


Leadership Executive Reporting

19

Reporting for a non L&D

audience that emphasize

efficiency, effectiveness

and outcomes.

Reporting for L&D

executives that focus on

satisfaction to impact and

value

Hold quarterly reviews

with L&D execs and

sponsors


On-the-Job Reinforcement

20

Use goal setting tools to establish and track changes in business results or

behaviors when participants are on-the-job. It also can identify scrap learning.


Measuring Leadership

Development Programs:

Case Examples


Manager Engagement Case Example

Consumer goods strategic program

Two teams with 2 different managers trained

One manager used reinforcement post training

One manager didn’t use reinforcement

Manager w/ reinforcement had 67% change

Manager w/out reinforcement had 33% change

22 June 26, 2013 © 2012,

KnowledgeAdvisors


Leadership Program Impact Example

23

Leadership program with optional ‘project’

Determine if optional project influenced employee retention in

leadership program

Employees with no learning projects averaged a turnover rate of

21%

Employees with 2 or more learning projects averaged a turnover

rate of 2%

Clearly the control group vs. learning group shows a difference

Source: Dr. John Sullivan, Presentation at Analytics Symposium


Business context

– Global energy company

– Large-scale merger

– Need to drive operational efficiencies and results

– Shift to leaner, team-oriented environment

Implications for front-line supervisors

– Driving functional results and meeting organizational objectives

– Developing employees to be productive and effective

– Achieving results through others

– Developing future leaders

Front-line Leadership Example


Front-line Leadership

Core Program Elements

Measurement and Documentation:

MTM “Smart Sheet” Surveys for Level 1 (Reaction), Level 2 (Learning), and

Level 3 and 4 (Subjective Evaluation of Impact and Results)

MTM Follow Up Surveys with Participants and Managers for Level 3 and 4

(Impact and Results)

Intermodule

Launch

Webinar:

Setting You Up

for Success as

a Supervisor

4-day Classroom Program:

Specific to Front-line manager

competencies such as coaching,

getting results through others, etc.

Reconnect

Webinar:

Share and

Evaluate

Results,

Recommit

Intermodule


Front-line Leadership Program

Measurement Reporting

• Utilization Reporting

• Cost Reporting

Efficiency Measures: Measures the extent to which

time or effort is well used for the intended task or purpose

• Metrics the Matter Program Reporting

• Post Report

• Follow-up Report

Effectiveness Measures: Measure the quality of the

program

• Selected Business Metrics Outcome Measures:

Measures the relationship and/or value of the program to

broader business goals



Measurement Diagnostic Think of an upcoming Leadership Development Initiative (LDI) you are leading (e.g. high potential leadership

development system, onboarding for supervisors, management essentials, leading change program). Answer

the following questions with respect to that LDI, indicating low, medium or high.

To What Extent… Low Med. High

1. Are executives involved in shaping/defining this specific LDI?

2. Does this LDI link to key business metrics?

3. Will leadership skills/tools introduced in this LDI be immediately applicable on the job for

participants?

4. Will executives of the company be directly involved in implementing this LDI?

5. Is there a measurement discipline and cadence for reporting on talent management?

6. Do managers of participants in this LDI hold their direct reports accountable for critical

leadership behaviors?

7. Is there a measurement discipline and cadence for reporting on leadership development?

8. Is there a documented and communicated leadership development strategy aligned to the

broader business strategy?

9. Is real work application (e.g. individual or group action learning) a part of this LDI?

10.Do managers of participants in this LDI provide ongoing development opportunities, support and

coaching to their direct reports?

TOTAL

(Add up your checks in each column for use on the next page)


Measurement Diagnostic Results

Leadership Development Strategy

Total the number of your “low,”

“medium” and “high” ticks from the


Measurement Diagnostic.

Mostly “

Low

”

Mix

of

“Mediu

m”

and

“Low

”

Mix

of”

Mediu

m”

and

“Hig

h”

Mostly “

Hig

h”

Efficiency measures: Measures the extent to which time or effort is

well used for the intended task or purpose

Effectiveness measures: Measure the quality of the program from

the participant perspective immediately following the program

Effectiveness measures: Measure the quality of the program from

the participant/participant manager perspective with follow up

reports (60 days after the program)

Outcome measures: Measures the relationship and/or value of the

program to broader business goals using existing business KPIs

Define what you will measure (below) based on

your responses (at right)


Four Takeaways for Today

We have shared:

1. Insights on the challenges and benefits of measuring

leadership development

2. The four big differences between the best

and the rest when it comes to measuring leadership development

3. Frameworks for developing a measurement plan

4. Case studies of what high performing organizations

are doing to measure their leadership initiatives


At Forum, we help our clients develop their leaders, managers and sales

teams to realize specific business objectives faster and more effectively. We

are proud to provide award-winning, comprehensive training solutions backed

by in-depth research to clients throughout North America and around the

world.

[email protected]

forum.com

KnowledgeAdvisors is a talent and learning analytics solutions

and technology firm that helps organizations measure,

communicate and improve the impact of their people by better

managing processes through reliable metrics. Its proprietary

measurement systems and benchmarking expertise help

companies more successfully manage their learning and talent

management investments.

[email protected] knowledgeadvisors.com

Questions?

Using Measurement to Drive the Impact and Effectiveness of Your Leadership Development Programs...

Education

Transcript of Using Measurement to Drive the Impact and Effectiveness of Your Leadership Development Programs...