Download PowerPoint Document

KPIDesigning Key Performance Indicators

© 2006 Dr Josephine Palermo & Martin CarrollTraining Module 03 v8

A Ministry of Higher Education &Oman Accreditation Council joint initiative

Module ObjectivesModule ObjectivesBy the end of this module, participants should:By the end of this module, participants should:1.1. Understand the role of KPI in a quality Understand the role of KPI in a quality

assurance system.assurance system.2.2. Understand the attributes of well designed KPI.Understand the attributes of well designed KPI.3.3. Recognise different types of KPI and the Recognise different types of KPI and the

consequences of using different types.consequences of using different types.4.4. Develop KPI for given goals.Develop KPI for given goals.5.5. Be able to constructively critique a KPI against Be able to constructively critique a KPI against

its intended purpose.its intended purpose.

Session OutlineSession Outline1.1. A model for designing good KPI:A model for designing good KPI:

– KPI in the planning hierarchyKPI in the planning hierarchy– Objectives, indicators, measures, targets and resultsObjectives, indicators, measures, targets and results

2.2. Conceptually understanding KPI:Conceptually understanding KPI:– Why use KPI?Why use KPI?– ADRI and KPIADRI and KPI– Lag and Lead indicatorsLag and Lead indicators

3.3. Tools for critically analysing the design of KPI:Tools for critically analysing the design of KPI:– SMARTER KPISMARTER KPI– Tips for good KPITips for good KPI

4.4. WorkshopsWorkshops

1.1.A model forA model for

designing good KPIdesigning good KPI

KPI in the Planning HierarchyKPI in the Planning HierarchyMission

Goal

Objective

Indicator

Measure

Target

Result

Strategy

A one-to-one relationship (e.g. one measure has one target)

A one-to-many relationship (e.g. one goal may have many strategies)

KPI in the Planning HierarchyKPI in the Planning HierarchyMission

Goal

Objective

Indicator

Measure

Target

Result

Strategy

KPI may be thought of as essential elements of the overall planning and monitoring system.

They are derived from the College’s Mission and help track performance towards accomplishing the Mission.

A KPI is not a single component, but an integrated collection of integrated collection of componentscomponents.

Indicator

Measure

Target

Result

Designing Good KPIsDesigning Good KPIs• Objectives: Objectives: what are we what are we trying to achievetrying to achieve??

– May be more than one indicator for each objectiveMay be more than one indicator for each objective– Each objective will have strategies on how to achieve themEach objective will have strategies on how to achieve them

• Indicators:Indicators: whatwhat are you going to measure? are you going to measure? – used to assess the present state of progress and to suggest used to assess the present state of progress and to suggest

an appropriate course of action. an appropriate course of action.

• Measures:Measures: howhow are you going to measure it? are you going to measure it?– can be qualitative or quantitative data related to inputs, can be qualitative or quantitative data related to inputs,

processes or outputs.processes or outputs.

• Targets:Targets: what is the what is the desired resultdesired result??– can be minima targets, stretch targets or a combination.can be minima targets, stretch targets or a combination.

• Results:Results: what have you what have you actually achievedactually achieved??

• ObjectiveObjective– High Quality TeachingHigh Quality Teaching

• Indicator (one of several)Indicator (one of several)– Student satisfaction with the teaching they experienceStudent satisfaction with the teaching they experience

• MeasureMeasure– mean student response, per class, to the question, mean student response, per class, to the question,

“overall, how satisfied are you with this teacher?”“overall, how satisfied are you with this teacher?”– on a 1 to 5 Likert-type scaleon a 1 to 5 Likert-type scale

• TargetsTargets– At least 3.5 on a 1 to 5 scale (where 5 is the best): ORAt least 3.5 on a 1 to 5 scale (where 5 is the best): OR– Best in class compared with benchmark partnersBest in class compared with benchmark partners

Constructing Good KPIsConstructing Good KPIs

2.2.ConceptuallyConceptually

understanding KPIunderstanding KPI

Why use KPIs?Why use KPIs?• Key Performance Indicators help us to know Key Performance Indicators help us to know

whether we are succeeding in our mission. whether we are succeeding in our mission.

• They are an important management tool for They are an important management tool for tracking progress against strategic goals.tracking progress against strategic goals.

• Working with KPIs encourages system thinking.Working with KPIs encourages system thinking.– in most systems 85% of problems can usually be in most systems 85% of problems can usually be

attributed to the system and 15% to the individual attributed to the system and 15% to the individual (Juran)(Juran)

• KPIs direct and prioritise behavior towards KPIs direct and prioritise behavior towards achievement of the vision, goals and objectives.achievement of the vision, goals and objectives.

APPROACH

RESULTSDEPLOYMENT

IMPROVEMENT

ADRI and KPIsADRI and KPIs• Each phase of the ADRI process can involve KPIs.Each phase of the ADRI process can involve KPIs.

• Examples of ‘Approach’ KPIs include:Examples of ‘Approach’ KPIs include:– Staff training program participation; Student recruitment by targetStaff training program participation; Student recruitment by target

• Examples of ‘Deployment’ KPIs include:Examples of ‘Deployment’ KPIs include:– Internet uptime; Teaching space usage; Student class attendanceInternet uptime; Teaching space usage; Student class attendance

• Examples of ‘Results’ KPIs include:Examples of ‘Results’ KPIs include:– Student pass rates; Program accreditationStudent pass rates; Program accreditation

• Examples of ‘Improvement’ KPIs include:Examples of ‘Improvement’ KPIs include:– Number of review recommendations resolved on time; Policy Number of review recommendations resolved on time; Policy

updatesupdates

Balanced Scorecard KPIsBalanced Scorecard KPIs• Kaplan & Norton designed a ‘Balanced Scorecard’, Kaplan & Norton designed a ‘Balanced Scorecard’,

with ‘lag’ and ‘lead’ KPIs. Useful for managing large with ‘lag’ and ‘lead’ KPIs. Useful for managing large numbers of KPIs in a complex organisation.numbers of KPIs in a complex organisation.

• ‘‘Lag’ indicators tell how well a system performed. Lag’ indicators tell how well a system performed. Mostly used in ADRI ‘Results’ and ‘Deployment’ Mostly used in ADRI ‘Results’ and ‘Deployment’ phases. Examples include:phases. Examples include:

– Student employment rates; research publications; number of prizesStudent employment rates; research publications; number of prizes

• ‘‘Lead’ indicators help predict system performance. Lead’ indicators help predict system performance. Mostly used in ADRI ‘Approach’ and ‘Improvement’ Mostly used in ADRI ‘Approach’ and ‘Improvement’ phases. Examples include:phases. Examples include:

– Staff training; Investment in advertising; Student-computer ratios Staff training; Investment in advertising; Student-computer ratios

Lag, Lead & the Planning HierarchyLag, Lead & the Planning HierarchyMission

Goal

Objective

Indicator

Measure

Target

Result

Strategy

Put another way, ‘lag’ KPI focus on the extent to which outcomes-based objectives have been achieved.

‘Lead’ KPI focus on the extent to which strategies for achieving the objectives are being successfully implemented.

Therefore we can structurally separate lag and lead KPI in the planning hierarchy.

Lag, Lead & the Planning HierarchyLag, Lead & the Planning Hierarchy

Strategy

Indicator

Measure

Target

Result

Lag KPI setLead KPI set

There may be separate KPI for objectives and their associated strategies.

Provided that objectives are outcomes-based,and strategies support objectives, then:

Mission

Goal

Objective

Indicator

Measure

Target

Result

3.3.Tools for criticallyTools for critically

analysing KPIanalysing KPI

SSMMAARRTTEERR

pecificeasurablelignedealisticimelythicalecorded

Other Tips for Good KPIsOther Tips for Good KPIs1.1. Have them!Have them!2.2. Align them to standards.Align them to standards.3.3. Test them for validity and reliability.Test them for validity and reliability.4.4. Discuss and review them. Are they really Discuss and review them. Are they really keykey? ?

Don’t overestimate their importance!Don’t overestimate their importance!5.5. Differentiate between lag and lead indicators.Differentiate between lag and lead indicators.6.6. Benchmark them.Benchmark them.7.7. Do something with them. KPI should lead to change.Do something with them. KPI should lead to change.8.8. Keep them simple – but not too simple!Keep them simple – but not too simple!9.9. Balance focus on positive and negative news (i.e. Balance focus on positive and negative news (i.e.

opportunities for improvement).opportunities for improvement).

Humanise the ProcessHumanise the Process• Address staff concerns and ideas about Address staff concerns and ideas about

performance management.performance management.• Address concerns about the uncertainty of the data.Address concerns about the uncertainty of the data.• Regardless of the topic, keep performance review Regardless of the topic, keep performance review

processes fair and transparent.processes fair and transparent.• Place performance review within a context of Place performance review within a context of

learning.learning.• Enable a two way dialogue about performance.Enable a two way dialogue about performance.• Remember to highlight good news!Remember to highlight good news!

Workshop ExampleWorkshop ExampleIndicators for Learning and TeachingIndicators for Learning and Teaching

Objectives

1. Students will learn to solve problems.

2. The program will be viable.

3. The program will be relevant to industry and employers.

ExampleExample• For each of these three

objectives, strategies are provided.

• KPIs are then constructed using the model:indicator → measure → target

• KPIs are then tested using SMARTER

Workshop Example #1Workshop Example #1Indicators for Learning and TeachingIndicators for Learning and Teaching

Objective Strategy KPIs

Students learn to solve problems

Design and deliver problem based curriculum

Indicator: courses include learning outcomes that relate to problem solving

Measure: number of courses each semester with problem solving learning outcomes included in institutional course guides

Target: 100% of new courses; 100% of all courses by 2010

Design and implement problem based assessment of student learning

Indicator: courses include PBL assessment exercises

Measure: student pass rate/semesterTarget: 80% by 2007

SPECIFIC?Yes. The indicator is tightly focused on the

single issue of courses design.

MEASURABLE?Yes. The number of courses that comply

can easily be counted.

ALIGNED?Yes. The measure

relates directly to the indicator; the target

relates directly to the measure.

REALISTIC?Yes. The KPI has been made

realistic by separating new courses, which can implement

the strategy directly, from existing courses, which will

need to be changed.

TIMELY?Yes. The measure is

each semester, allowing for current

data and trend data. The target is time-

specific. ETHICAL?There are no obvious ethical concerns with collecting and using

this information.

RECORDED?The target implies that trend data will

need to be kept and analysed until at

least 2010.

TIMELY?Yes. The measure is

each semester, allowing for current

data and trend data. The target is time-

specific.

REALISTIC?Yes. The target does

not appear unachievable.

SPECIFIC?Yes. The indicator is tightly focused on the

single issue of assessing student’s problem solving

abilities.

ALIGNED?Yes. The measure

relates directly to the indicator; the target

relates directly to the measure.

MEASURABLE?Yes. Student pass rates are

easily measured.

RECORDED?The target implies that trend data will need to be kept and analysed

until at least 2008.

ETHICAL?There are no obvious ethical concerns with collecting and using

this aggregated information.

These are SMARTER KPIs

These are SMARTER KPIs

Objective Strategy KPIs

The program will be viable.

Attract the best students by direct marketing of programs to school leavers.

Indicator: Student satisfaction.

Measure: (a) Student pass rates(b) Excellent program

Target:(a) 100%(b) All staff qualified

Workshop Example #2 Workshop Example #2 Indicators for Learning and TeachingIndicators for Learning and Teaching

TIMELY?No. There is no time definition

provided for this KPI

REALISTIC?No. Any national or

international benchmark will show that the target

of 100% pass rates is not realistic.

SPECIFIC?No. ‘Student

satisfaction’ is very broad. Satisfaction with

exactly what?

ALIGNED?No. Student satisfaction and student pass rates are not the same thing.MEASURABLE?

Yes. Student pass rates are

easily measured.

RECORDED?There is no indication on how this KPI will be

recorded, reported and stored.

ETHICAL?Maybe not. The

target of 100% may encourage staff to

mark too softly.

TIMELY?No. There is no time definition

provided for this KPI

REALISTIC?No. If the measures and

targets cannot be measured and do not

align, then how can they be realistic?

SPECIFIC?No. ‘Student

satisfaction’ is very broad. Satisfaction with

exactly what?

ALIGNED?No. Staff qualifications

is not necessarily related to whether the program is excellent.

MEASURABLE?No. What is the

means for measuring that the

program is ‘excellent’?

RECORDED?There is no indication on how this KPI will be

recorded, reported and stored.

ETHICAL?No. This KPI implies that staff

can be held accountable for ‘student satisfaction’ and the ‘excellence’ of the program

and by virtue of their qualifications. The case for

this is not justified here.

These are not SMARTER KPIs

These are not SMARTER KPIsLet’s improve them…Let’s improve them…


Goal Strategy KPIsThe program will be viable.

Attract the best students by direct marketing of programs to school leavers.

Indicator: Student program preferences by academic performance.

Measure: (a) Percentage of enrolled students who

listed the program as their 1st preference (via Admission Centre)

(b) Proportion of 1st preference students in national top 10% by academic rank

Target:(a) More than 50%(b) At least 20% and up to 40%

(assuming four competitor programs in Oman)


Goal Strategy KPIs

The program will be relevant to industry and employers.

Establish Industry Advisory Boards (IABs) for each program. Invite key employers to be members.

Indicator: Graduate employability rateMeasure: % graduates in full time

employment in the industryTarget: above national averageIndicator: Acceptance of program by

industryMeasure: (a) Professional accreditation of

program by 2009(b) Positive feedback from employers

and industryTarget: (a) Professional accreditation passed(b) Positive IAB Program Review

1.1. Teaching and Learning (done)Teaching and Learning (done)

2. Student Outcomes

3. Educational Support Resources

4. Research and Research Training

5. Community Service / Engagement

6. Organisational Leadership

7. Staffing

8. Institutional Infrastructure

9. Information and Communication

Workshop Instructions Workshop Instructions Select your topicSelect your topic

ActivityActivity• Select an area (from the list) that you

are currently working on.• What are two or three main objectives

for that area?• What strategies could you have in

place to achieve each objective?• Construct appropriate KPIs, including

Indicators → Measures → Targets.• Check them. Are your KPI SMARTER?• Are they Lag or Lead KPI?• When you have finished (and ONLY

then!), try clicking each area for further examples of good KPI.

• When you are ready you may EXIT

Workshop ActivitiesWorkshop ActivitiesStudent Outcomes (a)Student Outcomes (a)

Objective Strategy KPI

Students will achieve high rates of success in their learning

Progress Scheme identifies ‘at risk’ students, by Department, and invites them to participate in student learning support services.

Indicator: Student progress rates

Measure: Percentage of students progressing to the next level of study without having to repeat any units; differentiated by those who participated in the Progress Scheme and those who didn’t.

Target: Improvement trends over three years for both cohorts, but steeper for Progress Scheme students.

Workshop ActivitiesWorkshop ActivitiesStudent Outcomes (b)Student Outcomes (b)


Students will be highly satisfied with their career choices

Academic Advisors are available for career planning

Indicator: Graduate recruitment by employment preference

Measure: Percentage of graduates in preferred employment within 12 months of graduating

Target: At least 90%

Returnto list

Workshop ActivitiesWorkshop ActivitiesEducational Support ResourcesEducational Support Resources


The library supports learning outcomes

The library has developed a ‘one-stop-shop’ help desk for increased support to academic staff and students

Indicator: Staff and student satisfaction with the help desk

Measure: Library User Survey asks staff & students to rate their satisfaction with and relevance of helpdesk features. Gap measure is developed.

Target: The features with highest relevance also have equally high satisfaction

IT services support learning outcomes

Actively communicate with College departments to identify IT requirements and problems with IT services

Indicator: Client (Dept) satisfactionMeasure: Academic staff satisfaction

results on customer feedback surveys

Target: Average satisfaction is above 4 on a 1-5 likert type scale

Returnto list

Workshop ActivitiesWorkshop ActivitiesResearch and Research Training (a)Research and Research Training (a)


We will improve the level of research income via external grants

Staff can access institutional seed grants money to encourage external applications for research funding.

Indicator: External grant applications

Measure: (a) Proportion of FTE Teaching+

Research staff submitting an external grant application (Lead)

(b) Proportion of applications successful (Lag)

Target: (a) 50% of all FTE staff submitting an

external grant by 2010.(b) 20% success rate

Workshop ActivitiesWorkshop ActivitiesResearch and Research Training (b)Research and Research Training (b)


We will improve the postgraduate research experience

System of bi-annual progress reporting for HDR students that includes follow up and evaluation processes

Indicator: Students successfully completing postgraduate research degrees in a timely fashion

Measure: Proportion of HDR students graduating each year against total number of HDR students scheduled to complete each year.

Target:(a) at least 30 by 2010(b) Over 75% scheduled students

graduate

Returnto list

Workshop ActivitiesWorkshop ActivitiesCommunity Service / Engagement (a)Community Service / Engagement (a)


We will provide leadership to our local community

Staff are encouraged to nominate for community level boards and committees

Indicator: Participation on community boards / committees

Measure: Proportion of FTE staff (academic + non academic) who hold positions on boards and committees)


Click here to return to list

Returnto list

Workshop ActivitiesWorkshop ActivitiesCommunity Service / Engagement (b)Community Service / Engagement (b)


We will increase the number of new enrolments from students in the local region

Schools liaison program is conducted with schools in the College’s catchment area.

Indicator: Number of commencing students from our region

Measure: (a) Proportion of local region new

enrolments against College’s total new enrolments

(b) Proportion of local region new enrolments against total local region school leavers

Target: (a) At least 30% in 2007; with an

increasing trend of 3% every year for next three years.

(b) At least 50%; and more than any other local College.

Returnto list

Workshop Activities Workshop Activities Organizational LeadershipOrganizational Leadership


Strategic planning is practiced across all boards and committees

Every Board and Committee is required to develop a Plan that is aligned to the College Strategic Plan and includes performance indicators

Indicator: Board and Committee Plans

Measure: Proportion of Boards and Committees with a current Plan that complies with the College’s template

Target: 100%We will create and maintain a positive organisational climate

The leadership team meets monthly to identify, discuss and respond to any issues raised by staff about the organisational climate, and to review progress on previous responses.

Indicator: Staff satisfaction with their organisational climate.

Measure: Mean staff responses to questions about climate.

Target: Mean satisfaction is at least 4 on a 1-5 Likert-type scale.

Returnto list

Workshop ActivitiesWorkshop ActivitiesStaffing (a)Staffing (a)


People are placed in meaningful jobs

Staff are recruited into positions that are related to goals in the College Strategic Plan.Exit interviews are conducted with staff who leave the organisation.

Indicator: Staff turnover due to poor recruitment practices

Measure: (a) expenditure related to staff

turnover (eg. Recruitment expenses into recurrent jobs, retrenchment pay)

(b) Number of staff who resign each year

Target: (a) No more than 0.5% total

annual budget(b) Reduce by 3% per year

Returnto list

Workshop ActivitiesWorkshop ActivitiesStaffing (b)Staffing (b)


Staff skills and competencies are developed on the job

Professional development is a high priority within the organisation. All staff have access to a professional development program.

Indicator: Staff development activity

Measure: Annual percentage of all staff who participated in professional development


Returnto list

Workshop ActivitiesWorkshop ActivitiesInstitutional InfrastructureInstitutional Infrastructure

Objective Strategy KPIsSound financial management is practiced

All budget centre managers are trained in the College’s financial management processes.

Indicator: Expenditure against budget.

Measure: Monthly variance of expenditure against budget; reported by account code.

Target: Expenditure no more than 5% over or 10% under budget.

Effective space management supports learning outcomes

Space planning is incorporated into the Colleges Academic Plan

Indicator: Occupancy Adjusted for Usage Measure: The total amount of people using

the room over the total capacity of the teaching space.

Target: Usage at least 60% aggregated across all teaching spaces

Returnto list

Workshop ActivitiesWorkshop ActivitiesInformation and CommunicationInformation and Communication

Objective Strategy KPIEffective marketing maintains the high profile of the institution

Media and Marketing Strategy clearly targets media outlets

Indicator: Level of media awareness of the institution

Measure: (a) Number of ‘hits’ in targeted media

outlets (eg newspapers, radio)(b) Ratio of positive to negative ‘hits’Target: (a) increasing over time(b) Greater than 60%

Messages about quality assurance from management are consistently deployed to staff

Heads of Depts hold regular staff meetings where QA issues are discussed

Indicator: Evidence of discussions about QA policies and strategies in staff meeting minutes

Measure: Number of times QA issues are addressed in minutes

Target: At least once per month

Returnto list

KPIDesigning Key Performance Indicators

Congratulations on completing thistraining module.

Download PowerPoint Document

Technology

Transcript of Download PowerPoint Document