Metrics That Matter Learning Analytics

The standard inLearning Measurement

©2003 KnowledgeAdvisors. All rights reserved.

Metrics That MatterLearning Analytics

Web-enabled automated learning evaluation process Post-class, follow-up and bespoke surveys Detail, summary and management reporting Internal and external benchmark comparisons Analysis tools Gives a better insight into training’s Key Performance Indicators Demonstrate the business value of training Improve performance of the enterprise Better use of training resources and budgets Increase support from across the enterprise for training

©2003 KnowledgeAdvisors. All rights reserved.

MethodKirkpatrick methodology and Phillips ROI layer… model shows an IIP ‘Level 0’ layer

Level 1

Level 2

Level 3

Level 4

Level 5

Did they like it?Course content, instructors, course materials venue, LMS experience, etc

Did they learn?Test knowledge, intentions, relation to own situation, application

Do they use it?Time to job impact, level of job impact, barriers to use

What were the results?Sales, services, quality, cycle time, errors,timescale, etc

What’s the ROI?Financial return, timescale, value, human capital value

Needed by managers > > >

pro

vid

ed t

o m

anag

ers

> >

>

Level 0

Learning objectivesManager completes with delegate

©2003 KnowledgeAdvisors. All rights reserved.

Automated process

Data Collection Data Storage Data Reporting

Post Event SurveyFollow Up SurveyManager SurveyInstructor SurveyScored TestingNeeds AssessmentsPhillips ROI Input TemplatesPaper, web, emailLMS Integration

ASP solutionSQL Server Centralized storageSecure environmentRaw data access

Web-enabled reportsPush email alertsPhillips ROI reportCustom queryCustom reportsAnalytical toolsForecast calculator

Data Processing

Sophisticated queriesOLAP technologyMultiple aggregationsMultiple cuts / views

Because data collection, processing and reporting is automated, you have more time to analyze the data and

turn it into actionable business intelligence

Automated

Valued time

©2003 KnowledgeAdvisors. All rights reserved.

Post event learner survey - corporate providers

©2003 KnowledgeAdvisors. All rights reserved.

Follow up survey - learner

©2003 KnowledgeAdvisors. All rights reserved.

Follow up survey - manager

©2003 KnowledgeAdvisors. All rights reserved.

Comparative performance/standard reports

Client A

Client B

Client C

Client D

Client E

©2003 KnowledgeAdvisors. All rights reserved.

Performance to goals reports

Allows you to establish pre-set goals for each question category on the Post event and Follow up surveys and compare them to actual performance.

©2003 KnowledgeAdvisors. All rights reserved.

Variance reports

©2003 KnowledgeAdvisors. All rights reserved.

Learning levels score card – K1-5

©2003 KnowledgeAdvisors. All rights reserved.

Top/bottom report

Wade MediaTonnell GrpJKT North

Paris TextileQamoliDent Morcam

Customer

NottinghamAberdeenChester

LondonNorwichLiverpool

©2003 KnowledgeAdvisors. All rights reserved.

When is the time to job impact? This lets you know if and when training was applied to the job. It is collected months after the training, automatically by Metrics that Matter. It helps the organization understand when training effects the job, if at all.

How much more productive are employees? You can track specific business results, such as productivity. Asking a manager or a participant to estimate productivity once time has passed on the job is a good indicator of learning’s impact to a specific business objective.

What percent of skills were applied to the job? This indicator provides insight into the amount of new knowledge/skills picked up in the training for each course (or curriculum or program) that the participant directly applied when they went back on the job as the data is collected several months after the training when the participant is back on the job.

©2003 KnowledgeAdvisors. All rights reserved.

How is training aligned with business objectives?

©2003 KnowledgeAdvisors. All rights reserved.

ROI Score cardThe scorecard is a great management tool to showcase value to key stakeholders. This report shows you each level of learning and the Key Performance Indicators within each level.

Level five shows quantitative ROI metrics including:

Benefit to Cost RatioPayback PeriodROI%

©2003 KnowledgeAdvisors. All rights reserved.

Push Email Reporting

Allows you to setup reports to be sent to you via email at anytime you desire so you do not have to go into MTM to retrieve your favorite reports.

©2003 KnowledgeAdvisors. All rights reserved.

Phillips ROI Toolset in MTM

A summary of the ROI Methodology to use as a reading reference

Examples of case studies applying the Phillips ROI Methodology

A list of selected publications of Jack and Patti Phillips

The authorized Phillips End of Program Data collection instrument

The authorized Phillips Follow Up Data collection instrument

The authorized Phillips Cost Calculation Template

The authorized Phillips Follow Up Data collection instrument

KnowledgeAdvisors will be working with Jack and Patti Phillips to continue adding additional tools with each quarterly update to the Metrics that Matter software!

©2003 KnowledgeAdvisors. All rights reserved.

Is training aligned with business results?Survey

©2003 KnowledgeAdvisors. All rights reserved.

Phillips ROI Toolset in MTM

©2003 KnowledgeAdvisors. All rights reserved.

Sample clients

©2003 KnowledgeAdvisors. All rights reserved.

Contact Information

For further information on Metrics that Matter please contact:

New Wave Learning

4a Flemming Court, Glasshoughton

Castleford, West Yorkshire.WF10 5HW

Tel: 01977 628980

Email: info@newwavelearning.co.uk

Website: www.newwavelearning.co.uk