Metrics that Matter™ Multi-rater Assessments

Setup Guide

Metrics that Matter for MCTs

Contents

Essential MTM Links 3 MCT Requirements - Overview 4 Accessing MTM as an MCT 5

First Steps: Registering for MTM 5 Managing your MCT Profile 6 How to Schedule Classes in MTM 7

Scheduling Classes at a LEARNING PARTNER 7 Acknowledging a Certified Class 7 Cancelling a class that you will not be teaching 7 Schedule Classes at Non-LEARNING PARTNER Location 8 Review Your Class Criterion 8 Add Client/Customer Information 9

How to Use MTM in the Classroom 10 Use Metrics that Matter in the Classroom 10 Introduce Metrics that Matter to Students 11

Using MTM Reports 12 Accessing Reports in MTM 12 Important Reports in MTM 13 Scheduling Reports for Auto-Delivery by E-mail 16 Using the Filtering Tool in MTM 17

Additional Information and FAQ 19 Quick Reference Card 19 FAQ 19 Additional Services Available Error! Bookmark not defined.

Appendix 20

Metrics that Matter for MCTs

Essential MTM Links

• MCT Registration Page www.metricsthatmatter.com/NewMCTInMTM/EnrollMCT.aspx

• Resetting your MTM Password www.metricsthatmatter.com then click Forgot Password?

• MCT Reporting Log In www.metricsthatmatter.com/

• MTM Help Guides knowledgeadvisors.force.com/mtmhelp/pkb_Home?q=mct&l=en_US

• MTM User Support (Monday – Friday, 8:30 AM – 5:00 PM CT) MTMsolutions@explorance.com +1 (800) 561 3341 (within the US) +1 (312) 676 4400 (worldwide)

Metrics that Matter (MTM) Instructor Checklist:

Quick Reference Card Essential MTM Links

Metrics that Matter for MCTs

MCT Requirements - Overview

What is required of each MCT in using Metrics that MatterTM? The following is an overview of the MCT requirements regarding MTM. Each of these steps is explained in detail in the following pages of this guide.

1. Registration: You must receive a login ID from Metrics That Matter to access to system.

a. New Instructors: www.metricsthatmatter.com/NewMCTInMTM/EnrollMCT.aspx

b. Existing Instructors: Using the Profile Management tool, add your MSP # to your existing MTM account

2. Manage MCT Profile: (2 minutes) Log in and modify your profile. This must be done

for your information to be included in MCT reporting, without which clients will be unable to schedule you for training. This includes updating your MC ID, and your home region.

3. Manage Classes: (1 minute per class) Each class will need to be scheduled in the

MTM system prior to collecting evaluations. Classes will be scheduled by either by the LEARNING PARTNER, or by you, the Instructor.

4. Provide students with the URL to submit evaluations: (2 minutes per class)

Instructors are responsible for providing all students with the appropriate URL for submitting class feedback. You can find the URL by logging into your account, or the LEARNING PARTNER scheduling the class may provide it to you for distribution to your students.

5. Monitor results: (Ongoing) As an MCT you will also have the ability to monitor all of

the feedback entered by students in any class that you teach.

Metrics that Matter for MCTs

Accessing MTM as an MCT Accessing MTM as an MCT

Registering for MTM

You must have an MTM login ID and password to access the MTM system.

Step 1: Go to the MCT Registration Page: www.metricsthatmatter.com/NewMCTInMTM/EnrollMCT.aspx

Step 2: Fill in all requested fields. The following fields are required:

• Full name

• Email*

• MC ID** The following optional fields are also very useful, if applicable

• Your current MTM login email address***

• Date of first class ****

• Region

* Email address: the login information to access the system will be sent via email to the address provided here.

**MTM will automatically verify MC IDs with the Microsoft database, this enables your MTM courses and evaluations to be linked to your MC ID for annual renewal requirements. ***You may have already been provided an MTM user account through a LEARNING

PARTNER if you have already used the system to collect feedback in the past. Please provide that login information here, if applicable.

**** Date of first class taught in MTM: this information will only be used to help prioritize

requests. Step 3: Receive login information. Approximately one week from date of registration.

Step 4: Log in to Metrics that Matter and update your profile. See instructions under “How to Manage Your Profile” on page 6 of this document.

Regarding Lost or Forgotten Logins If you have lost or forgotten your MTM login, please contact MTMsolutions@explorance.com to reset your credentials. .

Metrics that Matter (MTM) Instructor Checklist:

Quick Reference Card Accessing MTM as an MCT S

Metrics that Matter for MCTs

Managing your MCT Profile An MCT’s profile in MTM includes all of the information that will be used to identify the instructor in the system, both for online reporting features as part of the MCT program, and for scheduling purposes at a LEARNING PARTNER. It is important that this information is updated as soon as login information is received –LEARNING PARTNER are unable to schedule classes for an instructor until registration is completed – this information is not available in MTM until updated by the MCT. By providing your name and MC ID to a LEARNING PARTNER you are giving them permission to find your name and schedule you for training. How to Start Managing your Profile in MTM:

1. Go to the MTM Login Page at: www.metricsthatmatter.com 2. Type in your user name and password 3. Arrive at the Dashboard tab in MTM. Click

on the My Settings tab 4. You will arrive at the My Settings Menu 5. Verify that your name is accurate, and

reflects the name you will plan on having the LEARNING PARTNER do a search on.

6. Update Email & Password information if necessary

7. Click Update Settings to commit any changes 8. Navigate to the My Setting menu and click on My Instructor Certifications 9. Input your MC ID and MCT Region to reflect those assigned by Microsoft. Click

Update Settings to commit any changes 10. If your MC ID is not accurate, profile changes will not save

Metrics that Matter (MTM) Instructor Checklist:

Quick Reference Card Managing your MCT Profile

Metrics that Matter for MCTs

How to Schedule Classes in MTM

When Scheduling Classes at a LEARNING PARTNER

In order for a LEARNING PARTNER to receive credit from Microsoft for offering the course you will be teaching, the courses should be scheduled under the LEARNING PARTNER’s account.

1. Provide the Last Name, as it appears in MTM, and your MC ID to the LEARNING

PARTNER MTM Administrator. They will use this to identify your user account in MTM a. Note: The LEARNING PARTNER will not be able to schedule you for a class

until you have entered your MC ID into your MTM account (see How to Manage Your Profile above).

2. Confirm with the LEARNING PARTNER that the correct official courseware has been scheduled for this class

3. An email will notify you that you have been scheduled for the class.

Note: Classes may be scheduled at separate Learning Provider

Acknowledging a Certified Class

Once the class has been scheduled, you must acknowledge that it is scheduled correctly and linked to your class.

1. Click the Administration tab in MTM

2. From the Schedule Training Menu, select Acknowledge Class from the Schedule

Classes box

3. Edit the class Start/End date to include the date of the class 4. Optionally, input specific information like the Course Name in the search 5. Click Search 6. From the list of classes, select the specific one in question and click Acknowledge

Selected

If you are scheduled for a class that you will not be teaching

1. Click the Administration tab in MTM 2. From the Schedule Training Menu, select Acknowledge Class from the Schedule

Classes box 3. Edit the class Start/End date to include the date of the class 4. Optionally, input specific information like the Course Name in the search 5. Press Search 6. Click the Remove button to reject the class

Metrics that Matter (MTM) Instructor Checklist:

Quick Reference Card How to Schedule Classes in MTM S

Metrics that Matter for MCTs

How to Schedule Classes at Non-LEARNING PARTNER Location

If an instructor is delivering training for a customer that does not utilize the Metrics that Matter system, then you - the instructor - will be responsible for scheduling each delivery in MTM. This describes the process that an independent instructor will follow to schedule classes when their customer/learning provider is not using MTM. Note: that by following this method only the instructor will be able to collect data for this class – not the learning provider.

1. Click the Administration tab in MTM 2. From the Schedule Training Menu, select Add from the Schedule Classes box

3. Chose a Client/Customer from the Drop Down Box* a. If there are no Client/Customers listed in the drop down box, you must Create a

Client/Customer. 4. Choose your correct Course from the Drop Down Box* a. Note that only Microsoft Official Courseware is available for scheduling 5. Chose a Start Date for the class. This is the day the class begins.* 6. Chose an End Date for the class. If the class finished the same day, this can be the

same as the Start Date* 7. Choose the appropriate Learning Methodology* 8. Select whether you would like to limit the number of Evaluations Submitted to the

# of Students Enrolled 9. Include the # of Students Enrolled in the course. 10. Click the Add Button. * Required Information

How to Review Your Class Criterion 1. Click the Administration tab in MTM 2. From the Schedule Training Menu, select Search from the Schedule Classes box 3. Chose the Start Date for the class

4. Chose the End Date for the class

Metrics that Matter for MCTs

5. Click Search

6. The Edit action mirrors the Class Add menu 7. The Summary action will provide you with an overview of the class information. It will

also provide you the information regarding the surveys attached to your class. Of note is the generic, direct link to your class survey

8. The Audit page identifies actions taken by users to the class details such as “add” or “modify” changes

How to Add Client/Customer Information Your classes must be tied to a Client/Customer before you can successfully schedule a class.

1. Click the Administration tab in MTM 2. From the Schedule Training Menu, select Add from the Schedule Classes box 3. From the Class Add screen, click the link in the upper right hand corner labeled

“Create a Client/Customer”

4. The Add Client / Customer screen consists of the following information: 5. Client/Customer Name (Example: Learning Provider Organization or Company for

whom the training is being provided. This must be a unique field)* 6. Site Name* (Example: A specific “location” at which the training takes place) 7. City* 8. Time Zone 9. Language * Required Information

Metrics that Matter for MCTs

How to Use MTM in the Classroom How to Use Metrics that Matter in the Classroom

To collect feedback from the students you will need to provide them with a URL to access the evaluation form online.

Step 1: Generating the URL. The URL that should be given to the students can be found in one of two ways: 1. If at a LEARNING PARTNER, the center will provide each instructor with the

appropriate URL Or –

2. The instructor can click Manage Classes. Search for the class, then select the Summary action and click Go. See page 9 of this guide which displays the exact location of the URL.

Step 2: Providing the URL to the students. This can be provided to the students a few ways:

• (Do we want these options?)Distribute via email to each student so they can click on the link and access the webpage

• Save the URL as a favorite or the homepage on the machines in the classroom. NOTE: This is the most preferable option as it eliminates the possibility of errant typos by the student.

Step 3: Accessing the Student URL online. When the students first access the site, they will see a screen that asks for their email address. Please explain that this is only for the purpose of sending a follow up evaluation that will be sent 60 days after training has completed.

Step 4: Introducing the Evaluation to the students in the classroom. The next page shows an Instructor Checklist that can be used to make sure that the main talking points are covered with the students.

Metrics that Matter (MTM) Instructor Checklist:

Quick Reference Card How to Use MTM in the Classroom S

Metrics that Matter for MCTs

How to Introduce Metrics that Matter (MTM) to Students

• Metrics that Matter (MTM) is the online course evaluation tool we are using to collect all evaluations. It is:

• Faster than completing a paper evaluation

• More confidential

• A more accurate way to collect data

Instructions to Students

Let’s review the steps to complete your evaluation.

1. Go to URL address __________________________. Reminder - ALL data is confidential.

2. Input your (student) Email address - used only for a follow-up evaluation: Instructor Note - Sixty days following a learning event, students will be sent a brief, follow-up evaluation and again, responses will be kept confidential. Emails will not be used for marketing purposes. (If student does not have email address, user may type in dummy address). It is very important to Microsoft that data be collected both on the Post event and Follow Up evaluations therefore please encourage students to enter their email address.

3. Select the appropriate class from the drop-down list. 4. Please complete your evaluation. 5. Click Submit upon completion.

Do not click Submit multiple times, it may take a few seconds for the next screen to appear.

6. Wait for Thank You to appear on the screen notify the instructor to receive your certificate.

Thank You for attending the Class!

Metrics that Matter for MCTs

Using MTM Reports

Metrics that Matter has many online reporting features available for tracking your performance. You can use this information to improve future training programs and prove the value that you provide as an instructor. The reporting suite looks at survey responses from students and provides insight into the results of the class. Microsoft uses particular measurements in order to evaluate different elements of a class, including Instructor performance. Following are the steps an MCT can follow in order to view the Instructor Performance score most valuable to Microsoft.

How to Access Reports in MTM 1. From the Dashboard tab, click the Reporting tab

2. Under the My Recent Reports tab will display any recently ran reports. If you haven’t

recently ran any reports, it will provide a list of Featured Reports, which will be most

useful to you as an MCT

3. Clicking on any of the report names will enable you to open the report and begin

inputting your criterion to retrieve your class data.

Metrics that Matter (MTM) Instructor Checklist:

Quick Reference Card Using MTM Reports S

Metrics that Matter for MCTs

Important Reports in MTM Access all of these reports from the My Recent Reports Menu under the Reporting tab

Evaluation Retriever

View by class and by student all evaluations submitted by students. Useful for the

most granular reporting needs.

a. Chose a Start and End date in which your class was taught

b. Click Search to produce a list of all the classes you taught during that timeframe

c. Click on the class that is populated in the list below the Search button

d. If you would prefer to have this e-mailed to you instead of shown in MTM, select

Auto Email in the Report Output box

e. The name of the report is customizable, so identifying information can be

included to help make the reports unique

Class Summary Report

Summarize class data by Question Category and by Specific Question

a. Chose a Start and End date in which your class was taught

b. Click Search to produce a list of all the classes you taught during that timeframe

c. Click on the class that is populated in the list below the Search button

Metrics that Matter for MCTs

d. Chose which of the Question Categories you would like to include in the query.

This can be as few as one question category, and as many as all.

e. Click Generate Report

Learner Comments

View the comments students have included in open ended questions on the class

survey. Comments are grouped by question at the course or overall level.

a. Used to view the comments students have included in open ended questions on

the class survey

b. Chose a Start and End date in which your class was taught

c. Click Search to produce a list of all the classes you taught during that timeframe

d. Click on the class that is populated in the list below the Search button

Quick Question Report

Summarize all responses to questions for a single class or custom query of classes

a. Used to summarize all question responses onto one report

b. Used to view the comments students have included in open ended questions on

the class survey

c. Chose a Start and End date in which your class was taught

d. Click Search to produce a list of all the classes you taught during that timeframe

e. Chose which of the Question Categories you would like to include in the query.

This can be as few as one question category, and as many as all.

f. You may also chose to Aggregate (sum) the instructor data in this report, and

optionally view All Comments.

g. Click Generate Report

h. Will display the number of responses to each answer selection, as well as the %

of respondents and the average student score.

i. Available at the end of the report is a link to View Comments as well.

Variance Report

Measures the percentage of learners whose responses fall within the bounds set in

the report. Excellent resource to measure response data associated with your Overall

NSAT measurement.

a. Excellent resource to measure response data associated with your Overall NSAT

measurement.

b. In Report Type, select Instructors for an overall view of performance, and

Course or Class for more granular levels

Metrics that Matter for MCTs

c. In the Filter section,

i. Select Microsoft and Courseware Library – MS as the Content Provider

ii. Select “Overall, how satisfied are you with this course? [Instructor

Data]” as the Question

iii. In the Pop-up Box, click the radio button for “My report should display all data

from students that selected these answers for this question only”

iv. Ensure all check-boxes are marked

v. Click Close

vi. The variance report will display your question scores so that

1. VSAT Scores (8 and above) = Green

2. SAT Scores (Between 4 and 8) = Yellow

3. NSAT Scores (Lower than 4) = Red

4. Note that this is only representative of the part of Overall NSAT

calculation associated with this question in the evaluation

Metrics that Matter for MCTs

Scheduling Reports for Auto-Delivery by E-mail

A powerful tool to employ to greatly reduce the amount of time spent administering reports

can be the utilization of Automatic Report Deliver function.

1. From the Reporting Tab, under the Manage Reports menu, select Schedule Class

Reports

2. From this tool, any of these reports can be scheduled to be automatically delivered to

the e-mail of your choice

3. To enable the reports to be scheduled automatically, check the Schedule in MTM

box

4. Select the # of Days after Class Ends to determine when you’ll receive the report.

Most students will respond to surveys within 7 days. Note that more data may be

collected after the report is sent.

5. Also select what Time of Day the report is delivered to your inbox and whether the

report is attached to the e-mail or embedded

6. In the Filter / Email Settings, chose the same Query criterion as you would in the

report query. Including Filtering Preferences

Metrics that Matter for MCTs

Using the Filtering Tool in MTM

The filtering tool allows the user to select specific criterion to include into their reports, and is

used in any of our manually ran Aggregate Reports, such as the Variance Report, or on the

Scheduled Class Based Report.

1. From the Reporting Menu, chose an Aggregate Report, such as the Comparative

Performance. This report will show you the average score of different criterion in a

graphical way.

2. At the bottom of this Report Query is the My Filters box. This will include any filters

that the user has created, and any filters that have been created for the user by a

system administrator (i.e. at a LEARNING PARTNER)

3. Clicking the Create New Filter button will open the Filter Dialog.

Each Filter Attribute opens a separate dialog box, populated with all the values of

that filter associated with your user.

Metrics that Matter for MCTs

4. Find the criterion you would like to filter to from the Available Filter Items and check the box to include it. Click Add to Filter.

5. Once finished click Save.

6. Once all the criterion for filtering have been selected, chose a Filter Name, and Save the

filter.

7. The filter will now be available in the My Filter box. To include it in the Report Query,

select the Checkbox next to the Filter.

8. Click Generate Report.

Metrics that Matter for MCTs

Additional Information and FAQ

How to Get Help on Metrics that Matter

The following support tools are provided and available to you: Welcome Kit: Introduction to Metrics that Matter for Microsoft Certified Trainers User Guides: Available online using your login ID and Password by clicking on:

• Help at the top right hand orner of MTM.

• The Help tab available from within every tool, for help and information on that specific tool.

• Using the Search feature available from the top blue selection bar on each page of the system.

• Explorance, MTM Support (Monday – Friday, 8:30 AM – 5:00 PM CT) MTMsolutions@explorance.com +1 (800) 561 3341 (within the US) +1 (312) 676 4400 (worldwide) * You will receive a response within 1 business day of your query.

FAQ Q. I have Multiple MTM Accounts or was given a new account by a LEARNING PARTNER. How do I manage my information? A: In cases of multiple accounts that have collected data about your classes, MTM team can merge the multiple accounts into should one. Please contact MTM Support at MTMsolutions@explorance.com and provide your email address that should become primary. While this data cleanup is taking place, collect data under your primary account and provide that account data for any new classes for which you are scheduled. Q: How can I review students’ feedback immediately after class and evaluations were entered? A: The last 5 classes that you’ve collected data for will be found by going to Legacy > Legacy Reports > (on the left pane) Shor All Reports, then find the following reports on the right window:

• Evaluation Retrieval – Displays individual evaluations for each student

• Class Summary – Combines all students’ responses for the class into one report

• Learner Comments – Displays all comments from students regarding the class

• Quick Question – Shows distribution on for the scale-based questions and class averages.

• Performance to Goals – Compares class scores to user preset goals through Configure Reports > Set Goals tab.

For additional information on the Reports available in your account, please access MTM Help and review the Reporting Catalog.

Metrics that Matter (MTM) Instructor Checklist:

Quick Reference Card Additional Information and FAQ

Metrics that Matter for MCTs

Appendix What is Metrics that Matter?

Explorance is a practical, scalable and repeatable way to quantify learning investments. Our proprietary Metrics that Matter® technology helps organizations continuously measure the satisfaction, effectiveness, impact, business results and ROI on learning investments. Through customizable templates and creative technology, Metrics that Matter™ automates the administrative aspects of measurement by collecting, storing, processing and reporting results—leaving time for analysis, training improvements and showcasing value to stakeholders.

Metrics that Matter™ (MTM) is Explorance’s market leading learning analytics and measurement tool – an online system that automates the process of collecting and analyzing feedback from learners. MTM wraps technology around timeless and industry proven methodologies. The result is a powerful way of measuring the impact of training at the point of delivery and back on the job. Leveraging standard data collection instruments that can be customized to gather data from various respondents (learners, managers) at various points in time (at time of training, back on the job), users of MTM can easily view over 100 web-based reports. Data from LMS, ERP, CRM, MRP, and other systems can be integrated into MTM for more comprehensive analysis. Linking measurements back to the Learning Levels gives the manager of a learning organization the power to continuously improve training and proactively showcase value to the stakeholders outside the learning organization. MTM utilizes standard measurement tools to collect feedback from learners in general at two points in time:

Post Event Survey: collect feedback from learners immediately after class to understand if the elements of the training experience were conducive to learning, did they actually learn anything, and do they plan to use that information. Follow Up Survey: 60 days after training has completed, we then follow up to ask additional questions to understand if the learners were actually able to use what they learned, and how it impacted their job performance.

Working with premier thought leaders, these measurement tools combined provide insight into each of the levels of learning, as illustrated below.

Benefits of Using Metrics that Matter

• Streamlined evaluation process, quick and easy to implement.

• Drive superior business results by making decisions based on quantifiable metrics.

• Improve client relationships by demonstrating value and sharing key feedback.

• Engage clients more proactively based on access to real-time feedback.

• Access critical marketing information to drive additional business.

• Access to industry benchmarks that enable you to compare performance to the rest of the industry.