The Power of Metrics Using Meaningful Measures to Ensure Project Success
Lydia Baude – Project Manager Systems & Procedures, Mayo Clinic David Mapes – Health Systems Analyst Systems & Procedures, Mayo Clinic PMI – Project Management Institute La Crosse Chapter February 27, 2013
Objectives Understand the case for meaningful
metrics and why it is important to project managers
Understand the challenges projects face when using poor metrics
Understand how to obtain meaningful metrics for your project and how/when to apply them
Types of project and metrics What types of metrics can you identify for
these industries? Service Construction IT Development Clinical
Challenges projects face with poor metrics Communication (How are we doing?) Prioritization (How do I know that I’m
working on what’s most important right now?)
Progress (How do I know who needs help?)
Change management (How do I know who is performing well so that I can recognize and reward them?)
What others can you think of?
Critical to Quality (CTQ) PURPOSE Translates broad, unspecified customer/ client
requirements (VOC) into measurable specifications of what is critical-to-quality and/or project success
Defines features or outputs of process or project that customer/client deems critical to satisfaction
When customer/client wants or needs are too vague to be acted on
Helps team move from high-level to detail specifications Ensures that all aspects of the need are identified
The answer to the question “what would that mean” becomes a driver for the Critical to Quality characteristics
Critical To Quality (CTQ) Tree
Need Drivers/Phase CTQs
Implement project tracking database
Effective
User friendly
Minimal cost and time to implement
Time from report request to receipt
# of reporting requests fulfilled
Time for user to generate reports Time needed by user to enter project data into database
Cost of software (develop vs. purchase) Time from project start to full user rollout
General Specific Hard to measure Easy to measure
User feedback survey results
Client survey results
Process
• Meeting site identified (sq.ft, equipment, $) • Speakers identified (#, $) • Advertising (types, #, $) • Registration (time, #) • Food (types, $, #) • Evaluation (time, type, #)
CP1000000-10
Types of Measures
(X) (X) (Y) Xs – Predictor Ys – Result
Input
• Budget • Date/Time • Duration • Objectives • Attendees
Counter-balance
Unanticipated consequences of the outcome •# Reschedules; # Participant Cancellations
within one week; Attendance vs last year; cost/participant
Output/Outcome
• Complete • On time • On budget • Profit • Participant
satisfaction • Client
satisfaction
CP1000000-11
Measure Selection Matrix
Fast
Tastes good
Hot
Correct ingredients
Repeatable
Time from order to
receipt < 2 minutes
% of orders
that are served
and match original order
# of customer
complaints about
consistency
% of orders
that are served at
110 degrees
# of customer
complaints about taste
Customer Requirements
Outcome Measures
Strong Moderate Weak
Planning for effective measurement Prioritize & monitor project critical metrics (not all
project measures will be CTQ) Identify the appropriate metrics that need to be
reported at each level of the project (Client / Sponsor, Administration, Operations, Front Line, etc.)
Clearly define what is being measured, when and how often it will be measured , and who will be responsible for measuring and reporting. (use a measurement plan)
Some measurements may need to stay in place after the project closes. Identify this need early in your project and define a plan to deliver these measures (use a control plan)
Measurement Plan
Indicator
Indicator Type
(input, process,
outcome, structure,
perception)
Operational Definition
Data Source
Person responsible
for collecting the data
When and where to
collect the data (incl. frequency)
Sampling info
(sample size, conf
level, allowable
error, sampling method)
Data entry plan
Data analysis
plan
Communication/ Dissemination Plan
# of report requests Process
For project tracking
database, the # of unique
report requests
submitted through the user portal
Project dB John Smith
Automated. Report available
every Monday at
1AM.
100% (sampling not used)
n/a - automated
report
Each Monday,
John Smith will enter the # of weekly report
requests into the run
chart
John Smith will distribute the run chart
to leadership for review at the weekly
project review meeting
Control Plan Critical
measure description
Process step
Output of this
process step
Who owns
this step activity?
What is the specification/goal for this measure?
How will the
measure be
controlled?
How will this be
measured
Who measures
this ?
What is the reaction
plan?
Who is responsible
for implementing the reaction
plan?
Location of related
documentation about this measure
Incoming error rate (missing or
incomplete info on report
requests)
Report request
Report request
submitted with all
necessary info
included
Project analyst
The goal is zero, but the process will still function as long as there are no more
than 2% of incoming requests with an
error. Callback to requestor by project
analyst
Database has hard-
stop if information is missing. Incorrect
information needs call
back to requestor.
Weekly run chart of all incoming
report requests with
errors
Project analyst
History tells us that new clients will
have a learning
curve using the new
database. If more than 2%
of requests contain
errors, the project
analyst will being a root
cause analyst and report
results to PM leadership.
Project analyst
Project Mgmt Dept
dashboard
Types of project and metrics Industry Types of Projects Possible Metrics Service • Restaurants • Hotel • Retail
• Introduce new menu • Introduce new clothing
line • Decrease the cost of
cleaning processes
• Staff/customer satisfaction • Inventory/supplies • Defects in clothing, supplies,
meals
Construction • Remodel • New
Building/house • Road
construction
• Build a new bridge • Remodel a restaurant • Install cell phone tower • Build a new hospital
• Safety: falls, fatalities, transportation, harmful substance
• Defects in supplies or work performance
• Functional performance of equipment or supplies (strength of concrete)
• Impact on the environment • Impact from the environment
Types of project and metrics Industry Types of Projects Possible Metrics IT Development • The App Store • IBM • Mayo-IT
• Apps • Software release • Hardware release • Upgrade
• Bug reports • Defects • User scenario testing results • Response time • Load time • Performance
Clinical Inpatient/Outpatient Inpatient/Outpatient Surgical Centers Rehabilitation Nursing Homes
• Process improvements • Safety Initiatives • Increasing patient
access to clinical appointments
• Improving Hand washing compliance
• Decreasing costs • Standardizing process
• Room/Patient Turn Around time • Accuracy of insurance
information • Compliance to documentation
requirements • Appointment volume • Cost per case • Utilization of operating
rooms/outpatient clinical rooms • Daily huddles/rounds • Infection rates • Sentinel events • Preventable harm • Readmission rates
Top Related