Using Metrics to Drive Research Administration Performance
38
Using Metrics to Drive Research Administration Performance NCURA FRA Meeting March, 2013 Marcia L. Smith, Assistant Vice Chancellor, Research Administration, UCLA Elizabeth H. Adams, Executive Director, Office for Sponsored Research, Northwestern University, Evanston Campus Susan Lin, Assistant Controller, Extramural Funds, UCSF Tracey Robertson, Director, Higher Education Consulting, Huron Consulting Group
description
Using Metrics to Drive Research Administration Performance. NCURA FRA Meeting March, 2013. Marcia L. Smith, Assistant Vice Chancellor, Research Administration, UCLA Elizabeth H. Adams, Executive Director, Office for Sponsored Research, Northwestern University , Evanston Campus - PowerPoint PPT Presentation
Transcript of Using Metrics to Drive Research Administration Performance
Using Metrics to Drive Research Administration Performance
NCURA FRA MeetingMarch, 2013
Marcia L. Smith, Assistant Vice Chancellor, Research Administration, UCLAElizabeth H. Adams, Executive Director, Office for Sponsored Research, Northwestern University, Evanston CampusSusan Lin, Assistant Controller, Extramural Funds, UCSFTracey Robertson, Director, Higher Education Consulting, Huron Consulting Group
Agenda
Introduction
Benefits of Metrics
Developing Metrics
Determine Metrics to Track
Collecting Metrics
Discussion (All)
2
Introductions
3
Benefit of Metrics
4
Benefit of Metrics
5
Metrics
Change Behavior
Drive Performanc
eSupport
Investments
Benefits of Metrics
6
Drive Performance1) Motivate teams to achieve desired outcomes
• Increasing transparency can trigger positive culture changes and improve outcomes.
• Using metrics to monitor business processes improves accountability so high performers can be recognized and bottlenecks addressed.
2) Define business processes and responsibilities• Implementing metrics requires an organization to identify its
desired outputs resulting in defined business processes.• Use of metrics helps identify operational bottlenecks which can
be a result of personnel having varied understandings of processes, roles and responsibilities.
3) Monitor the impact of new processes• As new processes are implemented, metrics provide confirmation
that the change is working.• Easily identify bottlenecks• Help motivate team
Benefits of Metrics
7
Change Behavior1) Manage stakeholder expectations
• Metrics enable clear communication of process goals and current status to stakeholders.
• Concrete information enables stakeholders to determine whether their needs are being met.
• If an institution sets reasonable and clear goals which are communicated then customer perception of performance can improve simply through a better understanding.
2) Evaluate staff performance• Metrics allow leadership to objectively track staff contributions
both individually and collectively against operational goals.• With metrics, personnel know their assessments are objective,
why they are receiving their assessments, and how to improve them.
Benefits of Metrics
8
Support Investments in Research Administration Infrastructure:1) Improve decision making and prioritization
• Metrics provide leadership with insights as to where attention and resources are needed.
• Concrete information improves decision-making and allows managers to better understand and identify opportunities to improve compliance, financial management, and operational performance.
Developing Metrics
9
Developing Metrics
10
Where to Begin?
Where are the most significant business pressures in your organization?• Strategic priorities• Compliance risk • Customer frustration
What data do you have access to?• What are the relevant systems? Start with enterprise systems, BI
tools• Once you get access, do you trust the data?• Do the datasets accurately describe processes? • Likely need for strengthening data definitions, data cleanup,
process changes and better reporting tools
Developing Metrics
11
Where to Begin?
What processes do you own, or can you influence?• Many research administration processes touch different teams and
offices• If you own the processes in your office, you can move much more
quickly• System reconfiguration and/or process modification to collect the
data you want (e.g., capture handoffs)• Undeniable connection between data integrity and standard
operating proceduresBaseline data can be hard to develop, and accept
• Some staff may embrace the challenge, others may not• Though change can be disruptive/controversial, some staff will feel
relieved that they know exactly what is being expected of them• Baseline data offers an invaluable opportunity to demonstrate
improvement, which can be very positive and morale building• If you embrace transparency, you can more easily expect this of
others
Developing Metrics
12
Where to Begin?
Anecdotal data complements empirical data• While indispensable to the story you want to tell about your
organization, empirical data doesn’t tell the whole story• Don’t forget to collect the personal stories, internally and
externally• Beyond providing important contextualizing information, collecting
different perspectives is valuable for building trust, relationships
Determine Metrics to Track?
13
Identify Greatest Opportunity
14
Operational Performance
• Turnaround times
• Workload Distribution
• Production vs. Goal
Compliance Management
• Cost transfers• Effort
Reporting• Expiring
Protocols• Risk
Assessment
Financial/Cost Management
• Cash Receipts• Timeliness of
financial reporting
• Bottlenecks• Outstanding
collections• Write-offs
Customer Perception
• Turnaround Times
• Workload Distribution
Taking a focused approach will allow you to more quickly implement successful, lasting, and measureable improvements
UCLA Case Study
15
Pre-Award Metrics
Post-Award Metrics
UCLA Case Study
16
6 9 7 7 11 9 9 16 16 12 11 6 18 1264 51
84 74107 132
100131
182214
147120
66 59
113
52
126 134
169 143
102
101
125
144
119131
11291
87
54
98 79
91 86
91
115
105
92
8890
68
64
126
85
139
120
150124
116
143
167
178
193204
114
104
2
1
2
3
58
3
10
10
15
13 10
5
0
7
8
12
11
711
9
15
14
25
4030
9
1
0
100
200
300
400
500
600
700
800
Nov 09Data
Dec 09 Data
Jan 10 Data
Feb 10 Data
Mar 10 Data
Apr 10 Data
May 10 Data
Jun 10 Data
Jul 10 Data
Aug 10 Data
Sep 10 Data
Oct 10 Data
Nov 10 Data
Dec 10 Data
OCGA Award Acceptance Details
Supplement
Renewal
New
No Cost Extension
Mod / Amendment
Continuation
Admin Change
Pre-Award Metrics
17
Area MetricProposal Submission Average days complete submission
received before deadline, missing/incomplete proposals, detailed metrics
Proposals/Awards Submission
Number and $$ of proposals/awards, detailed metrics by categories
Award Set-Up Timeline for central office(s) to process set-up*
Award Set-up Timeline for receipt of award document to financial account activation
Award/Outgoing Subcontract Execution
Timeline with central office
Award/Outgoing Subcontract Execution
Timeline with internal Departments, Principal Investigator, Sponsor, Subrecipient
Advance Accounts Number and how long they have been open
18
Proposal Submission
0-1 Day in Advance of Deadline
2-3 Days in Advance of Deadline
4-5 Days in Advance of Deadline
5+ Days in Advance of Deadline
261
65.2543.5
56.55
Proposal Submission to OCGANumber of Proposals Submitted on a Monthly Basis
19
Award Intake Process (Pilot) Turnaround time for Expedited Awards has improved by over 80%
during the award setup pilot
Award Set-Up
20
Award Intake Process (Current) Full implementation January 2012 Award setup has slowed for expedited awards, but is still 65% faster
than previous processing timeline
Award Set-Up
Award Setup (Current)
21
New process has identified hold-ups • Shaping policy and procedure decisions• Awards processed 6 days faster when all internal documents are
present
Post-Award Metrics
22
Area Metric
Letter of Credit • Number and $ of draws• $ unbilled• $ in LOC Clearing Account(s)
Invoicing • Monthly unbilled number and $• Monthly billed number and $
Accounts Receivable • Number and $ in aging buckets 30/60/90/120+ days
Overdrafts • Number and $ in aging buckets for overspent accounts
Closeout • Awards open 120+ days past end dateCost Transfers • Number submitted/approved less than 90
or 90+days
Post-Award Metrics
23
Area Metric
Financial Reporting • Monthly number submitted• Monthly percentage submitted on time• Monthly number past due (i.e. backlog)
Cash Application • Number and $ of payments sitting in holding accounts
Bad Debt • Number and $ of write-offs
Effort Reporting • Number and % certified on time• Number and % outstanding• Number and % certified by PI or individual• Number of recertifications
On-Time Submission
24
On-time submission increased by 35% for Invoices On-time submission increased by 48% for Reports, from a
low of 14% at the start of FY10
Backlog – Invoices and Reports
25
Backlogs have decreased by 64% since the start of FY10
26
Extramural Funds 2012 Non-LOC Cost Reimbursable Invoicing Status
System Temporary Invoice EMF Approved Invoice Count Amount Count Count% Amount Amount%
January 571 $ 11,779,837 522 91% $ 11,011,031 93%
February 542 $ 11,685,398 516 95% $ 11,329,685 97%
March 571 $ 11,360,929 541 95% $ 10,987,593 97%
April 591 $ 12,582,571 554 94% $ 12,114,788 96%
May 588 $ 11,947,661 530 90% $ 11,523,073 96%
June 567 $ 11,993,026 549 97% $ 12,129,138 101%
July 583 $ 14,935,457 502 86% $ 14,380,680 96%
August 524 $ 7,606,187 430 82% $ 7,227,730 95%
September 521 $ 9,824,166 413 79% $ 8,213,839 84%
October 510 $ 10,603,463 409 80% $ 9,464,135 89%
November 455 $ 10,375,438 375 82% $ 9,850,844 95%
December 448 $ 11,688,734 365 81% $ 10,632,106 91%
27
Extramural Funds Fixed Price Billing Status
Status Final Invoice Fixed Price Intercampus Prepayment TOTAL
Completed 11 375 2 1 389
Not Started 0 0 0 0 0
In Progress 0 28 0 0 28
Total 11 403 2 1 417
28
Extramural Funds Unreimbursed Cost for Type of awards
Jan-2012
Feb-2012
Mar-2012
Apr-2012
May-2012
Jun PL 2012
Jul-2012
Aug-2012
Sep-2012
Oct-2012
Nov-2012
Dec-2012
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
US
Dolla
r in
Mill
ions
(Rev
erse
d si
gn)
29
Extramural Funds Cash Receipts and Expense
Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun$0
$200
$400
$600
$800
$1,000
$1,200Total Receipts
US D
olla
r in
Mill
ions
Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun$0
$200
$400
$600
$800
$1,000
Sponsored Project Cumulative Expenses
US D
olla
r in
Mill
ions
30
Extramural FundsAccounts Receivable Aging Balances Over 120 Days
Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun$0
$1
$2
$3
$4
$5
$6 U
S D
olla
rs (M
illio
ns)
31
Extramural FundsTop 10 Sponsors with AR Balances Older Than 120 Days
Sponsor Name
≥ 365 Days ≥ 121 Days Total
Count Amount Count Amount Count Amount
XXXX Sponsor 0 $0 3 $785,156 3 $785,156XXXX Sponsor 0 $0 1 $592,000 1 $592,000
XXXX Sponsor 0 $0 1 $350,000 1 $350,000XXXX Sponsor 0 $0 1 $343,279 1 $343,279
XXXX Sponsor 0 $0 1 $127,135 1 $127,135XXXX Sponsor 0 $0 5 $69,316 5 $69,316
XXXX Sponsor 0 $0 1 $43,286 1 $43,286
XXXX Sponsor 0 $0 3 $39,633 3 $39,633
Notes (sponsors over $70K):
XXXX Sponsor: $586,495 paid on January 15, 2013; $1,218 approved for payment; $197,443 in approval process at sponsor. The delay is due a transfer of program between departments at Sponsor which required a new contract .
XXXX Sponsor : Payment delayed due to change in sponsor’s entity name, which requires new letters of agreement between the sponsor and UCSF. The new agreement has executed January 2013.
XXXX sponsor: Payment received on January 15, 2013.
XXXX Sponsor: Foundation intended to make a gift rather than entering into a research contract. We are in the process of changing the nature of agreement from research to gift.
XXXX Sponsor : Payment delayed due to lack of PO number and difficulty identifying and resolving issues through third-party Help Desk. The Invoice now has been posted in sponsor’s “Aspen” system for payment.
3232
Extramural Funds XXXX Sponsor’s Fund Deficit Status
0
2,000
4,000
6,000
8,000
10,000
12,000
US D
olla
r in
Thou
sand
s
Deficit Reasons
33
Award CloseoutCompletion status compared to goal
(Operational Goal – close within 150 days after end date of budget period)
*June is a peak month when award budget period ends. The reason of the delay for closing out June expired awards is because the Closeout Team is staffed to meet the workload of 100 to 200 expired awards per month. We will continue to explore options how to flex our resources to reduce the delays.
0
100
200
300
400
500
600
700
800
34
FFR Statistic OverviewCompletion Progress
0
50
100
150
200
250
300
350
400
450
35
Effort ReportingCertification Timeliness & Completion
09/10 Non-Academic Win-
ter
09/10 Academic and Non-Aca-demic Spring
10/11 Non-Academic Summer
10/11 Academic and Non-Aca-
demic Fall
January-June, 2011
July-December, 2011
January-June, 2012
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
82.00%77.90%
81.00%
72.00%
97.74%94.15% 97.88%
100.00% 100.00% 100.00% 100.00% 100.00% 99.98% 99.48%
Timeliness
Completion
Collecting Metrics
36
Collecting Metrics
37
Identify Opportunities
Determine Metrics to track
Identify systems to deliver outputs
Analyze output to ensure integrityThings to consider:
• Audience for metrics Effectiveness of metrics is greatly affected by the selection of
recipients who will be reviewing the metrics. Wrong metrics or wrong audience diminishes the value of the
metrics. Data integrity
Is the data that you are using to populate the metrics accurate? Is the correct logic being used? Establish standardized processes for entering data
Interpretation of data Are assumptions being made? Know what each data field means; develop a data dictionary
Discussion
38
