1 © 2000 Technology Commercialization Group LLC. All rights reserved TECHNOLOGY COMMERCIALIZATION...

1© 2000 Technology Commercialization Group LLC. All rights reserved

TECHNOLOGYCOMMERCIALIZATION

GROUP, LLCPAUL G. WAUGAMAN

LOUIS G. TORNATZKY

Higher Education PracticeWILLLIAM S. KIRBY

APPLYING BENCHMARKING TO SPONSORED PROGRAMS ADMINISTRATION

SRA - NCURASouthern Section/Southeastern Region Meeting

April 16-17, 2000

TCG


Points to Cover:

Benchmarking in research administration» Overview

The Higher Education Benchmarking Consortium» Overview

The KPMG - NACUBO - SRA Sponsored Programs Benchmarking Effort» Organization» Initial results» Next steps


Benchmarking

Systematic comparison of elements of the performance of an organization against that of other organizations, with the aim of mutual improvement.

McNair and Leibfried: “Benchmarking.” 1992


The Utility of Benchmarking

Provides a basis for comparison and self-analysis.

Accelerates the sharing of useful and novel approaches.

Energizes everybody to do better.


Purposes and Uses of Measurement in Research Administration

Measure against “industry standards”

Measure against goals» Performance drives improvement interventions.

Identify problems» Diagnose and forecast existing problems.

Build the “business case” for change or new resources.


The Problem:In Research administration .... There are no generally accepted criteria

for success. We rely on anecdotal data and informal

case study rather than systematic data collection and analysis.

Benchmarking and measurement are not part of the improvement culture

Competitive comparisons may be viewed as threats rather than improvement drivers.


The Problem (cont):

Breadth and diversity of the domain make data collection and measurement difficult.

Successful outcomes are difficult to demonstrate objectively.

There is little data available.

Who is going to do it?


The SRA Benchmarking Initiative

Benchmarking and Best Practices Task Force established in 1998.

Task Force recommends working with NACUBO and KPMG to facilitate performance data collection.

Collaboration with NACUBO and KPMG initiated in September 1998.

Data collection instrument refined with SRA Task Force input in Spring, 1999.


The KPMG Higher Education Benchmarking Consortium

The purposes of the consortium are:» To serve as a comprehensive source of

institutional comparative performance, benchmarking, and “best practice” information for the higher education community; and

» To facilitate the use of comparative performance data to meet strategic, operational improvement, and change management needs.



Operating Principles

» Participant-driven» “E-nabled”» Focused» Leading to "best practices” » Leveraged, where possible



Major Segments» Financial Health and Operations» Student Affairs» Physical Infrastructure» Information Technology» Human Resources» Research:

Sponsored Programs Technology Transfer


KPMG - NACUBO - SRA Sponsored Programs Survey

Purposes To provide a common set of sponsored

programs performance indicators that will allow meaningful comparisons with other organizations and with peer groups of institutions.

To facilitate the development of a benchmarking data base.


Sponsored Programs Survey

SRA-NACUBO-KPMG Roles

NACUBO and KPMG are financing and organizing data collection and analysis, and are promoting the program with their constituencies and clients.

SRA is participating in survey design, is promoting the program with their members, and is participating in dissemination of findings.



Features of the Program

Low cost to participating institutions. Individualized reports for

participants. Expert analysis. All institution-specific data will be

“blinded.” Only general reports will be available

to non-participants.



Critical Success Factors for Research Administration

Sustaining or Enhancing Sponsored Research Activity and Funding (competitiveness)

Containing the Costs and Increasing the Efficiency of Sponsored Research Administration

Improving Service to Faculty (in order to achieve Objective 1)

Maintaining and Improving Institutional and Sponsor accountability


Sponsored Programs Survey Indicators:

Sustaining or Enhancing Sponsored Research Activity and Funding

Number of proposals submitted per faculty FTE. Percentage of faculty working as principal

investigators. Sponsored project dollars received per

faculty FTE. Sponsored projects funding growth rate. Number of new awards as a percent of new

proposals submitted.



Indicators:Cost and Efficiency

Number of proposals per sponsored projects admin. FTE.

Sponsored projects admin. cost per proposal. Number of awards per Sponsored projects admin

FTE. Sponsored projects admin costs as percent of

dollars received. Number of awards per research accounting FTE. Research accounting costs per award Dollars received per research accounting FTE. Research accounting cost as percent of dollars

received.



Service to Faculty

Number of funded PI’s per sponsored projects FTE.

Number of active PI’s per sponsored projects FTE.

Number of funded PI’s per research accounting FTE.



Institutional and Sponsor Accountability

Percent of Sponsored Projects Accounts Receivable over 120 days.



Data Needed to Develop Indicators

Fy 1997-98 Data: Number of faculty FTE. Number of central administrative FTE (sponsored

programs, research accounting). Proposal data (number, dollars, sponsor: federal,

industry, other). Award data (number, dollars, sponsor: federal,

industry, other). Sponsored program expenditure data Administrative costs allocated to research (central and

departmental costs - from indirect cost proposal).



Benefits for Participants

Access to useful comparable data for program evaluation and benchmarking.

Low cost. Ease of preparation. Uses readily available data

assembled for internal reporting or other purposes.



Year One Questionnaire Design (Dec 98-Jun 99)

» Survey design was peer-based» Intentions were to use available data» Initial survey requested FY 1998 data

Data Collection (Jul-Oct 99)» A high response rate was a major objective» Timing did not accommodate academic calendar» Correct POC identification was a challenge» Follow up efforts: phone calls, letters, e-mails



Year One Data Collection Outcomes:

» 62 institutions participated

(40% of all US college/university research expenditures)

» Definition issues arose Examples: Who are “research faculty;” new vs. renewal

awards Caused confounding metrics

Data Processing» 34 institutions chose peers» 3 sub-sets of institutions



Year One Reporting:

» Institution-specific reports: Customized, Institution-specific. Provided ranking

information on 16 metrics, and 4 sub-groups for each metric.

» Data feedback workshops Gave participants opportunities to discuss data, and

next steps

» Revised Reports will be prepared after participants have an opportunity to revise submissions

» Public Report in preparation



Year OnePercentage of faculty working as principal investigators. Number Median

Value

Entire Sample 62 39.1%

Public 51 39.0%

Independent 11 40.0%

Research I 33 40.9%

Research II 16 40.4%

Other CarnegieClasses

13 20.8%

Land-Grant 18 44.1%

Medical Schools 27 40.4%



Year OneSponsored Project dollars per Faculty FTE

Number Median Value

Entire Sample 62 63,453

Public 51 61,289

Independent 11 86,118

Research I 33 80,923

Research II 16 53,944


13 23,320

Land-Grant 18 68,915

Medical Schools 27 80,923



Year One

Sponsored Projects Funding Growth - Fy 93 - 98Number Median

Value


Public 51 6.3%

Independent 11 6.6%

Research I 33 6.3%

Research II 16 3.7%


13 8.3%

Land-Grant 18 5.5%




Year One

Number of proposals submitted per central sponsored projects administrative FTE

Number MedianValue

Entire Sample 61 70.7

Public 50 73.7

Independent 11 54.9

Research I 32 83.9

Research II 16 58.8


13 45.1

Land-Grant 17 90.8

Medical Schools 27 86.3



Year One

Central sponsored projects administrative cost as a percent of sponsored project dollars

Number MedianValue


Public 51 0.97%


Research I 33 0.80%



13 2.11%

Land-Grant 18 0.81%




Year One

Central post-award financial administrative cost per active project

Number MedianValue

Entire Sample 60 $389

Public 49 $369

Independent 11 $477

Research I 32 $310

Research II 15 $470


13 $628

Land-Grant 18 $289

Medical Schools 25 $375



Year One

Central post-award financial administrative cost as a percent of sponsored project dollars

Number MedianValue


Public 50 0.65%


Research I 32 0.49%



13 0.95%

Land-Grant 18 0.56%




Year One

Number of active Principal investigators per central sponsored projects admin. FTE

Number MedianValue

Entire Sample 60 33.3

Public 49 34.2

Independent 11 23.6

Research I 31 37.3

Research II 16 27.7


13 23.5

Land-Grant 18 44.5

MedicalSchools

26 38.1



Next Steps

Complete Fy 1998 round» Reopen data base for revisions and

additional enrollments (April-May, 2000)

» Complete a brief public report (May, 2000)

» Complete revised individual reports (June, 2000)

» Brief Best Practices study (June - August, 2000)



Next Steps

Initiate Fy 2000 round» Target date to begin data collection: September-

November, 2000» Activate Web Site for data collection on-line

(September, 2000) and report preparation by participants (January, 2000)

» Reports: January, 2001

Web site (for informational purposes): www.us.kpmg.com/highered/benchmark/


Contact us ...

TECHNOLOGY COMMERCIALIZATION GROUP, LLC2237 OXFORD HILLS DRIVE

RALEIGH, NORTH CAROLINA 27608-1672 USA

TEL: 919-833-2569

FAX: 919-833-3277

E-MAIL: [email protected]

PAUL G. WAUGAMAN, PRINCIPAL

WWW.T-C-GROUP.COM

TCG

1 © 2000 Technology Commercialization Group LLC. All rights reserved TECHNOLOGY COMMERCIALIZATION...

Documents

Transcript of 1 © 2000 Technology Commercialization Group LLC. All rights reserved TECHNOLOGY COMMERCIALIZATION...