Evaluation System of Evaluation System of Government –Funded Research Government –Funded Research InstitutesInstitutesEvaluation System of Evaluation System of Government –Funded Research Government –Funded Research InstitutesInstitutes

2011. 11. 3

Woo Chul ChaiResearch Fellow

R&D Program Evaluation Division

ConclusionConclusionⅣ.Ⅳ.

Contents

Outline Outline Ⅰ.Ⅰ.

Utilizations of Evaluation ResultsUtilizations of Evaluation ResultsⅢ.Ⅲ.

Directions and Evaluation SystemDirections and Evaluation SystemⅡ.Ⅱ.

ⅠⅠ. Outline. Outline

11

1. Purpose and Object

To Set up Roles of GRIs and Promote R&D Productivity through

Reviewing Research- and Management-Performances

36 Government-funded Research Institutes under the MEST, DAPA, etc.

Management Performances

Research Performances

Comprehensive Performance

Purpose

Object

Section

22

1991• Performed by the Prime Minister’s Office• 22 GRIs evaluated together• According to the Evaluation Results, Merge and Abolition of some GRIs made

1998• Benchmark of Foreign GRIs

1999-2005 • 4 Research Councils were Established• Each Research Council performed Evaluation of Affiliated Institutes

The First Evaluation

Management Diagnosis by Foreign Consultant

Research Council System

2006-

• Achievement of Performance Goals in R&D and Management Evaluated

2. History of Evaluation System

Performance-based Evaluation System

3. Structure of GRIs’ Evaluation

Meta-Evaluation

Self-Evaluation

33

Ministry of Education & Science Technology

Ministry of Education & Science Technology

Ministry of Knowledge Economy

Ministry of Knowledge Economy

National S&T Council

MEST(KISTEP)

MEST(KISTEP) KRCFKRCF ISTKISTK DAPADAPA

Manag

-ment

KAIST, etc

Research project

Manag-ement

KIST, etc

Research project

Manag-ement

KRISS, etc

Research project

ADDADD

Ⅱ. Directions and Evaluation System

1. Directions (’11)

To Alleviate Burden of Researchers, Evaluation Interval in Management Section is

Expanded

※ 1 Year Every 3 years

Alleviation of Evaluation Burden

Reinforcement of Utilization of Evaluation Results

For Promoting Effective Performance of the Mission and Function of GRIs,

Consulting is Provided to the GRIs concerning the Appropriate Direction of R&D,

etc.

Provision of Consulting

55

Depending on Evaluation Results, Budget is Allocated Differently to Research

Programs (10% Decrease of Budget to Unsatisfactory Programs)

66

2. Evaluation Sections (Self- Eval.)

SectionSection ObjectObject IntervalInterval ItemsItems

Research Performance

Projects funded by Government

3 years - Achievement of Performance Objectives

Management Performance

Management Activities

1 year- Improvements of Management Activities and Achievement of Management Goals

Comprehensive Performance

Comprehensive R&D and Management

Performance3 years

- Qualitative Evaluation concerning the Development Strategy of Institutes in R&D and Management

88

3. Criteria in the Evaluation of Management Performance (Self- Eval.)

Evaluation Items Indicators

Establishment of Vision and Strategy of Institute

• Excellence of President’s Leadership and Level of Achievement of Management Goals

• Efforts of Promoting S&T Culture

Achievement of Monitoring• Improvements and Utilization of Internal- and External -Evaluation Results

• Efforts for Customer’s Satisfaction

Development ofHuman Resources

• Employment of Excellent Manpower and Establishment of Global Research Networks

• Efficiency of Institute Management and Utilization of Manpower

Research Management System

• Establishment of Cooperative System among Academia, Institutes and Industry

• Utilization of Performance Management System

• Utilization of Knowledge Management System

Budget and Finance Management

• Clarity of Budget Management and Execution

4. Criteria of Meta-Evaluation

To Review the Appropriateness of Self-Evaluation System

99

ItemsItems IndicatorsIndicators

1. Appropriateness of

Evaluation Plan (30)

• Appropriateness of Evaluation Objectives

• Appropriateness of Evaluation Planning

• Appropriateness of Evaluation System (Indicator,

Weight, etc)

2. Appropriateness of

Evaluation Process (35)

• Fairness of Evaluation Committee’s Composition

and Operation

• Appropriateness of Evaluation Indicators and Methods

• Fairness of Evaluation Execution

3. Effectiveness of

Evaluation Results (35)

• Composition of Evaluation Report

• Objectiveness of Evaluation Results

• Appropriate Utilization of Evaluation Results

5. 5. Grade of Meta-Evaluation

Grade is Divided into 5 Level

Final Grade : “Appropriateness (above 70)” or

“Inappropriateness (under 71)”

- Adding up the Score of 3 Evaluation Items

Grade Score Remarks

Excellent 100 - 91

“Appropriate”Good 90 – 81

Moderate 80 – 71

Bad 70 – 61“Inappropriate”

Worst 60 -

1010

Ⅲ. Utilization of Eval. Results

1212

1. Utilization of Evaluation Results

Evaluation Results of Management Performance

- Increase or Decrease of Operating Cost of GRIs (±10%)

- Increase or Decrease of President’s Annual Salary of GRIs

Evaluation Results of R&D Performance

- Decrease of Budget in Unsatisfactory Projects ( - 10% )

- Increase of Budget in Excellent Projects ( + 10%)

Evaluation Results of Comprehensive Performance

- Guideline for Planning of Research Projects and R&D Directions

Ⅳ. Conclusion

1. Improvements

  To Introduce the Absolute Evaluation System that Reflects the

Characteristics of R&D Institutes

- GRIs are Classified into 3-4 Groups by the Research Area

To Alleviate Burdens by Simplifying the Indicators in the Evaluation of

Management Performance

- Deleting the Indicators related with R&D such as ‘No. of Patents’, ‘No.

of Technology Transfer’, etc.

To Reinforce the Specialization of Evaluation

- Expanding the Tenure of Evaluation Committee Members

1414

Thank You !Thank You !