The Roles of Users in Enhancing Utility of Evaluation With Special Reference to the International...
-
Upload
buck-nelson -
Category
Documents
-
view
213 -
download
0
Transcript of The Roles of Users in Enhancing Utility of Evaluation With Special Reference to the International...
The Roles of Users in Enhancing Utility of Evaluation
With Special Reference to the International Evaluationon the Funding and Management Performance of the National Natural Science Foundation of China
Prof. Yonghe Zheng, Deputy Director, Bureau of Planning, NSFCProf. Zhaoying Chen, Deputy Director, NCSTE
Outline
Brief overview of the national evaluation effort in China
NSFC case: International Evaluation on NSFC’s Funding and Management Performance
The Roles of Users in Enhancing Utility of Evaluation
1. Brief overview of the national evaluation effort in China
Currently, to establish an evaluation system within the public administration system in China has become an urgent need in Chinese governments.
The whole – of - government performance evaluation system has not established in China.
MoF is the leading body
Ministry of Finance (MOF) is the leading body of the performance evaluation of public sectors in China.
There are already some pilot activities of performance evaluation, which carried out by line ministries or local governments in China.
2. NSFC case: International Evaluation on NSFC’s Funding and Management Performance
The evaluation is commissioned by MOF and NSFC together. The State Council attached importance to this evaluation.
The NSFC case is the most influential evaluation in public sector in China up to now and represents good practice from China.
6
Dual Objectives of the Evaluation
6
■ Accountability Provide an independent assessment of the overall performance of NSFC’s funding and management during the past 25 years
■ Lesson learning Improve the NSFC’s funding and management performance; redefine NSFC’s strategic role within the NIS of China
7
What the evaluation focuses on and what the evaluation does not attempt to do
7
The evaluation does
■Focus on NSFC’s funding and management performance over 25 years
■Focus on NSFC’s overall performance as a whole
The evaluation does NOT
Assess NSFC from the agency’s internal perspective
Assess individual projects supported by NSFC
Evaluation Approach and Principals
Modality: domestic preparation followed by international evaluation
Approach: Evidence-based Principals: Independent, Objective,
Fairness
The scope of the Evaluation will cover four dimensions:
(1) NSFC’s strategic positioning,
(2) funding performance,
(3) management performance,
(4) impact on China’s S&T system.
Evaluation Issues
Issue 1. NSFC’s strategic positioning in the national innovation system of China
Issue 2. Funding strategyIssue 3. Contributions to original innovationIssue 4. Promoting the balanced development of scientific
disciplinesIssue 5. Fostering innovative talentsIssue 6. Supporting the national demands and challenges Issue 7. Modality of funding managementIssue 8. Funding instrumentsIssue 9. Peer Review SystemIssue 10. Impact of NSFC’s funding
11
Implementation of the Evaluation
11
Phase Major Evaluation Activities
Phase12010. 2 - 2010. 6
Inception and evaluation design
Phase22010. 7 - 2010.12
Prepare evaluation evidence report The IEC first meeting in Beijing
Phase32011. 1 - 2011. 6
Complete the Evaluation Report
Phase42011 年 7 - 10 月
Use and dissemination of the evaluation results
12
Use of the Evaluation
12
■NSFC leadership understands why the organization needs evaluation
■The use of evaluation has been considered to be key issues to be addressed in the evaluation design from the very beginning and also made systematic arrangements
■During the implementation of the evaluation, some findings have already been put into application.
■There will be a series of follow-up activities to ensure that evaluation is used appropriately
3. User’s Role in Enhancing Utility of Evaluation
NSFC: Inception and evaluation design Ensure that the evaluation is targeted to
the demands of users, respond to current concerns and debates;
Discuss with NCSTE team about the conception framework of the evaluation; Identify the 10 key issues and evaluation questions;
Promote IEC and NCSTE team to understand NSFC’s funding and management system;
NSFC: Evaluation Implementation Provide coordination and support
requested by IEC and NCSTE team; Prepare a volume of case-based evidence
for the evaluation; Assure that NCSTE can access to all
relevant information and select the target group for interview or focus group meeting;
Judge the credibility of the message in the evaluation report
3. User’s Role in Enhancing Utility of Evaluation
NSFC: Response and Dissemination Take management response to the
evaluation report at the draft stage Distribute the evaluation report to line
ministries, the National People's Congress, the Chinese People's Political Consultative Conference
Arranging dissemination events of the evaluation.
3. User’s Role in Enhancing Utility of Evaluation
NSFC: Follow-up NSFC leading level and director level:
One week seminar to discuss the main results of the evaluation; identify possible action in NSFC strategy and funding management;
NSFC staff: improving conditions for evaluation-based learning in NSFC;
Release the report to the public
3. User’s Role in Enhancing Utility of Evaluation
Primary users should really understand why they need evaluation;
The use and dissemination of evaluation should be addressed from the beginning and also built into the work plan of the evaluation.
Lesson Learned
Thanks!