1

The evaluation process & evaluator’s view

Madrid, 26th January 2012

Workshop on proposal’s preparation

PTEPA & ANFACO-CECOPESCA

Dr Marina Martínez

Programme Officer

SOST-CDTI

2

Project idea

12 months Mid Term

Report Report

Final Report Project Results Submission of

the proposal

First indication from the

EC

Signed contract with the

EC

3 - 6 months Maximum 1 year

Proposal Preparation

Contract Negotiations

(several months) Project Execution (1-3 years)

KO meeting

Final meeting

Project Evaluation

(several months)

3

Basic principles of the evaluation process (1/2)

Excellence Projects selected for funding must demonstrate a high quality in the context of the topics and criteria set out in the calls.

Transparency Funding decisions are based on clearly described rules and procedures, and applicants should receive adequate feedback on the outcome of the evaluation of their proposals.

Impartiality All proposals submitted to a call are treated equally. They are evaluated impartially on their merits, irrespective of their origin or the identity of the applicants.

4

Basic principles of the evaluation process (2/2)

Confidentiality All proposals and related data, knowledge and documents communicated to the Commission are treated in confidence.

Efficiency and speed Evaluation, award and grant preparation should be as rapid as possible, commensurate with maintaining the quality of the evaluation, and respecting the legal framework. Time to grant is now about 7-9 months.

Ethical and security considerations Any proposal which contravenes fundamental ethical principles, or which fails to comply with the relevant security procedures may be excluded at any time from the process of evaluation, selection and award.

5

The evaluators…

They are experts chosen from a public data base from what the EC has ensure having:

A high level of expertise on (some of) the matters related to the topic to be evaluated Commission staff allocates proposals to individual experts taking account of their fields of core expertise and avoiding conflicts of interest.

The appropriate range of competencies for being able to evaluate.

Every topic is evaluated by a panel of at least 3 experts considering:

An appropriate balance between academic and industrial expertise and users;

A reasonable gender balance;

A reasonable distribution of geographical origins;

Regular rotation of experts

In addition, experts must have:

Abilities to appreciate the industrial and/or societal as well as innovation dimension of the proposed work.

Appropriate language skills required for the proposals to be evaluated.

6

https://cordis.europa.eu/emmfp7/index.cfm

7

Role of the experts… are at the heart of the FP7 system!

They provide independent, impartial and objective advice to the EC Significant funding decisions will be made on the basis of expert advice.

The integrity of the process is crucial Experts have to read the Code of Conduct annexed to the “appointment letter”…which is a contractual relationship with EC.

Travel and subsistence reimbursed

Experts sign confidentiality and conflict of interest declaration

They participate in the evaluations but also in “hearings” with the consortia

Names published after the evaluations

8

http://cordis.europa.eu/fp7/experts_en.html

9

Role of the Commission…

To check the eligibility of the proposals

To oversee work of experts

To moderate discussions

To organise the panel and its work

To ensure coherence and consistency

10

Full Proposal

Proposal

forms

Evaluators

Eligibility

Evaluators Evaluators Final ranking

list

Panel Submission Consensus Individual

reading

Proposals in

suggested

priority order

Rejection list

Finalisation

Criteria Criteria Criteria

COMMISSION COMMISSION

Role of experts

Evaluation process…

11

The individual reading…

Evaluate the proposal individually ,without discussing with the other evaluators

Check whether the proposal is „in scope‟

Complete an Individual Evaluation Report (IER) form giving comments and scores on all criteria

Sign and date the form

12

First Checking of the eligibility conditions of the proposal.

Minimum number of participants required.

EC requested limits.

Specific conditions on SMEs balance and % on the budget allocation.

Second Qualification of Part-B of the proposal on the basis of 3 criteria (score, from 0 to 5 in steps of 0,5).

13

14

Different evaluation procedures

One-stage proposal Full proposal submission as in any usual cooperation project.

15

Different evaluation procedures

Two-stage proposal It should respond only to S/T quality and impact of the idea, giving only few details on the budget allocation (simple table, no more). Only the proposals passing the threshold will be invited to submit a full proposal.

16

The consensus meeting…

The CM aims to agree on scores and comments Usually involves a discussion and is NOT just a simple averaging exercise

Moderated by a Commission staff-member EC helps the group reach a conclusion,

provides information if necessary, BUT DOES NOT contribute opinions.

A rapporteur is appointed, as responsible for drafting the consensus report (CR), which includes consensus marks and comments The ESR is built on it!

17

Evaluation Summary Report

18

The panel review…

EC ask some evaluators in each sub-activity to examine and compare the CRs of every proposal that passes all thresholds To ensure consistency

Ranking of proposals with identical consensus scores

Prioritise certain criteria?

Consider overall balance?

Budget?

Clear guidance for contract negotiation

Commission’s follow up…

Evaluation summary reports sent to applicants “initial information letter” & collects redress procedures

Draw up final ranking lists

Information to the Programme Committee Independent Observers‟ reports

Contract negotiation

19

Tips to be taken into account…

20

Defend the relevance and impact…

…from an EU point of view!

Based on existing (market) data and indicate:

Connect to important European policies & directives, societal challenges, other EU initiatives such as JPIs, EIPs, PPPs, etc…

Show how the project intends to solve an important problem!

Potential impact = improvement (in text, %, nominal value, according to the project) linked to the strategic objectives of the EU

Potential market value for EU industry / sector

The mechanisms and actions to bring results to the market (dissemination, take-up measures, validation by the end-users, etc.)

Check if there is alineation with national & regional strategies…

21

Prepare for the future

• Market assessment What about the current potential market…

• Think about sustainability of the project results Envision possible continuation: CIP, RSFF, Venture Capital, Business Angels…

• Prepare the use and exploitation of project results in the project:

– WP Exploitation

– WP Dissemination

• Assign specific responsibilities for these objectives

• Standarisation and certification issues, IPR…

22

Concluding remarks…

•The objective of the proposal is … To solve a problem identified by the same EC.

•Educate the evaluator with facts and figures.

•Be realistically ambitious.

•Use the right words Put graphs and prepare a very good summary .

•Documents quoted from the EC They are very pleased with their documents. Follow its indications in a detailed way.

•Remember to make easy the job of the evaluator Do not leave anything to the imagination of the evaluator… Point out, or underline, the main points of the proposal. Compliance matrix is useful for him to identificate where to find all the important issues in your proposal!

23

Thank you…

…and good luck!

24

Criteria -1: Scientific & technical excellence

25

Criteria -1: Scientific & technical excellence

26

Criteria -1: Scientific & technical excellence

27

Criteria -3: Impact

28

Criteria -3: Impact