Proof of Concept IGLO Veronica Beneitez Pinero March 2015.

Proof of ConceptIGLO

Veronica Beneitez Pinero

March 2015

Established by the European Commission

│ 2

The Scientific Management Department in theERC Executive Agency


│ 3

Functions of the Scientific Managament Department

• Evaluation of the proposals of ERC calls Scientific evaluation Ethics clearance of proposals proposed for funding

• Project follow-up Scientific project follow-up Ethics follow-up

• Ex-post qualitative assessment of the research funded

• IT and BP support in the evaluation process• Expert management recruitment and support• Redress• Implementation of international agreements


│ 4

B

Scientific Management Department

Process Management and Review

Call and Project Follow-up

Coordination

Life Sciences

Structure of the Scientific Management Department

Physical Sciences and Engineering

Social Sciences

and Humanities

Staff of the Department: 165

B1 B2 B3 B4 B5

PoC


│ 5

• Independent experts, evaluate proposals submitted in response to a given call.

• The experts are responsible for carrying out the evaluation of the proposals themselves.

• They are not allowed to delegate the work to another person!

• Significant funding decisions will be made on the basis of their assessment.

Role of independent experts


│ 6

Objective: The ERC Proof of Concept Grants aim to maximise the value of the excellent research that the ERC funds, by funding further work to verify the innovation potential of ideas arising from ERC funded projects.

What kind of activities that can be financed: • establishing viability, technical issues and overall direction• market research • clarifying IPR strategy • investigating business opportunities• Initial expenses for start-up

PoC : Principles Objective and activities


│ 7

The financial contribution will be up to a maximum of EUR 150 000 for a period of 18 months.

"The ERC expects that normally, proof of concept projects should be completed within 12 months. However, to allow for those projects that require more preparation time, projects will be signed for 18 months. Given this initial flexibility, extensions of the duration of proof of concept projects may be granted only exceptionally."

The ERC contribution will take the form of the reimbursement of up to 100% of the total eligible and of flat-rate financing of indirect costs of a maximum of 25% of the total eligible direct costs

PoC : Principles Financial contribution and duration


│ 8

PoC : Evaluation Flow

Reception of proposals -

SEP submission

Eligibility check

Remote Evaluation (SEP)

Evaluation results

Panel meeting

Feedback to

applicants

Preliminary evaluation

results


│ 9

PoC : Submission of proposals

Reception of proposals -

SEP submission

ERCEA published the call on the Participant Portal

- Work Programme- Guide for Applicants- Templates- Frequently Asked Questions

The work programme is a legally binding document. It is a dynamic document , published annually, set out the evaluation criteria.Main Changes 2015 WP:

Increase of PoC budget : €20 millionResubmission restriction: 1 PoC application per year.Simplification of evaluation criteria


│ 10

PoC : Submission of proposals

The applicant must submit the required documents before the call deadline

The documents are :

Administrative dataPart B ( 7 pages + budget table Max)Host Support Letter

Additional documents: Annexes ( Not relevant for the evaluation)


│ 11

PoC : Structure of the submitted proposal (1/2)

Section 1: The idea - Innovation potential (max. 2 pages) a. Succinct description of the idea to be taken to proof of conceptb. Demonstration of Innovation Potential

Section 2 – Expected Impact (max. 2 pages): a. Economic and/or societal benefitsb. Commercialisation process and/or any other exploitation process c. Proposed plans for :- Competitive analysis - Testing, technical reports (where applicable)- IPR position and strategy (where applicable)- Industry/sector contacts (where applicable)


│ 12

Section 3: The proof of concept plan (max 2 pages) a. Plan of the activitiesb. Project-management planc. Description of the team Section 4: The budget (max 1 page + costing table)a. Resources (incl. project costs)

b. Justification (description of the budget)

Only the material that is presented within this limit will be evaluated

PoC : Structure of the submitted proposal (2/2)


│ 13

PoC : Elegibility Check 1/2

Eligibility check

Done in house by ERCEA Scientific Officers

Eligible Project: Proposal complete and submitted on time The content of the proposal must relate to the

objectives and to the grant type set out in the call Demonstrate the relation between the idea to be

taken to PoC and the ERC research grant.

Eligible Principal Investigator: The PI has to be in an ERC frontier research that

is either ongoing or has ended less than 12 months before the opening date of the call.

The PI is subject to resubmission restrictions ( one eligible application per call)

Eligible Host Institution: In a Member State or Associated Country


│ 14

PoC : Elegibility Check 2/2

The check is done in parallel to the evaluation. So a proposal can be declare ineligible at any stage.

If ineligible, you will be informed and the proposal will dissapear from your list of assigned proposal.

Applicants will be informed as soon as the decistion is taken.

Applicants can redress on the elegibility decision.


│ 15

PoC : Remote evaluation 1/4

Remote Evaluation (SEP)

5 reviews/proposal No discussions between reviewers Report any Conflict of Interest (CoI) Remote, using SEP

• PASS/FAIL mark on each criterion

• Succinct explanatory comment for each mark

• Sign and submit your reviews


│ 16


We have aligned the template of Part B with the evaluation criteria to make the review process a bit more simple.

Evaluation Criteria 1 Excellence (Innovation potential)

Section 1: The idea - Innovation potential (max. 2 pages)

Does the proposed proof of concept activity greatly help move the output of researchtowards the initial steps of a process leading to a commercial or social innovation?

Demonstration of Innovation Potential


│ 17


2. Impact Section 2 – Expected Impact:

2.1 Is the project to be taken to proof of concept expected to generate economic and/orsocietal benefits which are appropriately identified in the proposal?2.2 Does the proposal indicate a suitable process that is designed to result in a concreteapplication, including outlining a process of commercialisation or a process of generatingsocial benefits?

a. Economic and/or societal benefitsb. Commercialisation process and/or any other exploitation process c. Proposed plans for :

The proposal should include:- plans for the analysis of whether the project’s outcomes are innovative or distinctivecompared to existing solutions;- plans for seeking confirmation of the actual effectiveness of the project’s results;- plans to clarify the IPR position and strategy33;- plans for setting up contacts with industry partners, societal organisations or potential ‘endusers’ of the projects’ results.

- Competitive analysis - Testing, technical reports (where applicable)- IPR position and strategy (where applicable)- Industry/sector contacts (where applicable)


│ 18


3. Quality and efficiency of the implementation (Quality of the proof of concept plan)

Section 3: The proof of concept plan (max 2 pages)Section 4: The budget (max 1 page + costing table)

Does the proposal provide a reasonable and acceptable plan of activities against clearlyidentified objectives and towards establishing the feasibility of the project?This should include:- a sound project-management plan, including appropriate risk and contingency planning;- demonstration that the activities will be conducted by persons well qualified for thepurpose;-demonstration that the budget requested is necessary for the implementation of theproject and properly justified.

a. Plan of the activitiesb. Project-management planc. Description of the team


│ 19

PoC : Preliminary Results ranking 1/3

Preliminary evaluation

results

Proposals which fail a criterion will not be ranked

Proposal’s score = S PASS marks

Same number of reviews for each proposal

Every mark has an impact

Proposals will be funded up to depletion of budget6,6 M€ for each deadline = ~44 proposals


│ 20


If there is not enough budget to fund all the proposals which pass all three evaluation criteria, those proposals which pass all three evaluation criteria will be sorted by the number of pass marks awarded by peer reviewers to criterion 1 (Excellence- Innovation potential), then by the number of pass marks awarded to criterion 2 (Impact), then by the number of pass marks awarded to criterion 3 (Quality and efficiency of the implementation). Proposals will be funded in order of the ranking resulting from this 3-level sorting exercise until depletion of the available budget per evaluation round.


│ 21


If there is a group of equally ranked fundable proposals that crosses the budget cut off line, the panel will proceed as follows:

All the experts involved in the evaluation of at least one proposal in this group will be sent the reviews of all the proposals in the group. They will then examine all the proposals in the group and the existing reviews, and decide on their own personal ranking.

The ERCEA will compile a sub-ranking within the group taking into account the CoIs1, and will then come up with an overall final ranking list.


│ 22

PoC : Panel meeting

Panel meeting

After the evaluation, all the experts will have to confirm by sending an e mail that they agree with the results of the evaluation.

Only in the case a consensus cannot be reached, the ERCEA can call for a panel meeting in order to discuss the ranking order of the proposals.


│ 23

PoC : Feedback to applicants

Feedback to applicants

Applicants are informed on the results of the evaluation:

- Status of their proposal (Retained, Rejected, Failed)

- Scores (pass/ failed) + Comments

Ethical

Granting


│ 24


│ 25


│ 26

Who? PI / PI host institution

Deadline? within 1 month of feedback

How? online form

PoC : Evaluation ProcessRedress

Scope? Formal review = procedural errors only

No scientific judgment

No evaluation of the proposal on the substance

Successful redress ≠ financed project !


│ 27

• Wrong eligibility decision ( 2 cases 2014 DL1)• Application of irrelevant criteria (2 cases 2014 DL2)• Incorrect application of criteria• Wrong thresholds or scoring• Clear conflicts of interest• Unqualified experts• Factual errors affecting whole evaluation

PoC : Evaluation Process Cases that may lead to re-evaluation


│ 28

PoC : Tentative Evaluation Calendar( depending on workload)

DEADLINE 1 DEADLINE 2 DEADLINE 3

Deadline for submission of proposals : 05/02/2015



Allocation of proposals in SEP: 16/02/2015



Deadline Remote evaluation: 16/03/2015



Ranking etc 25/03/2015 Ranking etc 15/07/2015 Ranking etc 20/11/2015

Expected feedback to applicants: April 2015

Expected feedback to applicants: July 2015

Expected feedback to applicants: January 2016

ERC PoC 2014 - Overview

• Call published in December 2013.• ERC received 442 proposals

182 proposals DL1 260 proposals DL2

• 167 LS (79 DL1/ 88 DL2)

• 215 PE (75 DL1/140 DL2)

• 60 SH (28 DL1/32 DL2)

• 2 Synergy proposal (1DL1/1DL2)

• 205 proposals passed all the thresholds• 91 DL1 / 114 DL2

• The SC decided to increase the budget by 20% . Finally 121 proposals funded

61 proposals DL1 60 proposals DL2

• Success rate 27% DL1/ 20% DL2

• Publication of results in the website: list of projects , PI and Host Institutions

ERC PoC 2014 - Overview

ERC Proof of Concept Evolution of Success rates and Budget used

DL1 DL2 DL1 DL2 DL1 DL2 DL1 DL22011 2012 2013 2014

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

41%

33%

55%

45%

24% 24%27%

20%

88%

57%

96%

79%

97% 99% 99% 100%Success rates Budget used

ERC Proof of Concept 2011-2012-2013-2014 - Success rates by domain

PE PE PE PE LS LS LS LS SH SH SH SH2011 2012 2013 2014 2011 2012 2013 2014 2011 2012 2013 2014

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

67%

58%

30%

37%

23%

54%

18%

25%

10%

50%

5%11%

77%

65%

32%

24%18%

30%

20%16%

5%

27%

9%3%

DL1 DL2

PE

5

LS

7

PE

6

PE

8

PE

4

PE

7

PE

2

LS

5

LS

9

PE

3

LS

2

LS

1

SH

4

SH

2

SH

5

LS

8

LS

4

PE

10

SH

3

LS

3

SY

G6

LS

6

PE

9

SH

1

PE

1

0

5

10

15

20

25

30

35

31

25

18 18

16 1615

13

11

9

76

54 4 4 4

32 2

1 1 1 1 1

Number of PoC Projects SIGNED per panel as at January 2015

ERC Proof of Concept 2014- DL1- DL2 # of Funded proposals by panel.

PE

8

LS

7

PE

5

PE

7

PE

4

PE

3

PE

6

LS

5

LS

4

LS

1

PE

2

LS

2

LS

6

SH

4

LS

3

LS

8

LS

9

PE

1

SH

1

SH

5

0

1

2

3

4

5

6

7

8

9

6 6

4

5

4

2

5

4

2 2

1

2 2

1 1 1 1 1

8

4

6

4 4

5

2 2

3

2

3

1 1 1 1 1 1 1

# FUNDED PROPOSALS DL 1

# FUNDED PROPOSALS DL 2


Proof of Concept Grant

PoC 2014: all the applicants have been informed

4 redress cases submitted ( to be dealt with)

│ 35


Number of PoC proposals submitted by deadline until now

DL1 DL2 DL1 DL2 DL1 DL2 DL1 DL2 DL12011 2012 2013 2014 2015

0

50

100

150

200

250

300

78 73 72 67

146 147

183

260

97


│ 37

2014 DL1

2014 DL2

2015 DL1

2014 DL1

2014 DL2

2015 DL1

2014 DL1

2014 DL2

2015 DL1

PE LS SH

0%

10%

20%

30%

40%

50%

60%

41%

54%51%

43%

33% 34%

15% 13%9%

% of PoC submissions by domain


│ 38

LS1

LS3

LS5

LS7

LS9

PE1PE3

PE5PE7

PE9SH2

SH4SH6

0%

2%

4%

6%

8%

10%

12%

14%

2%

4%

1%

5%4%

3%

9%

2%

4%3%

4%

1%

6%

9%

4%

12%

10%

1%2%

3%4%

1%1%

2014 DL1 2014 DL2

2015 DL1

ERC PoC 2015: Submissions by panel %


│ 39

UK DE FR NL ES IL BE CH SE IT EL AT CY DK FI IE PL TR02468

101214161820

18

14

9 97 7

6 6 65

21 1 1 1 1 1 1

# of proposals

Number of PoC proposals submitted by HI country


│ 40

PoC Challenges

What is the life after the PoC?

Applicants behaviour

Fairness of the process

Get good SH applications

Thank you for your attention

Proof of Concept IGLO Veronica Beneitez Pinero March 2015.

Documents

Transcript of Proof of Concept IGLO Veronica Beneitez Pinero March 2015.