The ERC:

a Success Story for the EU

Reinhilde Veugelers

Prof @ KULeuven/FEB/MSI,

Senior Fellow @ Bruegel

Scientific Council Member @ ERC

The European Research Council

Does ERC meet its ambition

to support

excellence, frontier, risky research

• EU science only slowly catching up with US top;

• In selected areas, China is catching up fast to the top

• US Trump administration “less science-prone” in contrast to China’s expanding science budget and ambition to become the N°1 leading science nation

• EU’s low growth & austerity leading to likely shrinking next EC budget for science; shrinking public (research) budgets in many EU MS;

• Growing divide in EU public funding for science between North, South, East;

• Monitoring, evaluation of public research budgets

• More emphasis on (measuring) impact of (public) research (funding) on society

• More emphasis on contribution of public research to (local) economic & societal development

• Debate on public funders becoming more short-term impact oriented, risk-averse

• Increased reliance on bibliometric measures—particularly short-term impact bibliometric measures: highly cited papers, journal impact factors

• Rising science inequality: funding more concentrated on few excellent: star researchers, top institutes, leading countries

• …

A challenging context for EU science…

In the European landscape of funding for

scientific research looms since 2007 a new star:

European Research Council

What difference can/does the ERC make?

What type of research is the ERC selecting and

funding? Excellence, frontier research?

ERC

5

Reasons to celebrate first 10 years

“Scientific excellence is the sole selection criterion. In particular, high risk/high gain

pioneering proposals which go beyond the state of the art, address new and emerging fields of research, introduce unconventional,

innovative approaches are encouraged”.

“its grants will help to bring about new and unpredictable scientific and technological

discoveries - the kind that can form the basis of new industries, markets, and broader social

innovations of the future”.

ERC’s ambitious mission

Why should the ERC target frontier research?

• Frontier research is (like basic research in general) a public good, which is “undersupplied” andtherefore motivates public funding

• Frontier research is especially important for advancement in science

Skewed distribution of contributions

Instigates a multitude of incremental improvements

• Frontier research overproportionally important for linking to technology and innovations

• ERC’s subsidiarity over Member States public funding

Scale advantages from: larger pooling of projects and selection expertise

Scale advantages especially important for risky frontier research

Bottom-up: How the ERC budget is allocated is the sole responsibility of the

Scientific Council, no top down EC determined priority areas

Allocation between Grants Schemes:

StG,CoG, AdG: Priority to young researchers: 2/3 to StG/CoG and 1/3 to AdG

PoC is a small top up on existing grants Mainly to illustrate the contribution of bottom up research to commercial innovation

Synergy grants will be reintroduced in 2018, with total budget comparable to AdG Bottom-up collaborative projects (up till 4 PIs)

Allocation between 25 panels: on basis of proposals;

equal success rate across panels

No pre-fixed allocation between countries/nationalities

│ 8

ERC basics: Allocation of ERC funds to grantees

│ 9

Panel members: typically 375 / call

High-level scientists

Recruited by ScC from all over the world: ~14%

from outside Europe

Referees: typically 2000 / call

Selected by panel members

Panels decide on the ranking/who-gets-funded

• Reviewers are asked to evaluate the proposals on their

ground breaking nature, their level of ambition to go

beyond the state of the art and push the frontier.

EU and AssociatedCountries

(86%)

US (7%)

Other

(7%)

ERC basicsEvaluation of proposals: peer review procedure

It is well known in the science evaluation literature that peer review systems have the tendency to lead to risk

aversion, having the tendency to favor well established, well known research avenues.

The ERC should be able to leverage its scale, quality and reputation to overcome this risk aversion trap:

quality of the evaluators

ERC Achievements After 10 Years

Highly competitive (overall success rate tending to 10%)

65% are at an early-career stage

69 nationalities represented

Working in over 700 different institutions in 33 countries

50% of grantees in 50 institutions

Impact on national programmes and agencies; eg national funding for

best "runners-up“; visiting fellowship programs

│ 10

│ 11

Success Rate by Country of HI

Does ERC meet its ambition

to support

excellence, frontier, risky research

│ 13

ERC’s performance: KPIs

• Headline KPI: Share of publications from ERC funds in top 1% highly

cited

• Target in H2020: 1.6%; Realised: 7% of the ERC-acknowledging

publications were among the top 1%

• International prizes/awards of ERC grantees

• eg Nobel prize winners as grantees

• Qualitative assessement (pilot project):

• 71% of the first 200 completed ERC-funded projects made scientific

breakthroughs and major advances in science (as judged by panels of

peer reviewers)

• Impact on careers of young scientists; young scientists trained on

ERC projects (project starting)

HEADLINE KPI-INDICATOR Top 1% cited papers

Share of all ERC publications in Top 1%

│ 15

Publication Performance by host country

Beyond Headline KPI: ”Share in 1% highly cited”

• 1% KPI only captures “big immediate impact in science” (3yr citation window)

• Frontier research also about

Long-run impact

Wide impact

Risky (big breakthroughs, high failure prob)

Novel, new recombinations of know-how (existing pieces of know-how in new applications)

Crossing disciplines

…

• Other indicators capturing “high risk”

Wang, J, Veugelers, R., Stephan, P. 2015,Bias against novelty in science: a cautionary tale for users of

bibliometric indicators,

• Develop a bibliometric measure of novelty: papers making new combinations of journal references, taking into account the difficulty of

making such new combinations through the distance between the journals

• Find a ‘high risk/high gain” profile of novel research More likely to become top cited (top 1%)

But only when using a long enough time window (at least 4 years);

More likely to stimulate follow-on breakthroughs;

Appreciation of novel research comes from outside its own field; not within its field.

• Also find bias against novelty in standard bibliometric indicators Less likely to be highly cited in typically short-term citation window (3yrs)

More likely to be published in journals with lower Journal Impact Factor

Does ERC reach its ambition of supporting frontier

research?

• What does ERC select? Check “high-gain, high-risk” profile of grantees pre grant Comparing granted vs rejected ERC applicants

Comparing marginally accepted vs marginally rejected ERC applicants

Comparing ERC applicants with non-applicants

• What is the impact of ERC funding? Check “high-gain, high-risk” results of grantees from ERC grants post-grant Compared to counterfactual: similar grantees without ERC funding

Various techniques to assess causality

ERC only just starting to have finished grants (2007, 2008, 2009)

Some first descriptives on publications from finished grantsWork in progress: do not quote

BEFORE

Funded Rejected

Step1

Rejected

Step2

Ratio of TOP1%

papers (C3)

5.4% 2.1% 3.8%

Ratio of TOP 1%

NOVEL (HIGH)

1.8% 2.4% 1.9%

Some first descriptives on publications from finished grantsWork in progress: do not quote

BEFORE AFTER

Funded Rejected

Step1

Rejected

Step2

Funded Rejected

Step1

Rejected

Step2

Ratio of

TOP1%

papers

(C3)

5.4% 2.1% 3.8% 4.5% 1.7% 3.9%

Problem of too short window ex post for most call years (2007-2008) vs (2009-2011)

• ERC good in selecting “high gain”, also in “high risk” ?

• Too soon yet for post-grant impact analysis but its evaluation procedure should be able to pick up early warning signs

• More work in progress Other dimensions of frontier research/impact

Impact on technology: scientific publications as references in patent applications, patents, spin-offs

International orientation: extra-EU co-authorship

• Beyond publication analysis: Impact on careers of PIs & their hires and position in the scientific community

Does ERC reach its ambition of supporting

frontier research?

Beyond publication analysis: career impact of ERC

22

FINDINGS FROM A SMALL SAMPLE OF FINISHED FINAL REPORTS (ABOUT 200 STG)

Impact on career of PI

• 45% StG2009 grantees were promoted full professors or senior researchers; 26% were promoted

Readers / Associated professors or obtained a permanent research position

• 92% StG2009 grantees reported that they could establish/consolidate their research group during the

grant

Impact on career of the team members

• In 44% of the projects the team members became established scientists (researchers employed on

permanent or tenure-tracked positions in research institutions).

• In 23% of the projects the team members found good jobs in industry after the project.

Ongoing career impact monitoring exercise

Thank you for your support and trust in the ERC in the

past 10 years

We hope we can prove we deserve it

We hope we can have your trust in the future: we will need it

We value your views, comments…