Horizon 2020 SME Instrument Experts Briefing

Part 2

2016/2017

CONTENT

1. The Evaluation Process;

2. Your role as independent experts:

3. The Individual Evaluation Report;

4. Tips & tricks to complete the IER.

1. The Evaluation Process

Overview of the Evaluation Process for the SME Instrument

Receipt of proposals

Individual evaluation

Automated Consensus

Report Finalisation

Remote evaluation

Individual Evaluation Reports

(explanatory comments)

Automated Consensus

Report (arithmetic median/

No negotiation)

Standardised Evaluation Summary

Report

Eligibility check

Allocation of proposals to evaluators

Final ranked list

Informing the

applicants

Admissibility and Eligibility Checks Done by EASME

Admissibility - proposals must be:

Readable, Accessible and Printable;

Complete (all requested forms);

Eligibility:

SME status, country;

Limited number of pages (10 – Phase 1; 30 - Phase 2)

Excess pages are watermarked - ignore them;

Only one application per company allowed for all phases.

If you spot an issue relating to eligibility, please inform EASME.

Evaluation time line

Allocation of all proposals right after the cut-off date;

Accept/Decline task within 24hrs- otherwise the task will be reallocated to another expert;

7 calendar days to complete the evaluations;

Inform us as early as possible if you face difficulties to meet this deadline;

Completing the evaluation reports before the deadline is appreciated.

Payment of experts Reimbursement rate = 450 €/day (6 - Phase 1 & 4 – Phase 2

proposals/day);

Payment is done separately for Phase 1 & 2 and starts at the end of the evaluation process;

You will receive an email informing you that the payment procedure is starting;

Please submit your request for payment within the given deadline;

If you miss claiming your reimbursement within the designated period, we will be unable to do ad-hoc payments and there will be a long delay in payment;

Detailed information is included in the FAQ for experts.

2. Your role as independent expert

Guiding principles

Independence: evaluating in your personal capacity;

Impartiality: treating all proposals equally;

Objectivity: evaluating each proposal as submitted;

Accuracy: evaluating against the official evaluation criteria;

Consistency: applying same standards of judgment.

Confidentiality

As we are dealing with close-to-market activities, confidentiality is crucial and is the basis for a trusted evaluation.

Do not discuss evaluation matters;

Do not contact partners in the consortium, sub-contractors or any third parties;

Maintain the confidentiality of documents: destroy or delete all confidential documents upon completing your work.

Conflict of interest

It is the responsibility of the evaluator to determine whether a conflict of interest exists.

Before accepting any evaluation:

• Check if you are in a situation of conflict of interest;

• If yes, decline the task assigned and justify the reason;

• Inform the SME Instrument team.

Please note that EASME has the final decision on whether a conflict of interest exists.

Definition of conflict of interest (1/2)

Check Article 2 – Annex 1 - Code of conduct of the experts' contract.

A conflict of interest arises if an expert:

a) was involved in the preparation of the proposal;

b) stands to benefit directly or indirectly if the proposal is accepted;

c) has a close family or personal relationship with any person representing an applicant legal entity;

d) is a director, trustee or partner or is in any way involved in the management of an applicant legal entity;

e) is employed or contracted by one of the applicant legal entities or any named subcontractors;

f) is a member of an Advisory Group set up by the Commission to advise on the preparation of EU or Euratom Horizon 2020 Work Programmes, or Work Programmes in an area related to the call for proposals in question;

g) is a National Contact Point, or is directly working for the Enterprise Europe Network;

h) is a member of a Programme Committee.

Definition of conflict of interest (2/2)

In the following situations, EASME will decide whether a COI exists

Were employed by an applicant or sub-contractor in the last 3 years;

Were involved in a grant agreement, the membership of management structures or a business collaboration with an applicant in the last 3 years;

Are in any other situation that casts doubt on your impartiality or that could reasonably appear to do so

Am I in a conflict of interest situation?

I have been involved as a consultant/advisor/service provider/applicant preparing a proposal.

No, as long as you are not involved in the evaluation of proposals submitted in the same topic in which you are evaluating.

Yes, if you are evaluating proposals in the same topic. Please note that you may be required to suspend your evaluator activities during the ongoing evaluation.

I have been asked to give a presentation on the programme.

No, there is no conflict of interest if you speak in general about the Programme.

Yes, if you mention the contents/details of a proposal you have evaluated.

Can I be an evaluator and a coach at the same time?

No, in this combination of roles there is a potential conflict of interest.

Am I in a conflict of interest situation?

3. Individual Evaluation Report

Proposal scoring Each evaluation sub-criterion is scored out of 10 points (one

decimal may be used);

Each evaluation sub-criterion question has the same weight, except overall perception that weights 25% of the total score of that criterion;

The individual scores (from 0 to 10) given to each sub-criterion are automatically converted to a scale from 0 to 5 to calculate each of the three Criterion scores per evaluator.

The total possible score for a proposal is 15 points.

The scale used to obtain the qualitative assessment is the following:

Scores from 0 to 2.99 – generate "Insufficient" Scores from 3 to 4.99 – generate "Insufficient to Fair" Scores from 5 to 6.99 – generate "Fair to Good" Scores from 7 to 8.99 – generate "Good to Very Good" Scores from 9 to 10.0 – generate "Very Good to Excellent"

The final score

Each proposal is assessed by 4 evaluators;

The median score across all 4 evaluators for each criterion

Overall score is the sum of the 3 criteria median scores.

How is an IER structured?

PHASE 1 PHASE 2

Operational capacity

Impact Impact

Excellence Excellence

Quality and efficiency of implementation

Quality and efficiency of implementation

Subcontracting

Scope of the proposal Scope of the proposal

Threshold: 13 Threshold: 12

Impact PHASE 1 PHASE 2

Threshold: 4 Threshold: 4

Sub-criteria: 8 Sub-criteria: 9

Main aspects to tackle in the evaluation:

• Compliance with the relevant Work Programme; • The demand/market of the innovation proposed; • The targeted users or user groups; • The market conditions; • The impact on the growth of the applying company; • The commercialisation plan; • The European dimension; • The IPR filing status and ownership, licensing; • The Regulatory and standard requirements. To determine the ranking, the score for the criterion ‘impact’ will be given a weight of 1.5.

Excellence

PHASE 1 PHASE 2

Threshold: 4 Threshold: 3

Sub-criteria: 6 Sub-criteria: 7

Main aspects to tackle in the evaluation: • The new market opportunities for EU/global challenges; • The current stage of development of the innovation (TRL6

or above except SMEInst-05); • The comparison with known commercial solutions; • The objectives and the approach/activities to be

developed; • The commercial viability of the innovation proposed; • The risks and opportunities of the market introduction.

Implementation

PHASE 1 PHASE 2

Threshold: 4 Threshold: 3

Sub-criteria: 4 Sub-criteria: 5

Main aspects to tackle in the evaluation: • The resources to develop the activities; • The technical/scientific knowledge/management

experience; • The time frame and the implementation description; • The work packages and major deliverables and

milestones.

For Phase 2 only: subcontracting

Only to be used when a proposal is very clearly submitted in a wrong topic.

If a proposal is partially relevant to the topic, it should be considered within scope.

If a proposal is considered not innovative, not disruptive, not well explained, incomplete, etc., this opinion should be expressed by means of a lowered score in the relevant (sub) criteria.

The 'out of scope' option

The 'out of scope' option

As long as there is a link between a proposal and the challenges described in the relevant Work Program, it is IN scope. Examples: • A proposal aiming at developing a technology for conversion of

pressurized gas to energy is at TRL3 instead of TRL6. The proposal is IN scope!

• A proposal concerning an innovative textile for the fashion industry is submitted under the Transport topic. The proposal is OUT of scope!

Operational capacity - Phase 2 only

If you believe that an applicant does NOT have the operational capacity to carry out the proposed work, you should choose NO, justify the reason and score the Quality and efficiency of implementation below the threshold (<3).

Operational capacity - Phase 2 only

Examples: • The company does not have technical & financial resources to

manage the project/subcontractors. NO!

• The company will subcontract many tasks but has the resources to manage the core activities. YES!

Suggestions: • Check the company's website

• Check the team involved in the project

Subcontracting - Phase 2 only

• Regulated under Art 13 of the H2020 SME Instrument Phase 2 Model Grant Agreement (see link - page 496);

• Subcontracting is NOT restricted to a limited part of the action;

• It is in the SME Instrument spirit that the applying SME has the capacity to carry out the activity;

• Compliance with best value-for-money is assessed during the evaluation (not for topic SMEInst-05-2016-2017).

Subcontracting - Phase 2 only

• A table in annex 4-5 to detail each subcontractor and task subcontracted.

Subcontracting - Phase 2 only

For each subcontracted task, there are only two options: • Yes • No or lack of explanation If "no or lack of explanation", experts need to justify and reflect this in the assessment in the Quality & efficiency of implementation criterion (score below the threshold: <3). By default, the task is set to 'yes' even when the proposal does not foresee any subcontracting. If there are no subcontractors in the proposal you shall not change this "yes" set by default.

New – 2 Health topics

SMEInst-06-2016-2017 - Accelerating market introduction of ICT solutions for Health, Well-Being and Ageing Well (same rules as other topics)

SMEInst-05-2016-2017 - Supporting innovative SMEs in the

healthcare biotechnology sector Specific Rules

• TRL indication does not apply • Contribution up to € 5 million • Funding rate at 100% • General model Grant Agreement applies • No evaluation of best value for money of subcontracting

4. Tips and tricks to complete the IER

How to write a quality IER?

Maintain a very high quality standard – do not be too positive;

Reflect shortcomings in a lower score for the relevant criterion;

Provide explanation of shortcomings but do not give recommendations;

How to write a quality IER?

Yes

No

Dedicated and thorough comments for each sub-criterion

Comments repeating the sub-criterion description and/or identical comments for all sub-criteria and/or blank text boxes

Consistency between scores and comments

Good score but negative comments

Coherence between operational capacity/subcontracting and the score of the 'Implementation' criterion

Selecting NO for operational capacity/subcontracting and scoring 'Implementation' above threshold (3)

During each cut-off

Due to the time constraints your availability and accountability is crucial! • Be accurate when you specify your availability; • Accept/decline the tasks assigned within 24 hours; • Let us know if you can take more evaluations; • Stick to the deadlines; • Inform us asap if you cannot cope with the workload; First cut-off Phase 2 on the 3 of February 2016 !

Further sources of information Horizon 2020 Participant Portal

http://ec.europa.eu/research/participants/portal/

EASME website

https://ec.europa.eu/easme/en/horizons-2020-sme-instrument

FAQ:

https://ec.europa.eu/easme/sites/easme-site/files/FAQ_for_experts_SME%20Instrument_02_10_15.pdf

Functional mail box for all your questions and comments

EASME-SME-EXPERTS@ec.europa.eu

Project Officer responsible for each topic;

firstname.familyname@ec.europa.eu

THANK YOU FOR YOUR ATTENTION

All images except SME Support logo © DrAfter123 iStockphoto.com