H2020 Evaluation process the importance of impact. Dan Andreé, Vinnova, Brysselkontoret

Transcription

1 H2020 Evaluation process the importance of impact Dan Andreé, Vinnova, Brysselkontoret

2 Bild 2 H2020

3 2014 Juncker s priorities A new boost for jobs, growth and investment A connected digital single market A resilient Energy Union with a forward-looking climate change policy A deeper and fairer internal market with a strengthened industrial base A deeper and fairer Economic and Monetary Union (EMU) A reasonable and balanced free trade agreement with the United States An area of Justice and Fundamental Rights based on mutual trust Towards a new policy on migration Europe as a stronger global actor A Union of democratic change. Bild 3

4 Award Criteria in Horizon 2020 (pillar 2/3) Type of action All types of action Excellence The following aspects will be taken into account, to the extent that the proposed work corresponds to the topic description in the work programme. Clarity and pertinence of the objectives; Credibility of the proposed approach. Each criteria is scored 0-5: #Impact Impact The extent to which the outputs of the project should contribute at the European and/or International level to: The expected impacts listed in the work programme under the relevant topic #Excellence Quality and efficiency of the implementation The following aspects will be taken into account: Coherence and effectiveness of the work plan, including appropriateness of the allocation of tasks and resources; Complementarity of the participants within the consortium (when relevant); Appropriateness of the management structures and procedures, including risk and innovation management.

5 Bild 5

6 Examples of good comments in Impact criterion The high level of industry/end-users engagement and clear industry/end-users involvement indicate a high potential impact. The expected outcome is of primary importance for EU strategy. A specific WP is devoted to knowledge transfer to industry The proposal addresses very well all the expected impacts listed in the call. The past successful collaboration of the majority of the participants in previous programmes ensures. Based on the previous experience, it seems that this proposal could achieve the necessary critical mass for.

7 Less good comments: Not possible to assess due to incomplete information The description of the expected impact is very short and difficult to assess. The long-term impact of the project is unclear. These activities represent standard approaches of publication (e.g. websites). The expected impact is difficult to assess as the description does not follow the structure in the call text.

8 Evaluation Process Proposal Eligible proposal Expert Expert Expert Expert Expert Minimum 3 experts but usually more Individual Evaluation Report Individual Evaluation Report Individual Evaluation Report Often 5-8 Individual evaluation Individual Evaluation Report Consensus group Individual Evaluation Report Consensus HORIZON 2020 Consensus Report 8

9 Consensus meetings Leads to agreement ESR Evaluation Summary Report Is certainly not about average score and seldom about median score Outlying opinions could be right or wrong Evaluators have different roles Commission does not influence the scores but may significantly improve the ESR

10 Excellence Implementation Impact Description of impact is relevant in Bild 10

11 Excellence (impact angle) Impact starts with Excellence: Clarity and pertinence of the objective Soundness of the concept, including trans-disciplinary considerations, where relevant Extent that proposed work is ambitious, has innovation potential, and is beyond the state of the art Credibility of the proposed approach

12 Implementation Coherence and effectiveness of the work plan, including appropriateness of the allocation of tasks and resources Complementarity of the participants within the consortium Appropriateness of the management structures and procedures, including risk and innovation management

13 Impact criterion often decisive Excellence necessary but not sufficient

14 Impact The expected impacts listed in the work programme under the relevant topic Always follow list in the call text to 110% - table - you can add but not leave out Quantitative/qualitative benefits for EU-citizens, jobs and growths who will gain - show that you think about it Enhancing innovation capacity and integration of new knowledge Business plan relevant partners in consortia Strengthening the competitiveness and growth of companies by developing innovations meeting the needs of European and global markets; and, where relevant, by delivering such innovations to the markets Show that all parts of the proposals excellence management impact are linked All partners should describe exploitation plans Any other environmental and socially important impacts (not already covered above) Can you measure impact? Key-performance-indicators (KPI) Effectiveness of the proposed measures to exploit and disseminate the project results (including management of IPR), to communicate the project, and to manage research data where relevant Not enough to copy a template at least adapt it! Difference between dissemination and communication IPR often it is enough to describe that you are going to deal with it

15 Conclusions - impact In many calls the score on Impact will decide if a project will be funded or not Start with Impact description followed by Management and Excellence Follow exactly the Impact description in the call text and cover all aspects Even with modest impact a good description might give 0,5 1 point extra A good description might not always give more points but avoids reduced score Quantify impact but be realistic Refer to relevant EU-policy Juncker's priorities Make references to relevant WPs Include Impact partners users Track record of partners Pay attention to the composition of the Advisory Board

16 Impact Very difficult to prove impact but, e.g. 20 % increase Put together a chain of evidence of activities which, when viewed in sequence, justifies the impact

17 General advice & observations Evaluators do not know more than call text and application. The three parts of the proposal (excellence, management, impact) must form ONE proposal. The proposal must be understood by evaluators with different background specialists generalist (c.f interdisciplinary projects) The importance of impact leads (sometimes) to that projects of highest scientific excellence do not get funded i.e. projects with lower quality get funded. It is difficult to stand out in a proposal. However, it is unfortunately. easier to be punished Evaluators sometimes focus on finding mistakes avoid them!

18 Impact ABC Excellence EU policy Management Partners Refer to earlier track records Governance Advisory Board Relevant WPs/tasks Impact Not enough to repeat call text! Put together chain of proof

19 Important FP9 key-words: Juncker s 10 priorities Moedas Open Science - digitalisation Open Innovation, Open to the World - global research area European Innovation Council Excellence impact - openness European Added value - Excellence UN Sustainable goals Industrial competitiveness Mission oriented impact in society

20 Forecast FP9 Dan Andrée Open Excellence Science Open European Innovation Council Open to the world Challenges/missions (focus on EU-policy with impact + UN Sustainable Development) Juncker priorities indicates public sector/end users involvement very important to deliver impact Bild 20

21 pen Science pen Innovation pen to the world