Il ruolo dei valutatori

Transcription

1 Corso di Formazione: HORIZON 2020 IL NUOVO PROGRAMMA QUADRO DI FINANZIAMENTO PER LA RICERCA E L INNOVAZIONE Il ruolo dei valutatori Elisabetta Agostinelli ISM-CNR elisabetta.agostinelli@ism.cnr.it!"

2 Previous Project & evaluation experience (EU Commission) Framework Science: Physical properties of high Tc superconducting single crystals Framework Industrial and Materials technology Brite- Euram: High Tc superconducting thin films for microwave applications Framework Human Capital and Mobility Network: Critical Current in high Tc superconductors Framework Competitive and sustainable Growth (STREP- Project HIDEMAR - Descartes Prize Finalist) Framework NMP-STREP: ProjectNANOSPIN Framework ICT-STREP- Project TERAMAGSTOR; Marie Curie IRSES- Project NANOMAG Framework People-2007-ITN People-2008-ITN People-2009-IEF People-2009-RG People-2010-RG People-2012-IXF People-2013-IXF EACEA-Erasmus Mundus-2011-Action1 EACEA-Erasmus Mundus 2012-Action1 H present EACEA-2014-Quality review EACEA-2014-JMD People-2014-CAR-GF-RI EACEA-2015-EMJMD FET-OPEN-2015-REA

3 Outline 1 - The evaluation process 2 - How to become an expert 3 - Why to become an expert 4 Statistics about Italian participation 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori

4 Outline 1 - The evaluation process 2 - How to become an expert 3 - Why to become an expert 4 Statistics about Italian participation 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori

5 Principles for the Evaluation Process Excellence Transparency Fairness and impartiality projects must demonstrate a high level of quality in relation to the topics and criteria set out in the calls funding decisions must be based on clearly defined rules and procedures, and applicants should receive adequate feedback on the outcome of the evaluation all proposals must be treated equally and evaluated impartially on their merits, irrespective of their origin or the identity of the applicants

6 Principles for the Evaluation Process Confidentiality Speed and efficiency all proposals and related data, knowledge and documents must be treated in confidence proposals should be evaluated and grants awarded and administered as swiftly as possible, without compromising quality or breaking the rules

7 Working as an expert H2020 Research domains Science & Technologies Project management Innovation Gender Ethics Social science & Humanities Communication NEED: to increase the number of specialists from the commercial and business communities working on the programme, in addition to the numbers of researchers and academics.

8 Individual Experts roles Evaluator Observer Monitor Support is responsible for evaluating proposals submitted in response to a call for proposals based on the work programmes (may act as rapporteur, chair, vice-chair). observes the practical workings of the evaluation process, gives independent advice on the conduct and fairness of the evaluation sessions, suggests ways in which the procedures could be improved. assist the Commission in monitoring the implementation of projects that have received funding, taking into account work descriptions, reports and deliverables. assists in the implementation of EU research and innovation (R&I) programmes or policy; in the evaluation of R&I programmes; in designing R&I programmes or policy.

9 Award criteria

10 Evaluation scores Poor the proposal fails to address the criterion or cannot be assessed due to missing or incomplete information (unless the result of an obvious clerical error) the criterion is inadequately addressed or there are serious inherent weaknesses 2 - Fair the proposal broadly addresses the criterion but there are significant weaknesses 3 - Good 4 - Very Good the proposal addresses the criterion well but with a number of shortcomings the proposal addresses the criterion very well but with a small number of shortcomings 5 - Excellent the proposal successfully addresses all relevant aspects of the criterion; any shortcomings are minor.

11 The evaluation exercise Before starting the evaluation process, the experts are briefed on: # the evaluation processes and procedures (including selection and award criteria) # the content of the R&I topics under consideration # the terms of their contract (e.g. confidentiality, impartiality, conflicts of interest, completing tasks and approving reports, penalties for non-compliance) # disregarding excess pages # the need to evaluate proposals as they were submitted, rather than their potential should certain changes be made. *** In Horizon 2020, there will no scope for recommending improvements to proposals

12 Evaluators: Conflict of interest There is conflict of interest if he/she: # was involved in the preparation of a proposal # benefits directly or indirectly if a proposal is accepted # has a close family or personal relationship with any person representing an applicant # is a director, trustee or partner or is in any way involved in the management of an applicant # is employed or contracted by one of the applicants or any named subcontractors

13 Steps in the evaluation process

14 Outline 1 - The evaluation process 2 - How to become an expert 3 - Why to become an expert 4 Statistics about Italian participation 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori

15 Experts recommendation Universities EU COMMISSION Research Agencies Enterprises Standardization Organizations List of experts in H2020 database Civil Society Organizations

16 How are the evaluators selected? # A high level of skill, experience and knowledge in the relevant areas (e.g. project management, innovation, exploitation, dissemination and communication) and, provided the above condition can be satisfied, a balance in terms of: # Skills, experience and knowledge # Geographical diversity # Gender # Where appropriate, the private and public sectors # An appropriate turnover from year to year.

17 CALL FOR EXPERTS (H2020) Announced at OJC342/03 of 22 Nov 2013

18 Individual Expert registration Register

19 Individual Expert registration GET INSTRUCTIONS: Table of Contents: 1-Introduction 2. Creating Your Expert Profile 3. Managing Your Expert Contracts 4. Managing Your Payment Requests Expert Registration, Contract Management and Contract Payments - All Videos

20 Individual Expert registration Create your profile Get your Expert ID Provide your personal details

21 If appointed: Useful tips (CNR) i ricercatori/tecnologi possono impiegare fino a 160 ore annue (circa 22 gg/anno) aggiuntive rispetto all'orario di lavoro, in attività destinate ad arricchimento professionale CNR-rules documents to download Circolare n. 3 /1998: Applicazione in ambito CNR della normativa relativa alle autorizzazioni allo svolgimento di incarichi e attività extraistituzionali. Circolare n. 8/1999: Applicazione in ambito CNR della normativa relativa alle autorizzazioni allo svolgimento d'incarichi. # C99-8aA (15 days in advance to CNR - Dip. del personale ) # C99-8aB (in case N. hours > 160) # C99-8aC (ANPR02 (Anagrafe Prestazioni e incarichi) by April 30 of the following year for the activity performed during the previous year) then # CNR will forward the model (together with the amount of fees received) to Dip. di funzione pubblica.

22 If appointed: Experts Duties Attend briefing/ evaluation sessions Write reports Write recommendations Briefing on the topics of the relevant calls for proposals and the terms of their contracts can be done in Brussels or remotely (e.g. webstreaming). For each proposal, experts presents their evaluation results in an individual evaluation report (IER), explaining the evaluation scores. Ethical issues

23 If appointed: Experts Duties Participate in the Consensus group The consensus group discussion is led by a moderator who seeks a consensus and ensures that proposals are evaluated in a fair manner and in line with the established criteria. The consensus group discussion results in a consensus report (CR) including justifications of scores and dissenting views, if any, written by the Rapporteur. The discussion usually takes place in Brussels and includes experts who participated in the individual evaluation. It is also possible to convene a remote consensus group. Important for CNR staff: (extra regular time hours) Non in missione

24 Outline 1 - The evaluation process 2 - How to become an expert 3 - Why to become an expert 4 Statistics about Italian participation 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori

25 Working as an expert: Pros and Cons Pros: # Evaluating proposals is the fastest way to learn how to write a good one. # You will start thinking as an evaluator and this will teach you how to avoid weaknesses. # Reading excellent scientific proposals is always stimulating. # Working in an international contest can widen your horizon. # It is well-paid. Covent Garden-Bruxelles

26 Working as an expert Pros and Cons Cons: # Evaluating proposals is time consuming (time subtracted to your spare time). # You must be available for occasional, short-term assignments # It implies a lot of paper work. # Having to strictly respect deadlines, it could be stressful. Covent Garden-Bruxelles

27 Outline 1 - The evaluation process 2 - How to become an expert 3 - Why to become an expert 4 Statistics about Italian participation 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori

28 List of experts /03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori

29 Geographical distribution IDEAS Total : 5313 EXP (Male: 4326; Female: 978) IT: 446 EXP 97 EXP from Res. Org.;.24 EXP from CNR DE UK US FR IT NL ES SE BE CH AT DK Out of 446 Italian EXP, 106 work in other countries Only Countries with No. of EXP>100 are reported

30 Geographical distribution PEOPLE ,00 Total 1520 EXP (Male: 837; Female 682) 180,00 160,00 140,00 120,00 100,00 80,00 60,00 40,00 20,00 IT 174 EXP 64 EXP from Res. Org., 20 EXP from CNR 0,00 IT UK DE FR ES PL EL RO BE PT HU BG IE CZ FI SE AT NL TR LT SK IL DK CY LV EE SI CH Only Countries with No. of EXP>10 are reported 174 expert from Italy (out of which 28 work abroad and 146 work in Italy)

31 Italy/Germany comparison Data base experts FP Total data base : EXP (Male: 8689; Female: 3434) IT: 1178 EXP DE:1506 EXP % 14.2% 8%< ITA/TOT <11% 9%< DE/TOT <14% 0 DE EXP COOPERATION IDEAS PEOPLE CAPACITIES ITA EXP

32 ITA Res. Org./ CNR comparison Data base experts FP Total data base : EXP (Male: 8689; Female: 3434) IT: 1178 EXP CNR Res. Org. 340 EXP Total 68 EXP Total

33 Thanks for your attention! 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori