Corso di Formazione: HORIZON 2020 IL NUOVO PROGRAMMA QUADRO DI FINANZIAMENTO PER LA RICERCA E L INNOVAZIONE Il ruolo dei valutatori Elisabetta Agostinelli ISM-CNR elisabetta.agostinelli@ism.cnr.it!"
Previous Project & evaluation experience (EU Commission) Framework 2 1990-1992 Science: Physical properties of high Tc superconducting single crystals Framework 3 1993-1995 Industrial and Materials technology Brite- Euram: High Tc superconducting thin films for microwave applications Framework 4 1994-1998 Human Capital and Mobility Network: Critical Current in high Tc superconductors Framework 5 1998-2002 Competitive and sustainable Growth (STREP- Project HIDEMAR - Descartes Prize 2005 - Finalist) Framework 6 2002-2006 NMP-STREP: ProjectNANOSPIN Framework 7 2007-2013 ICT-STREP- Project TERAMAGSTOR; Marie Curie IRSES- Project NANOMAG Framework 7 2007-2013 People-2007-ITN People-2008-ITN People-2009-IEF People-2009-RG People-2010-RG People-2012-IXF People-2013-IXF EACEA-Erasmus Mundus-2011-Action1 EACEA-Erasmus Mundus 2012-Action1 H2020 2014 present EACEA-2014-Quality review EACEA-2014-JMD People-2014-CAR-GF-RI EACEA-2015-EMJMD FET-OPEN-2015-REA
Outline 1 - The evaluation process 2 - How to become an expert 3 - Why to become an expert 4 Statistics about Italian participation 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori
Outline 1 - The evaluation process 2 - How to become an expert 3 - Why to become an expert 4 Statistics about Italian participation 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori
Principles for the Evaluation Process http://ec.europa.eu/smart-regulation/evaluation/docs/evaluation_forward_plan_2014_en.pdf Excellence Transparency Fairness and impartiality projects must demonstrate a high level of quality in relation to the topics and criteria set out in the calls funding decisions must be based on clearly defined rules and procedures, and applicants should receive adequate feedback on the outcome of the evaluation all proposals must be treated equally and evaluated impartially on their merits, irrespective of their origin or the identity of the applicants
Principles for the Evaluation Process http://ec.europa.eu/smart-regulation/evaluation/docs/evaluation_forward_plan_2014_en.pdf Confidentiality Speed and efficiency all proposals and related data, knowledge and documents must be treated in confidence proposals should be evaluated and grants awarded and administered as swiftly as possible, without compromising quality or breaking the rules
Working as an expert H2020 Research domains Science & Technologies Project management Innovation Gender Ethics Social science & Humanities Communication NEED: to increase the number of specialists from the commercial and business communities working on the programme, in addition to the numbers of researchers and academics.
Individual Experts roles Evaluator Observer Monitor Support is responsible for evaluating proposals submitted in response to a call for proposals based on the work programmes (may act as rapporteur, chair, vice-chair). observes the practical workings of the evaluation process, gives independent advice on the conduct and fairness of the evaluation sessions, suggests ways in which the procedures could be improved. assist the Commission in monitoring the implementation of projects that have received funding, taking into account work descriptions, reports and deliverables. assists in the implementation of EU research and innovation (R&I) programmes or policy; in the evaluation of R&I programmes; in designing R&I programmes or policy.
Award criteria
Evaluation scores 0 1 - Poor the proposal fails to address the criterion or cannot be assessed due to missing or incomplete information (unless the result of an obvious clerical error) the criterion is inadequately addressed or there are serious inherent weaknesses 2 - Fair the proposal broadly addresses the criterion but there are significant weaknesses 3 - Good 4 - Very Good the proposal addresses the criterion well but with a number of shortcomings the proposal addresses the criterion very well but with a small number of shortcomings 5 - Excellent the proposal successfully addresses all relevant aspects of the criterion; any shortcomings are minor.
The evaluation exercise Before starting the evaluation process, the experts are briefed on: # the evaluation processes and procedures (including selection and award criteria) # the content of the R&I topics under consideration # the terms of their contract (e.g. confidentiality, impartiality, conflicts of interest, completing tasks and approving reports, penalties for non-compliance) # disregarding excess pages # the need to evaluate proposals as they were submitted, rather than their potential should certain changes be made. *** In Horizon 2020, there will no scope for recommending improvements to proposals
Evaluators: Conflict of interest There is conflict of interest if he/she: # was involved in the preparation of a proposal # benefits directly or indirectly if a proposal is accepted # has a close family or personal relationship with any person representing an applicant # is a director, trustee or partner or is in any way involved in the management of an applicant # is employed or contracted by one of the applicants or any named subcontractors
Steps in the evaluation process
Outline 1 - The evaluation process 2 - How to become an expert 3 - Why to become an expert 4 Statistics about Italian participation 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori
Experts recommendation Universities EU COMMISSION Research Agencies Enterprises Standardization Organizations List of experts in H2020 database Civil Society Organizations
How are the evaluators selected? # A high level of skill, experience and knowledge in the relevant areas (e.g. project management, innovation, exploitation, dissemination and communication) and, provided the above condition can be satisfied, a balance in terms of: # Skills, experience and knowledge # Geographical diversity # Gender # Where appropriate, the private and public sectors # An appropriate turnover from year to year.
CALL FOR EXPERTS (H2020) http://ec.europa.eu/research/participants/data/support/h2020_callindividual_experts_oj_c342_03.pdf Announced at OJC342/03 of 22 Nov 2013
Individual Expert registration https://ec.europa.eu/research/participants/portal/desktop/en/home.html Register
Individual Expert registration https://ec.europa.eu/research/participants/portal/desktop/en/home.html GET INSTRUCTIONS: Table of Contents: 1-Introduction 2. Creating Your Expert Profile 3. Managing Your Expert Contracts 4. Managing Your Payment Requests Expert Registration, Contract Management and Contract Payments - All Videos
Individual Expert registration https://ec.europa.eu/research/participants/portal/desktop/en/home.html Create your profile Get your Expert ID Provide your personal details
If appointed: Useful tips (CNR) http://www.urp.cnr.it/documenti i ricercatori/tecnologi possono impiegare fino a 160 ore annue (circa 22 gg/anno) aggiuntive rispetto all'orario di lavoro, in attività destinate ad arricchimento professionale CNR-rules documents to download Circolare n. 3 /1998: Applicazione in ambito CNR della normativa relativa alle autorizzazioni allo svolgimento di incarichi e attività extraistituzionali. Circolare n. 8/1999: Applicazione in ambito CNR della normativa relativa alle autorizzazioni allo svolgimento d'incarichi. # C99-8aA (15 days in advance to CNR - Dip. del personale ) # C99-8aB (in case N. hours > 160) # C99-8aC (ANPR02 (Anagrafe Prestazioni e incarichi) by April 30 of the following year for the activity performed during the previous year) then # CNR will forward the model (together with the amount of fees received) to Dip. di funzione pubblica.
If appointed: Experts Duties Attend briefing/ evaluation sessions Write reports Write recommendations Briefing on the topics of the relevant calls for proposals and the terms of their contracts can be done in Brussels or remotely (e.g. webstreaming). For each proposal, experts presents their evaluation results in an individual evaluation report (IER), explaining the evaluation scores. Ethical issues
If appointed: Experts Duties Participate in the Consensus group The consensus group discussion is led by a moderator who seeks a consensus and ensures that proposals are evaluated in a fair manner and in line with the established criteria. The consensus group discussion results in a consensus report (CR) including justifications of scores and dissenting views, if any, written by the Rapporteur. The discussion usually takes place in Brussels and includes experts who participated in the individual evaluation. It is also possible to convene a remote consensus group. Important for CNR staff: (extra regular time hours) Non in missione
Outline 1 - The evaluation process 2 - How to become an expert 3 - Why to become an expert 4 Statistics about Italian participation 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori
Working as an expert: Pros and Cons Pros: # Evaluating proposals is the fastest way to learn how to write a good one. # You will start thinking as an evaluator and this will teach you how to avoid weaknesses. # Reading excellent scientific proposals is always stimulating. # Working in an international contest can widen your horizon. # It is well-paid. Covent Garden-Bruxelles
Working as an expert Pros and Cons Cons: # Evaluating proposals is time consuming (time subtracted to your spare time). # You must be available for occasional, short-term assignments # It implies a lot of paper work. # Having to strictly respect deadlines, it could be stressful. Covent Garden-Bruxelles
Outline 1 - The evaluation process 2 - How to become an expert 3 - Why to become an expert 4 Statistics about Italian participation 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori
List of experts 2013 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori
Geographical distribution IDEAS 2012 800 Total : 5313 EXP (Male: 4326; Female: 978) 700 600 500 400 IT: 446 EXP 97 EXP from Res. Org.;.24 EXP from CNR 300 200 100 0 DE UK US FR IT NL ES SE BE CH AT DK Out of 446 Italian EXP, 106 work in other countries Only Countries with No. of EXP>100 are reported
Geographical distribution PEOPLE 2013 200,00 Total 1520 EXP (Male: 837; Female 682) 180,00 160,00 140,00 120,00 100,00 80,00 60,00 40,00 20,00 IT 174 EXP 64 EXP from Res. Org., 20 EXP from CNR 0,00 IT UK DE FR ES PL EL RO BE PT HU BG IE CZ FI SE AT NL TR LT SK IL DK CY LV EE SI CH Only Countries with No. of EXP>10 are reported 174 expert from Italy (out of which 28 work abroad and 146 work in Italy)
Italy/Germany comparison Data base experts FP7 2013 Total data base : 12123 EXP (Male: 8689; Female: 3434) IT: 1178 EXP DE:1506 EXP 800 700 600 500 400 300 200 100 8% 14.2% 8%< ITA/TOT <11% 9%< DE/TOT <14% 0 DE EXP COOPERATION IDEAS PEOPLE CAPACITIES ITA EXP
ITA Res. Org./ CNR comparison Data base experts FP7 2013 Total data base : 12123 EXP (Male: 8689; Female: 3434) IT: 1178 EXP 120 100 80 60 40 20 0 CNR Res. Org. 340 EXP Total 68 EXP Total
Thanks for your attention! 24/03/2015 CNR - Giornata Formativa: Il ruolo dei valutatori