IEE Project Performance Indicators. Final report Executive summary

Transcription

1 IEE Prject Perfrmance Indicatrs Final reprt Executive summary Reprt fr EACI Ricard-AEA/R/ED57321 Issue Number 3 Date 06/12/2012

2 IEE Prject Perfrmance Indicatrs Custmer: Executive Agency fr Cmpetitiveness and Innvatin Custmer reference: EACI/IEE/2011/001 Cntact: Lisa Beardmre Ricard-AEA Ltd Gemini Building, Harwell, Didct, OX11 0QR t: e: Ricard-AEA is certificated t ISO9001 and ISO14001 Authr: Lisa Beardmre, Heather Haydck Apprved By: Heather Haydck Date: 06 December 2012 Signed: Ricard-AEA reference: Ref: ED Issue Number 3 Ref: Ricard-AEA/R/ED57321/Issue Number 3 i

3 Executive Summary Intrductin The EU has set itself ambitius targets fr reducing greenhuse gas emissins, imprving energy efficiency and increasing the use f renewables. The Intelligent Energy Eurpe Prgramme (IEE), nw in its secnd phase (IEE II, ), has a key rle t play in this by funding specific actins within the market place t vercme (nn-technlgical) barriers t bth the efficient use f energy and the greater use f new and renewable energy surces, with the verall aim f cntributing t the prvisin f secure, sustainable and cmpetitively priced energy fr Eurpe. In addressing nntechnical barriers, the prgramme cmplements wrk being dne under the EC Framewrk prgrammes and elsewhere t imprve energy efficient and renewable energy technlgies. The IEE prgramme is managed, n behalf f the Eurpean Cmmissin, by the Executive Agency fr Cmpetitiveness and Innvatin (EACI). IEE prjects typically address ne f fur areas fr funding: Energy efficiency and the ratinal use f energy (SAVE) Funding in this area mainly targets the imprvement f energy efficiency and the ratinal use f resurces in the industry, prducts and building sectrs. New and renewable resurces (ALTENER) In this area, funding is prvided t increase the share f renewables in the prductin f electricity, heat and cling, and t integrate them in the lcal energy systems. Energy in transprt (STEER) This strand cvers initiatives targeting energy savings and energy efficiency in the transprt sectr, including stimulatin f demand fr alternative fuels and clean and energy-efficient vehicles. Integrated initiatives These initiatives cver several ecnmic sectrs r several f the main areas f energy efficiency, new and renewable resurces and energy in transprt at the same time. Different prjects have a different fcus but each addresses at least ne f five defined fields f delivery, which are: Enabling plicy Activities supprting develpment and implementatin f plicies and legislatin at EU, natinal and lcal level, e.g.: strategic prjects prviding technical inputs and market feedback t plicy makers; actins supprting the implementatin f Eurpean directives and plicies; actins assisting reginal and lcal plicy makers and civil servants t develp and implement lcal strategies and actins plans. Market transfrmatin Actins transfrming markets making them mre efficient, reliable and transparent are expected under this field f delivery, e.g.: accelerating uptake f certified prducts and services; helping industries and ESCOs t prmte trading f cmpetitive prducts acrss the EU, tackle cmmn market barriers, and imprve transparency in the market thrugh imprved standards, labelling and quality assurance schemes. Preparing the grund fr Investment Activities triggering investments and facilitating access t finance e.g.: prjects facilitating an extended use f structural funds and ther investment funds; actins prviding the necessary 1

4 infrmatin and tls t public and private decisin makers t mbilise investments; actins supprting prject develpment assistance and making energy efficiency and renewable energy prjects bankable; preparatin f tendering prcedures and cntractual arrangements. Building capacities and skills Activities which imprve the capacity and skills f Eurpean market actrs e.g.: actins aimed at creating apprpriate skills, cmpetencies and abilities f all market players in rder t strengthen their rle in a sustainable energy market. Changing behaviur Activities aimed at infrming stakehlders with a view t changing behaviur and fstering cmmitment e.g.: actins prmting large scale infrmatin and raising awareness activities n energy issues; actins encuraging market players t change behaviur and mtivate them t take actin n sustainable energy issue. In ttal, ver the perid f IEE II mre than 370 prjects have been funded t date. Under the Prgramme, the requirements fr prjects t adpt an impact-fcused apprach t perfrmance indicatrs were strengthened; t this end, the applicatin frm and the evaluatin criteria were adapted t make SMART indicatrs an integral part f the prject. Mrever, since the 2009 call fr prpsals, the EACI requires prpsers - in additin t their prject specific indicatrs - t deliver Cmmn Perfrmance Indicatrs (CPIs) cvering the renewable energy prductin triggered, primary energy savings, investments mbilised, and GHG reductin. In this effrt t ensure an apprpriate mnitring and evaluatin f the impacts f the prgramme, the EACI launched (in August 2011) an invitatin t tender t establish a set f typical utputs and perfrmance indicatrs f prjects supprted in the framewrk f IEE, as well as an applicatin methdlgy and guide fr their use. This aim f the tender was: Help prpsers t define apprpriate perfrmance indicatrs (and in turn frmulate gd quality prpsals); Prvide supprt t IEE prject cnsrtia t mnitr their perfrmance indicatrs; Prvide supprt t IEE prject cnsrtia n hw t turn their measured impact int cmmunicatin messages; Encurage prjects with higher and mre visible impacts; Supprt the EACI in extracting results and aggregating them at bth sectr (by field f delivery) and prgramme level; Supprt the EACI t identify prjects with imprtant impacts, t be used as best practices and t be taken int accunt fr the priritisatin and definitin f future wrk. The main activities f the wrk carried ut within the tender cnsisted f: Review f a number f finished and/r nging IEE II prjects, in rder t carry ut a cllectin, analysis and reality checking f the actual utputs and utcmes and impacts and f the perfrmance indicatrs; Definitin f a set f specific perfrmance indicatrs t measure the utputs and utcmes and impacts f the IEE II prjects; Preparatin f an applicatin methdlgy and guide fr their use; Analysis f the Cmmn Perfrmance Indicatrs f the IEE prjects apprved in the framewrk f the IEE Call Fllwing the evaluatin phase f the received ffers, the EACI cncluded a service cntract with AEA Technlgy plc (Ricard-AEA Ltd since Nvember 2012). The cntract ran frm March 2012 t December

5 Challenges and apprach As IEE prjects address nn-technlgical barriers, there are specific issues t be faced in estimating and measuring the impacts f prjects n the IEE targets. The develpment f perfrmance indicatrs fr IEE prjects and the IEE prgramme is particularly challenging because: IEE prjects address nn-technical barriers, which are influenced by many different factrs and makes it difficult t link utputs t impacts (the attributin gap) Prjects vary widely aims, tpics, scpe, scale, lcatin Within the typical 2-3 year duratin f the cntracts, many prjects have relatively limited impact; lnger term impacts can be significant but are difficult t frecast and there is n pst-prject mnitring in place IEE prgramme participants find it difficult t define prject impacts in terms f the existing Cmmn Perfrmance Indicatrs (CPIs). They take different appraches t the prblem; this lack f cnsistency is a prblem fr prject fficers and prpsal evaluatrs Diverse impacts are difficult t aggregate at the prgramme level All f the abve make prgramme impacts hard t measure r prve The main purpse f this service cntract was t develp a set f typical prject utputs and perfrmance indicatrs, plus guidance, t assist prpsers in selecting meaningful and effective ways t mnitr, measure and cmmunicate prject perfrmance and impact, in a manner which is cnsistent acrss the prgramme. Insights frm Prject Crdinatrs, literature, and expert wrkshps Within the current system prject impact indicatrs vary depending n: Sectr & key actin Types f activity undertaken Level f implementatin Target grups, e.g. plicy makers r industry stakehlders Field f delivery & bjective, e.g. technlgy awareness r market develpment Many prjects define their perfrmance indicatrs in terms f invlvement f the stakehlder grups that the prject is targeting. Mst prject-specific indicatrs are utput indicatrs rather than impact indicatrs. In sme prjects there is als a tendency t underestimate shrt term (within prject duratin) impacts while verestimating likely lng term (2020) impacts. Between May 2012 and August 2012, interviews were carried ut with 31 prject crdinatrs. The key feedback that can be taken frm the interviews is that an almst equal number f crdinatrs had sme experience in setting IEE indicatrs befre, as the number that had n experience. Mst crdinatrs fund indicatrs useful fr shaping and defining their prjects, and used them thrughut the prject t mnitr prgress. A large number f crdinatrs fund that gaining access t data in rder t calculate quantitative impacts was very challenging. Crdinatrs, mainly thse in the ALTENER strand, indicated that a Green Jbs indicatr wuld be a useful and plitically relevant indicatr fr IEE. Hwever, relatively few prject crdinatrs felt that they wuld be able t calculate this indicatr with a reasnable degree f cnfidence. In mst cases, prject crdinatrs had suggestins t make arund Cmmn Perfrmance Indicatr use and guidance. Mst crdinatrs wanted better guidance, and felt that the EACI shuld identify 3

6 methdlgies that they shuld use when calculating their impacts, s that prjects becme cmparable. In reviewing the existing literature, and ther prgrammes delivering similar utputs, it was fund that very few prgrammes have perfrmance indicatrs fr sfter measures (e.g. LIFE+, GEF), and many acknwledge this is very difficult t d. A majr difference nted between IEE and ther wellestablished prgrammes, is that these ther prgrammes tend t use highly prescriptive sets f utput indicatrs, and prvide very clear guidance n each stage f the results chain. A key finding frm a number f reviews f perfrmance indicatr systems is that guidance n hw t establish plausible linkages between their utputs and their lng term impacts has t be clear and well written. Indicatrs shuld be apprpriate t the prject aims and bjectives and nt be gverned by hw measureable they are. An internal wrkshp f evaluatin, innvatin, prgramme management, renewable energy, energy efficiency, and transprt experts brught a number f perspectives t bear n methds t imprve the way IEE prjects can link their utputs t their impacts, and make the impacts that are measured mre reliable and relevant. In this wrkshp, the idea f using the AIDA framewrk (awareness, interest, desire, actin) was prpsed as a way f linking utputs and impacts. This apprach is explained in the guidance dcument, and culd be a very useful tl, especially t thse prjects wh s activities are fcussed at the awareness end f the chain. Awareness Interest Desire Actin Reviewing the existing perfrmance indicatr system based n CPIs The current system f perfrmance indicatrs based n the fur CPIs asks applicants t define impact indicatrs based n a set f utputs that are linked t the prject s bjectives. The CPIs are: - Investments made by Eurpean stakehlders in sustainable energy, triggered by IEE ( EUR) - Renewable energy prductin triggered by actins supprted by IEE (te/ year) - Energy savings triggered by actins supprted by IEE (te/ year) - Reductins f greenhuse gas emissins triggered by actins supprted by IEE e/ year). (tco2 The CPIs are estimated fr bth shrt-term (by the end f the prject) and lng-term (by 2020). A review f the 56 Call 2011 prjects CPI data and methdlgies used t estimate the data was undertaken as part f this prject. The results f the review fed int the develpment f the new indicatr system and guidance dcument. The aim f the review was t separate the methds used by prjects t calculate their CPIs int three categries: Reliable; Acceptable; Uncertain T d this each f the prject's calculatins was reviewed frm the perspective f the 4 steps utlined in ur guidance dcument. Step 1: Define impacts and scpe Step 2: Set baseline Step 3: Estimate shrt-term impacts Step 4: Prject frward t lng-term (2020) impacts 4

7 Which methd each prject had used was als determined, in relatin t the suggested methdlgies in the guidance dcument. Methd 1 invlves aggregating the savings attributed t each individual actin, e.g. the savings per actin plan r per trained persn; this being fundamentally a bttm-up apprach. Methd 2 invlves estimating verall savings against a baseline, e.g. using the percentage savings achieved by a pilt prject: this being fundamentally an empirical r bservatinal apprach. Each step was given a subjective scre f between 0 and 5 where: Scre f 0: N infrmatin is prvided n a data surce r basis f assumptin, r an assumptin seems cmpletely illgical Scre f 5: Data and assumptins are fully dcumented; assumptins seem very lgical; seems likely t lead t a highly accurate estimate f impacts. This was dne fr the first three CPIs tgether (RE deplyed, energy saved, GHG) and separately fr the furth CPI (investment). The reasning being that the same basic methd is typically used t estimate either RE deplyed r energy savings and then the GHG impacts are calculated frm there. The scres were then weighted fr each step as fllws: Step 1 Step 2 Step 3 Step 4 Shrt term CPIs Methd 1 30% N/A 70% Shrt term CPIs Methd 2 30% 30% 40% Lng term CPIs Methd 1 20% N/A 50% 30% Lng term CPIs Methd 2 20% 20% 30% 30% A prject achieving a weighted scre f >3.5 was deemed Reliable; a scre between f 2 and 3.5 Acceptable and a scre f <2 Uncertain. Any prjects that scred 0 r 1 n step 1, 2 r 3 were autmatically mved t the Uncertain categry even if they scred highly elsewhere. Any prjects that scred 0 r 1 n step 4 nly were deemed Uncertain fr the lng-term (2020) CPIs nly. All f the prjects in the 2011 call were fund t have made a gd attempt at develping their CPIs and the methdlgies were well described in the CPI dcuments. This was nt always the case fr 2009 and 2010 call prjects, which suggests that the prject crdinatrs have benefited frm the training they have received frm the EACI, the supprt f prject fficers and a grwing familiarity with the cncept f CPIs and their estimatin. Hwever, there remains an incnsistency in apprach, particularly t the frward prjectin f 2020 impacts, which cnfirms the need fr further guidance and supprt. This need was further cnfirmed by the interviews with prject crdinatrs, mst f whm said that the calculatin f CPIs was very challenging and they wuld have liked mre detailed guidance and examples. The review f CPI methdlgies helped t infrm ur thinking n the selectin and definitin f a new set f Prject Perfrmance Indicatrs (PPIs, see fllwing sectin), the cntents f the guidance dcument and the way in which PPIs culd be aggregated and reprted as IEE prgramme impacts. Specifically: Guidance n the attributin gap. Further guidance is needed n hw t bridge the attributin gap fr thse prjects chsing t adpt the Methd 2 methdlgy. The attributin f savings was the key uncertainty in several f the prject s CPI calculatins. In general the Methd 1 appraches were mre likely t rate Reliable r Acceptable but even then there were sme key assumptins made and nt always justified r referenced. A 2-stage apprach. Many f the prjects prvided mre infrmatin n their CPI calculatin methdlgy in the CPI dcument than the prpsal. This suggests that they were shrt f time when preparing the prpsal and dedicated much mre time nce they were in cntract. This supprts the recmmendatin fr a tw-stage prcess as regards the calculatin f the perfrmance indicatrs. It may be beneficial t prduce a cut-dwn versin f the guidance fr use at the bid stage while als allwing applicants t access the larger guidance dcument shuld they wish t. Assessment f CPIs. Based n the evaluatrs cmments, abut half f the 2011 prjects were asked t review and clarify ne r mre f their CPI values. Furthermre, a mre in depth analysis f the CPIs in the first mnths f the prject, resulted in further refinement f the values. This suggests that the EACI shuld nt put t much emphasis n the indicatr values at the prpsal stage r use them as criteria fr evaluating prpsals. 5

8 Tp-dwn methds. The tp-dwn apprach shuld nly be used as a methd fr checking the impacts derived frm bttm-up analysis using methds 1 r 2; the results are t unreliable. Mandatry CPIs. Fr sme EU-wide plicy and market transfrmatin prjects, it may nt be pssible t quantify the impacts except thrugh these unreliable tp-dwn methds and s it is recmmended that the existing CPIs are nt mandatry fr such prjects in future. This des nt apply t plicy prjects fcused n lcal r city-level plicies r mre narrwly defined market transfrmatin prjects. Decimal places and cmmas. It may be easier t rule ut all use f decimal places and cmmas t avid cnfusin. The prblem is highlighted particularly when Excel has been used t calculate a figure that has then been transpsed int a Wrd dcument. Aggregatin. Submissin f CPI data shuld be via a spreadsheet t avid the lengthy exercise f pulling all data frm wrd dcuments. Prpsed new perfrmance indicatr system and guidance The initial guidance dcument was drafted in mnth 6 f the prject. The dcument has then been thrugh 4 iteratins, based n feedback frm the EACI staff, further internal wrkshps, and the review f 2011 CPI methdlgies. The final guidance dcument is designed t be the full versin f the guidance, which prvides infrmatin and supprt fr every step f the new perfrmance indicatr system. Althugh a tw phase prcess has been develped with the EACI, the 1 st phase guidance will be further develped by the EACI, which will utilise elements f the full guidance dcument. The guidance dcument includes the necessary steps fr: Selecting typical utputs which are relevant t a specific prject, Selecting perfrmance indicatrs which are relevant t a specific prject, Adapting them t the bjectives f that prject. The guidance dcument is based arund the results chain f bjectives activities - utputs - impacts (shrt-term) impacts (lng-term), and each stage f the indicatr-setting prcess is clearly explained using examples. A template t be cmpleted by applicants which will capture the chain shwn belw has als been develped. 6

9 Applicants are instructed t set n mre than 5 bjectives, in rder t keep them t a manageable and meaningful number, as there will be utputs and impacts related t each bjective. Objectives shuld be a mixture f shrt and lng-term. At the pint f setting their bjectives, applicants are als asked t identify which f the five fields f delivery their prject is assciated with: Enabling plicy Market transfrmatin Preparing the grund fr Investment Building capacities and skills Changing behaviur It is pssible fr prjects t address mre than ne field f delivery, but it is recmmended that the maximum is three. The reasn fr this is that prjects need t be fcussed in rder t achieve their aims, and as each field f delivery is quite different, prjects may be attempting t much if they try t address mre than three fields f delivery. Outputs are then determined by the bjectives and the activities t be delivered thrugh the prpsed wrk packages. An Annex t the guidance dcument has been prvided cntaining tables f standard utputs in rder t guide applicants as t the chices they have in expressing quantity indicatrs and quality indicatrs. Examples f basic quantitative utput indicatrs include: Number f participants at wrkshps Number f training hurs prvided Number f technical design slutins Hwever, ttal numbers ften d nt prvide the full picture f an utput. Examples f indicatrs that measure interest r effectiveness include: % f wrkshp participants that rated the wrkshp useful r very useful % f trainees that rated the training useful r very useful % f wrkshp participants that wuld recmmend the wrkshp t a clleague The ptins faced fr utput indicatrs were whether t prvide a very prescriptive set f indicatrs fr applicants t use vs. a high degree f flexibility supprted by detailed guidance. The advantages f prviding a standard set f indicatrs include; an increase in cnsistency between prjects, imprved identificatin f indicatrs fr applicants especially where they have little time/budget t cmplete the indicatr sectin f the applicatin, and the increased ease f aggregatin f utputs t prgramme level. The disadvantages f this apprach include; missing utput ptins due t the wide range f IEE prject types, and encuraging a bx-ticking mentality. It was decided t pt fr standard utput tables that are t be used as guidance, and nt be mandatry. This allws applicants t review their utputs, be guided n the best way t frame thse utputs, but still be free t express the innvatin in their prjects. The guidance dcument then fcuses n a new prpsed set f impact indicatrs, the new Prgramme Perfrmance Indicatrs (PPIs) and methdlgies t calculate the impacts f prjects mre reliably. In lking at prescriptive sets f prject impact indicatrs, it was decided that althugh this wuld imprve cnsistency between prjects, the very nature f these prjects mean that they all have very different impacts. The idea f IEE is t effect a range f impacts, with the gal that these prject-specific impacts then g n t be measureable thrugh energy efficiency, renewable energy generatin and therefre reductins in GHG emissins. Feedback frm the prject crdinatrs nt nly indicated that they wanted t see guidance n the methdlgy they shuld use fr calculating lng term impacts, but that they ften struggled t find a sensible replicatin factr t use. In reviewing the 35 initial prjects, and the 56 Call 2011 prjects, many similar prjects were fund t have vastly different estimates fr their 2020 impacts, due t the assumptins made abut replicatin. The new indicatr system asks applicants t estimate a minimum and maximum range f impact, which shuld help imprve cmparability between direct prject impacts by External experts cntracted t evaluate the submitted prpsals can als then clearly identify the replicatin assumptins and review their validity. 7

10 In reviewing the existing CPIs f the current system, a number f additinal indicatrs were cnsidered that may be relevant t a prprtin f prjects, and that wuld allw prjects that find the existing CPIs very difficult, t be able t use ther metrics t measure their impact at a prgramme level. A brad range f new PPIs were cnsidered, included green jbs created, SMEs influenced, innvatin, increased skills/capability, plicy makers influenced, attitudes and behaviur changes. The final set f seven PPIs is presented belw, alng with the units t be used. PPI Shrt term (by end f prject) Unit Lng term (by 2020) Unit Additinal cumulative investment made by Eurpean stakehlders in sustainable energy Additinal cumulative Renewable Energy prductin triggered Primary energy savings cmpared t baseline prjectins Reductin in greenhuse gas emissins cmpared t baseline prjectins tnnes f il equivalent (te)/year te/year tco 2e/year te per year AND cumulative t 2020 te/year AND cumulative t 2020 tco 2e/year AND cumulative t 2020 Plicy makers influenced ttal cumulative Number f peple with increased skills/capability/cmpetencies ttal cumulative Number f peple changing their behaviur ttal cumulative In the current system, applicants are asked t estimate cumulative investment impacts and in-year RE, energy efficiency, and GHG reductin impacts. The new system asks fr bth in-year and cumulative estimates, which will take accunt f the different impact prfiles fr different types f prject. We recgnised that PPI estimates will always be apprximate, particularly fr 2020 impacts, and easier t estimate fr sme prjects than thers. The purpse f the guidance dcument is t help applicants t estimate the likely impacts f their prjects, during the prject duratin and by 2020, in a lgical and sensible way that is apprpriate t their prject. T make the PPIs mre relevant t each prject, a system f mandatry and ptinal chices has been prpsed: 8

11 Field f delivery Investment RE triggered Energy saved GHG reduced Plicy maker influence (all levels) N. f peple with increased capability N. f peple changing their behaviur Enabling plicy O O O O Market transfrmatin Preparing the grund fr investments Building capacities and skills Changing behaviur Mandatry PPI O O O O O O O O O O O O O Optinal PPI The prpsed guide then ges n t describe a 5-stage methdlgy fr estimating PPIs. The methdlgy was tested during the analysis f the Call 2011 CPI data, where it was fund that mst prjects had used the bttm-up appraches described in the prpsed methdlgy. A key recmmendatin that has been made is that prjects use bttm-up appraches wherever pssible, and supprt their cnclusins by using tp-dwn methds t verify the estimatins. STEP 1: DEFINE IMPACTS AND SCOPE STEP 2: SET BASELINE STEP 3: SHORT TERM IMPACTS STEP 4: LONG TERM IMPACTS STEP 5: CHECK IMPACTS 9

12 Cnclusins and recmmendatins The key verall cnclusins and recmmendatins that can be made frm this prject are: 1. IEE is a unique prgramme There are n ther prgrammes delivering the same breadth and mixture f prjects as IEE. This makes it uniquely difficult t assess the impacts f the prgramme using nly a small number f cmparable metrics. The key metrics used in the current system are highly imprtant, and are cmplemented by the new PPIs, which allw measurement f impact fr thse prjects that particularly struggle t measure energy, carbn, r investment impacts in a reliable manner. The PPI data is vital t the prgramme, fr understanding the prgrammes impact, and t justify its existence. It is recmmended that the new set f PPIs is released with the next Call fr IEE prpsals, and supprted by the new guidance dcument. 2. It takes time and effrt t calculate reliable impact indicatrs This means that estimatins f the PPIs at the applicatin stage shuld nt be used directly in the evaluatin prcess; althugh it is reasnable t take int accunt hw well the applicant has thught thrugh their prpsed indicatrs. A new wrk package/task fr indicatr develpment, mnitring, and evaluatin (including ex-pst evaluatin) culd be intrduced fr prjects t fully study their impacts. Althugh the guidance dcument prduced fr this prject is designed t be a full guide, able t supprt applicants in cmpleting each step f their perfrmance indicatrs, it was felt that the effrt required t reliably calculate impacts, especially lng term impacts, wuld be a heavy burden at the applicatin phase. The review f Call 2011 CPIs shwed that it was necessary fr mst f the prjects t review and clarify their CPI estimatins during cntract negtiatin and in the first mnths f the actin. A tw phase apprach is sensible, as it allws applicants t fcus n determining the mst apprpriate indicatrs fr their prject at the applicatin stage, and then utilises a dedicated wrk package/task at the start f the prject t study the impacts f the prject and mnitr them in greater depth. It is imprtant t nte that many prjects have made a very reasnable attempt at estimating the CPIs, given a lack f directin fr their methdlgical appraches. Prject Officers at the EACI have als been invaluable in guiding prjects t address their prpsed CPIs during the negtiatin phase. The aim f the guidance dcument sectin n methdlgies is nt nly t imprve these estimatins, but t help prjects t fully dcument their assumptins and data surces s that their estimatins can be understd by the EACI, and help mre PPIs t be rated as reliable. This is especially imprtant where prjects have a multi-natinal impact. 3. It is imprtant t recgnise that sme prjects will nt be able t reliably calculate their energy r investment impact There will be certain prjects, especially pan-eurpean plicy prjects fr which it is particularly challenging t reliably estimate any impacts in the frm f energy r CO2 savings, RES capacity r investments. The best effrts that can be made will ultimately always result in a rating f uncertain. Hwever, these plicy prjects ften have the largest predicted impacts if they influence plicy in the intended manner. This is an acceptable situatin, as indicatrs shuld be apprpriate t the prject aims and bjectives and nt be gverned by hw measureable they are. This is why it is prpsed t intrduce a new indicatr n 'plicy maker influence'. 4. Prject mnitring and evaluatin are key t develping ur knwledge f impact A number f utputs frm this prject are designed t aid in the mnitring and evaluatin f the utputs and impacts f IEE prjects. An Excel template culd be used fr simpler data-gathering and can be used as an aggregatin and tracking tl. The recmmended wrk package/task n indicatr develpment, mnitring and evaluatin will als prvide prject crdinatrs with a clear message that they are t spend time and effrt nt nly in develping their estimatins f their impact at the start f the prject, but then t als mnitr their prgress against their indicatrs, and t evaluate their success at prject cmpletin. If feasible, it is recmmended that a database system is used t enter the indicatrs, s that analysis f aggregated utputs and separatin f impacts by field f delivery r key actin is relatively simple t achieve. 10