Frequently Asked Questions for UNDAF Evaluations

Transcription

1 Guidance Dcument Frequently Asked Questins fr UNDAF Evaluatins Prepared in cllabratin with UNDOCO, these Frequently Asked Questins prvide answers t questins regarding varius aspects related t UNDAF evaluatin. UNEG/G(2011)1

2 BACKGROUND 1. The January 2010 Guidelines fr United Natins Cuntry Teams (UNCT) n Hw t Prepare a United Natins Develpment Assistance Framewrk (UNDAF) 1 include the requirement f evaluatin. This Frequently Asked Questins (FAQ) dcument, develped by the United Natins Evaluatin Grup (UNEG) in cllabratin with United Natins Develpment Operatins Crdinatin (UNDOCO), aims t supprt the fulfilment f this requirement by prviding answers t questins that are ften received regarding varius aspects related t UNDAF evaluatin. It respnds t several reslutins by the United Natins General Assembly in the Triennial Cmprehensive Plicy Review (TCPR) f Operatinal Activities f the UN develpment system 2 :..the TCPR Recgnizes the need t ptimize the link f evaluatin t perfrmance in the achievement f develpment gals and encurages the United Natins develpment system t strengthen its evaluatin activities, with particular fcus n develpment results, including thrugh the effective use f the results matrix f the United Natins Develpment Assistance Framewrk, the systematic use f mnitring and evaluatin appraches at the system-wide level and the prmtin f cllabrative appraches t evaluatin, including jint evaluatins (#132); Requests the United Natins develpment system t further develp guidance and versight mechanisms fr the funding, planning and implementatin f the mnitring and evaluatin f United Natins Develpment Assistance Framewrks with a view t assessing their cntributin t natinal develpment and the achievement f the internatinally agreed develpment gals, including the Millennium Develpment Gals (#136). 2. This dcument is rganized in 3 sectins. Sectin 1 prvides answers t questins related t UNDAF evaluatin in general. Sectin 2 fcuses n questins related t the planning and management (r implementatin) f an UNDAF evaluatin and Sectin 3 relates t the use f UNDAF evaluatins. This FAQ dcument cmplements the cntents f the 2010 UNDAF Guidance and the 2006 UNDAF Evaluatin Terms f Reference (TR) that is underging revisin. 1 UNDOCO (January 2010), Hw t Prepare an UNDAF - Guidelines fr UN Cuntry Teams, UNDOCO, New Yrk. Available at: 2 United Natins, Triennial Cmprehensive Plicy Review f Operatinal Activities f the United Natins Develpment System, General Assembly Reslutin A/RES 62/208, Frequently Asked Questins fr UNDAF Evaluatins

3 SECTION 1: GENERAL QUESTIONS 1. Why cnduct an UNDAF evaluatin? What are the benefits? 3. Evaluatin is a systematic assessment which answers the questins Are we ding the right thing? Are we ding it the right way? and Are there better ways f achieving results? Evaluatin is used fr imprving accuntability and fr learning what has wrked, what has nt and why. By answering the abve questins, UNDAF evaluatins can prvide imprtant infrmatin fr strengthening prgramming and results at the cuntry level, specifically infrming the planning and decisin-making fr the next UNDAF prgramme cycle and fr imprving United Natins (UN) crdinatin at the cuntry level. The UNCT, hst gvernment and ther UNDAF stakehlders can learn frm the prcess f dcumenting gd practices which can then be used fr the benefit f ther cuntries. 4. By bjectively verifying results achieved within the framewrk f the UNDAF, the evaluatin enables the varius stakehlders in the UNDAF prcess, including natinal gvernments and dnrs, t hld the UNCT and ther parties accuntable fr fulfilling their cmmitments. This is seen as crucial given their brad-based scpe and the large resurces invlved in mst UNDAFs. In recgnitin f these benefits, the UNDAF guidance makes evaluatin fr each UNDAF mandatry. 2. What is the relatinship between an UNDAF evaluatin, annual review and prgress reprt? 5. UNCTs are required t undertake an annual review f the UNDAF and an UNDAF prgress reprt at least nce per cycle, althugh the UNCT may chse t undertake mre than ne UNDAF prgress reprt shuld it deem beneficial. The findings f the UNDAF annual reviews feed int the UNDAF prgress reprt. Bth f these reviews are self- assessments and, as such, are part f the mnitring exercise f the UNDAF. They are different frm the UNDAF evaluatin in terms f scpe, methdlgy and cntributin t accuntability. 6. An UNDAF evaluatin is an bjective and independent exercise as ppsed t an annual review, which is a self-assessment exercise. 3 By independently assessing UNCT perfrmance, the UNDAF evaluatin therefre cntributes t ensuring the accuntability f the UNCT t varius stakehlders, including natinal gvernments (fr standard definitin f independence, please refer t UNEG Nrms and Standards fr Evaluatin in the UN System). The UNDAF evaluatin will g beynd lking at cntributin t the natinal utcmes set ut in the UNDAF results framewrk t examine the strategic 3 Mnitring tracks prgress twards the results agreed in the results matrix and checks if the assumptins made and risks identified at the design stage are still valid r need t be reviewed. It helps the UNCT and implementing partners make mid-curse crrectins as an integral part f prgramme management. Evaluatin determines whether results made a wrthwhile cntributin t natinal develpment pririties and t the cherence f UNCT supprt. While it makes an essential cntributin t managing fr results, it is an external functin that is separated frm prgramme management. Fr further infrmatin n what is expected in bth mnitring and evaluatin, including the annual review and prgress reprt, see pages in Hw t Prepare an UNDAF (Part I). Frequently Asked Questins fr UNDAF Evaluatins 3

4 psitining f the UNCT in a cuntry. The findings frm annual reviews and the prgress reprt shuld feed int the UNDAF evaluatin prcess and prvide imprtant infrmatin that will be validated (r nt) during the evaluatin. Infrmatin n cnducting annual reviews and prgress reprts is available in the Standard Operatinal Frmat and Guidance fr Reprting Prgress n the UNDAF. 3. Is it pssible t integrate an UNDAF evaluatin within the brader framewrk f a gvernment led evaluatin (Natinal Develpment Plan (NDP), Pverty Reductin Strategy Paper [PRSP])? 7. An UNDAF evaluatin shuld usually be a stand-alne exercise, but it may be pssible t integrate it int a brader evaluatin framewrk. This is mre likely when UNDAF results are identical t the brader natinal planning framewrk, such as thse related t NDP r PRSP. Full integratin may nt be pssible since the UN cntributin is ften smaller in the cntext f a brad natinal develpment strategy. If the brader evaluatin lks at changes in natinal level utcmes then it may be feasible t assess hw the internatinal cmmunity cntributed t the changes and identify the specific rle f the UNDAF. In situatins where the UNDAF evaluatin is cnsidered as part f a brader evaluatin, there is a need t ensure that the scpe f the evaluatin is guided by the recmmended scpe f UNDAF TRs; the prcess is participatry; independence, bjectivity and ther aspects f quality assurance are maintained; and the UNDAF evaluatin findings and recmmendatins feed int the subsequent cycle. 4. Are UNDAF evaluatins fr Delivering as One (DaO) cuntries different in scpe (i.e., inclusin f an assessment f cmmn services and business peratins) r prcess? 8. The underlying assumptin f the DaO Pilt Initiative is that the implementatin f the Fur Ones (1- ne prgramme, 2- ne leader, 3- ne budgetary framewrk and ne fund; and 4- ne management system and cmmn services) will lead t increased effectiveness and efficiency f the UN at cuntry level, reductin f transactin csts fr the hst gvernments and UN agencies, and will ensure that prgramme cuntries benefit frm the cllective expertise and technical assistance f the entire UN system irrespective f the presence f UN agencies in the cuntry. The UNDAF evaluatin TRs shuld identify the specific factrs t be examined in the cntext f DaO cuntries. Generally, the fcus f bth types f evaluatin is n assessing results, prcesses and in UNDAF s cntributin t natinal develpment results. Bth evaluatins shuld examine the factrs that have affected the UNCT's cntributin, which may include the existence f cmmn services, harmnized business peratins r ther agreed features f DaO cuntries. Examples f and experiences in DaO evaluatins can be viewed n the UNDG website in the DaO sectin fr Evaluatin. 5. What is the timing f an UNDAF evaluatin as part f the UNDAF cycle? 9. The evaluatin shuld ideally be made available at the start f the prcess f preparing the new UNDAF and befre the develpment f the UNDAF results matrix. This implies that an UNDAF evaluatin wuld nly be able t cver three years f UNDAF implementatin. Hwever, there may be cases, fr example, when there is cntinuity between tw UNDAFs; this ccurs when the scpe f the 4 Frequently Asked Questins fr UNDAF Evaluatins

5 evaluatin cvers certain aspects f the preceding UNDAF cycle that wuld require examining three t five years r mre f perfrmance. In any event, the planning f an evaluatin shuld be integrated fully int the radmap fr UNDAF preparatin. 6. What is the link between agency specific evaluatins and the UNDAF evaluatin? What is the link between evaluatin f jint prjects/ prgrammes and the UNDAF evaluatin? 10. Generally, gd quality agency-specific evaluatins, especially prgramme/prject evaluatins, shuld feed int the UNDAF evaluatin as a surce f evidence. Fr larger evaluatins, such as cuntry prgramme evaluatins, there are tw ptential prblems t simply feeding int the UNDAF evaluatin: (a) in relatin t timing: it may be very early t d a cuntry prgramme evaluatin if it has t be undertaken befre an UNDAF evaluatin, i.e. it wuld cver less than three years; (b) there is the risk f excessive burden n gvernment, i.e., ne evaluatin team asks ne set f questins fr the agency evaluatin and anther asks a smewhat similar set f questins t the same infrmants fr the UNDAF evaluatin. As part f UNDAF planning, it is imprtant t ensure that agency specific evaluatin plans take int accunt the UNDAF evaluatin plan and vice versa. It is imprtant t ensure that the scpe f the UNDAF evaluatin is distinct, fcusing at the strategic level, and that duplicatin is avided. 11. Similarly, evaluatin f jint prgrammes shuld feed int the UNDAF evaluatin r, alternatively, evaluatin f jint prgrammes can be planned at the same time as the UNDAF evaluatin, with duplicatin f effrt minimised. UNCTs shuld encurage jint prgramme/prject evaluatins as imprtant surces f infrmatin fr the UNDAF evaluatins and in their wn right. 7. What are the rles and respnsibilities in the cnduct f UNDAF evaluatins (UN DOCO, UNEG, Agency specific evaluatin units, UNCT, evaluatin team, natinal gvernments, dnrs and ther stakehlders)? The UNCT is respnsible fr rganizing the evaluatin and managing the evaluatin team wh will cnduct the evaluatin. This is generally dne by establishing an Evaluatin Steering Cmmittee (SC) and an Evaluatin Management Grup (EMG). Natinal gvernments jintly cmmissin the UNDAF evaluatin with the UNCT and shuld participate in the EMG. They will als be asked t facilitate the flw f data t the evaluatin team, arrange meetings and prvide ther lgistical supprt as apprpriate. UNEG cllabrates with UNDOCO t prvide guidance (including the UNEG Nrms and Standards fr Evaluatin in the UN System) and maintains a rster f cnsultants. In particular, thse with expertise in UNDAF evaluatin. Agency-specific evaluatin units at respective UN agency headquarters wrk thrugh UNEG. At the cuntry level, agency-specific mnitring and evaluatin (M&E) units can be called upn by the Resident Crdinatr s Office (RCO)/EMG t play a technical rle in supprting the UNDAF evaluatin. Frequently Asked Questins fr UNDAF Evaluatins 5

6 Internatinal develpment partners and ther stakehlders, such as natinal civil sciety rganizatins, may participate in the evaluatin SC r EMG as apprpriate. The RCO and the EMG may engage the reginal United Natins Develpment Grup (UNDG) Teams frm the utset in planning the UNDAF evaluatin. In a fully evlved reginal set up, the Reginal UNDG Advisr, in full cnsultatin and crdinatin with the reginal UNDG Team and the regin-specific Peer Supprt Grup, advises and supprts the planning, develpment and implementatin UNDAF evaluatins upn request. 12. All f the abve actrs are als users f the evaluatin. Additinally, natinal gvernments and internatinal develpment partners will be infrmants during the evaluatin. 8. What are useful links t reference site n methdlgical guidance? 13. Tw infrmatin surces fr reference shuld be used by UNCTs in develping and implementing UNDAF evaluatins: a. The UNDG website includes: The UNDAF guidance package and Examples f UNDAF evaluatins b. UNEG brings tgether the prfessinal units and individuals respnsible fr evaluatin within the UN system. UNEG s missin is t prmte the independence, credibility and utility f the evaluatin functin and evaluatin acrss the UN system, t prmte its visibility and advcate n the imprtance f evaluatin fr learning, decisin making and accuntability. In 2005, UNEG agreed n a set f Nrms and Standards t guide evaluatin in the UN system. These and ther useful dcuments can be fund n the UNEG website r by clicking n the links belw. Standards fr Evaluatin in the UN System Nrms fr Evaluatin in the UN System Gd Practice Guidelines fr Fllw up t Evaluatins Quality Checklist fr Evaluatin Reprts UNEG Ethical Guidelines UNEG Cde f Cnduct fr Evaluatin in the UN system UNEG Quality Checklist fr Terms f Reference and Inceptin Reprts UNEG Handbk/Guidance n Integrating Human Rights and Gender Equality in Evaluatins in the UN System (frthcming). 6 Frequently Asked Questins fr UNDAF Evaluatins

7 Sectin 2: Planning and Managing the UNDAF Evaluatin 1. What are the phases f cnducting an UNDAF evaluatin? 14. The three key phases f cnducting an UNDAF evaluatin are illustrated in the fllwing table: 4 Phase 1: Planning, preparatin Phase 2: Cnducting the evaluatin Phase 3: Using the results 1. Review the evaluability r readiness fr evaluatin 2. Agree n the management structure f an evaluatin, and rles and respnsibilities 3. Draft the Terms f Reference (TR), including wrk plan 4. Organize the relevant dcumentatin 1. Brief and supprt the evaluatin team 2. Review the inceptin reprt prepared by the evaluatin team 3. Evaluatin team cnducts data cllectin and analysis 4. Review the draft evaluatin reprt / validate findings by stakehlders 1. Prepare the management respnse and implement the evaluatin recmmendatins, as apprpriate 2. Prepare and disseminate evaluatin prducts and rganize knwledge sharing events 3. Use results - Review evaluatins prir t designing the next UNDAF 5. Select the evaluatin team 5. Finalisatin and presentatin f reprt by the evaluatin team 4 Sme f the terms indicated in the table belw (i. e., management respnse) are explained in detail in later questins. Frequently Asked Questins fr UNDAF Evaluatins 7

8 2. What is the recmmended management structure fr UNDAF evaluatins (rles and respnsibilities)? 15. UNDAF evaluatins are jintly cmmissined by the UNCT and the natinal gvernment, and bth shuld agree at the utset n the management structure fr the evaluatin, as well as establish clear rles and respnsibilities. 16. It is recmmended that a SC be cnstituted by the cmmissiners that are vested with decisinmaking pwer in regards t the evaluatin. The SC wuld make apprval decisins fr the TR, final selectin f the evaluatin task manager and evaluatin team and the final reprt. The SC, in turn, shuld cnstitute an E M G with the key rles f: 1) quality assurance in the evaluatin prcess t ensure that it meets UNEG Nrms, Standards and Ethical Guidelines; and 2) disseminating the findings f the evaluatin (e.g., thrugh Reginal UNDG Teams and reginal Peer Supprt Grups, etc.). Recmmended Management Structure fr UNDAF Evaluatin Evaluatin Cmmissiners UNCT & Nat'l Gv Steering Cmmittee Selected UNCT members Natinal Cunterparts Stakehlders (CSOs, Dnrs, etc.) Evaluatin Management Grup (EMG) Selected UNCT members & Resident Crdinatr's Office Natinal Cunterparts Evaluatin Task Manager UNDAF Evaluatin Team 17. The EMG has the key rle f keeping the SC apprised f the prgress f the evaluatin, and must bring any pertinent issues t the SC fr their decisin/apprval. The EMG shuld appint r recruit an UNDA F Evaluatin Task Manager wh will be directly respnsible fr the day-t-day implementatin 8 Frequently Asked Questins fr UNDAF Evaluatins

9 f the evaluatin, including drafting f the TR and the recruitment and direct management f the evaluatin team. The Task Manager will reprt directly t the EMG and will play a key rle in liaising between the SC, EMG, stakehlders and the evaluatin team. The UNDA F Evaluatin Team is respnsible fr cnducting the evaluatin as per the TR in an independent and impartial basis fllwing the UNEG Cde f Cnduct. They will undertake all data cllectin and analysis and are respnsible fr prducing all deliverables in line with UNEG quality standards. 3. What are the ptential funding mechanisms f an UNDAF evaluatin? 18. The UNDAF evaluatin shuld be jintly funded by the evaluatin cmmissiners, the UNCT and the natinal gvernment t enhance natinal wnership f the prcess. Ideally, the funding arrangement fr the UNDAF evaluatin shuld be discussed, decided upn and allcated during the UNDAF planning stage when the UNDAF M&E Plan is being develped, r at least ne year in advance f when the UNDAF evaluatin is expected t cmmence. 19. The Cmmn Cuntry Assessment (CCA)/UNDAF Guidelines (2009) indicate that mst UNDAF evaluatins require at least a minimum f US$100,000 fr cnducting a high quality evaluatin prcess in line with UNEG Nrms, Standards and Ethical Guidelines. Hwever, the situatin will vary depending n the cuntry cntext and the scpe and cmplexity f the evaluatin. In mst cases, a budget range f US$100,000 - $500,000 is mre realistic and warranted t prduce a quality assessment that will be useful fr stakehlders. 20. A key determinant f the cst f an evaluatin is the extent t which natinal institutins/ cnsultants can undertake majr wrk related t the evaluatin and whether a majr survey is invlved, which culd help t lwer csts. Generally, the evaluatin is funded thrugh a cst-sharing agreement amng UNCT members, with the RCO als making a cntributin thrugh the UN Cuntry Crdinatin Fund. Other pssible funding ptins include direct cntributins frm multilateral r bilateral agencies and multi-dnr trust funds/millennium Develpment Gals Fund, if a percentage f the fund is set aside fr UN crdinatin purpses. In additin, there may be situatins where a gvernment may want t prvide partial funding t an UNDAF evaluatin. It is ften a gd idea t explre varius surces f funds t pl resurces fr cnducting an UNDAF evaluatin. 4. Hw t draft the TR in a participatry manner (stakehlder analysis)? 21. The UNDAF Evaluatin TR utlines the design f the evaluatin and serves t ensure that there is agreement amng evaluatin cmmissiners, stakehlders and the evaluatin team as t why the evaluatin is being cnducted, hw it will be used, what it will assess (and hw), and wh will be invlved in the prcess and hw. The TR defines the scpe, requirements and expectatins f the evaluatin and serves as a guide and pint f reference thrughut the evaluatin. 22. It is recmmended that the drafting f the TR begin with a stakehlder analysis that draws frm the UNDAF stakehlder analysis t identify the key persns with a vested interest in the evaluatin, hw they will ptentially use the evaluatin results, and in what ways they culd participate in the evaluatin (e.g., as infrmants, as members f EMG, etc.). The stakehlder analysis shuld ensure cverage f duty-bearers, rights hlders, wmen and ther grups subject t discriminatin. Frequently Asked Questins fr UNDAF Evaluatins 9

10 23. Mre specifically, it is recmmended that UNDAF TRs include: Purpse, audience and use f the evaluatin A clear justificatin fr the evaluatin shuld be prvided, hw it will be used and by whm. The UNDAF evaluatin generally prvides the UNCT and natinal gvernment with relevant infrmatin (lessns learned, pprtunities, areas fr strengthening, etc.) n UNDAF perfrmance that can infrm the subsequent UNDAF planning cycle. Please see questin 1 under General Questins. Objectives and scpe Each f which are t be clearly defined given the cntext, resurces and timeframe available, with indicatins f what will and what will nt be assessed and why. In general, UNDAF evaluatins assess the rle and relevance f the UNDAF, its design and fcus, the achieved results, the prcesses that have led t results r nn-achievement f results and the cllective cmparative advantage f the UN system in cuntry. Given UN mandates n human rights and gender equality and their inclusin as key prgramming principles fr UNDAF, a main bjective f the evaluatin shuld include the substantive assessment f the Human Rights-Based Apprach (HRBA), including gender equality, within its key bjectives as well as the ther prgramming principles: Results Based Management (RBM), envirnmental sustainability and capacity develpment. Special attentin shuld be paid t the mst vulnerable grups. Evaluatin cntext and subject Adequate cntextual infrmatin n the develpment and implementatin f the UNDAF in the perid t be assessed shuld be cvered, including descriptins f any human rights, gender r envirnmental analyses. Evaluatin apprach, criteria and questins Overall apprach fr the evaluatin shuld be as participatry as pssible and seek t be inclusive f the vices f vulnerable and marginalized grups. UNDAF evaluatins shuld assess the relevance, effectiveness, efficiency, sustainability and impact f the UNDAF, but culd als include additinal criteria as relevant. Fr example, humanitarian criteria including cherence, cnnectedness, etc. Specific, targeted and tailred evaluatin questins, framed arund selected evaluatin criteria, shuld address human rights, gender equality, capacity develpment and envirnmental sustainability issues and ensure assessment f results fr the mst vulnerable grups. Evaluatin methdlgy Specific data cllectin and analysis methds t be used and the verall methdlgy that will generate rigrus data and an assessment that is fair and unbiased shuld be used; use f mixed methds which cnsider bth quantitative and qualitative analysis is recmmended. Evaluatin management and stakehlder participatin Descriptin f the SC and the EMG and hw stakehlders (UNCT, natinal gvernment, individual UN agencies, civil sciety rganizatins, etc.) will participate thrughut the evaluatin prcess is needed. This includes respnsibility fr develping a management respnse t the evaluatin and fllw-up t the findings and recmmendatins. Evaluatin wrk plan Timeframe and timeline fr key evaluatin deliverables (including utline f final reprt), which shuld ensure that the final reprt will be 10 Frequently Asked Questins fr UNDAF Evaluatins

11 available befre the new UNDAF planning cycle cmmences, is necessary; the cnduct f UNDAF evaluatin is nrmally an 8 mnth t 1 year prcess (including the time devted fr develping the TR). Evaluatin budget range 5 See questin 2 abve under Planning. Qualificatins f the evaluatin team T include experience in cnducting multistakehlder cmplex evaluatins, participatry evaluatins and knwledge f human rights and gender equality, and previus experience with cuntry-level r UNDAF evaluatins. See questin 6 under Planning. Ethical cde f cnduct t gvern the evaluatin Helps t ensure that all parties carry ut the evaluatin in accrdance with primary values and ethical standards, e.g., UNEG Cde f Cnduct r ther relevant ethical cdes that pertain t wrking in the cntext, especially fr vulnerable r marginalized grups. 5. Hw t select the Evaluatin Team? Hw t ensure independence and cmpetence? 24. The chice f the evaluatrs is imprtant fr the quality f evaluatins. Evaluatrs shuld be selected thrugh a cmpetitive and transparent prcess in accrdance with the rules and regulatins f the United Natins pertaining t prcurement. 25. The UNEG Standards fr Evaluatin in the UN System prvide guidelines n the required cmpetencies fr all persns engaged in the cnduct f evaluatins. These include: Standard 2.1: Persns engaged in designing, cnducting and managing evaluatin activities shuld pssess cre evaluatin cmpetencies. Standard 2.2: Evaluatrs shuld have relevant educatinal backgrund, qualificatin and training in evaluatin. Standard 2.3: Evaluatrs shuld have prfessinal wrk experience relevant t evaluatin. Standard 2.4: Evaluatrs need t have specific technical knwledge f, and be familiar with, the methdlgy r apprach that will be needed fr the specific evaluatin t be undertaken, as well as certain managerial and persnal skills. 26. The UNEG Standards fr Evaluatin in the UN System, which can be accessed n the UNEG website, elabrates further n each f these standards if mre infrmatin is required. 5 UNEG Standards require that evaluatin TORs include the budget range fr the evaluatin; hwever, this may be in cnflict with UN rules and regulatins fr prcurement. Frequently Asked Questins fr UNDAF Evaluatins 11

12 27. T the extent pssible, evaluatin teams shuld be external and balanced by expertise, by technical knwledge and experience in the areas f fcus f the evaluatin, and by knwledge f the natinal situatin and cntext. The evaluatin team shuld als reflect a balanced gender and gegraphical cmpsitin, as apprpriate. 28. Typically, evaluatin teams are cmpsed f a team leader with significant evaluatin and leadership cmpetencies, wh guides the wrk and cntributins f the evaluatin team members, and wh may cntribute specific technical and/r cuntry-specific knwledge t the cnduct f the evaluatin. 29. In additin t the cmpetency f the evaluatrs, cnsideratins shuld be made t ensure the independence f the evaluatin exercise. At the time f selecting the evaluatin team members, the evaluatin manager shuld ensure that: T avid cnflict f interest and undue pressure, evaluatrs need t be independent, implying that members f an evaluatin team must nt have been directly respnsible fr the plicy-setting, design, r verall management f the subject f evaluatin, nr expect t be in the near future (UNEG Nrms fr Evaluatin in the UN System). 6. Hw t cnduct a stakehlder analysis and ensure stakehlder participatin in UNDAF evaluatin? 30. Mapping f key stakehlder institutins (gvernment cunterparts, UNCT members, Reginal UNDG Teams, bilateral agencies, partnering nn-gvernmental rganizatins (NGOs) and civil sciety institutins, private sectr, etc.) and individuals within thse institutins is an essential step in the stakehlder analysis exercise, accessible n the UNDG Tlkit website. 31. The rle f varius stakehlders in the UNDAF evaluatin prcess needs t be cmmunicated and agreed upn in advance. The rle f the stakehlders includes prviding inputs in shaping the scpe f the evaluatin, quality assurance during the implementatin phase, brad-based wnership f the evaluatin prcess and ensuring the use f the UNDAF evaluatin at varius levels. These rles can be specified fr varius sub-grups f stakehlders thrugh such mechanisms as the evaluatin SC and EMG, which als cntribute t ensuring evaluatin quality and use. The stakehlder analysis and rles and respnsibilities shuld be spelled ut and agreed upn in advance fr varius grups. 32. A useful apprach f engaging stakehlders thrughut the evaluatin is t plan wrkshps, virtual meetings (r ther pprtunities fr electrnic review) invlving selected stakehlders at key steps f the evaluatin prcess, namely n defining the evaluatin scpe, feedback n the inceptin reprt, feedback n draft reprt findings and recmmendatins and disseminatin/use f evaluatin findings and recmmendatins. 7. What are the necessary measures fr ensuring quality UNDAF evaluatin thrughut the evaluatin prcess? 33. Quality assurance f UNDAF evaluatin entails the use f quality standards at the level f evaluatin planning, implementatin, disseminatin and utilisatin. A number f guidance dcuments and tls fr quality assurance are available n the UNEG website. It is crucial t ensure that: 12 Frequently Asked Questins fr UNDAF Evaluatins

13 Evaluatin planning and TR develpment are participatry with inputs frm bth evaluatin experts and prgramme managers f different UN agencies. The TR shuld fllw the recmmended utline and guidance and be as cmplete as pssible in terms f the scpe and methds, as well as all aspects f evaluatin management. A well-qualified, experienced and balanced evaluatin team (gender, natinal versus internatinal experts) shuld be recruited. Since qualified cnsultants are ften bked in advance and recruitment can be a lengthy prcess, it is critical t allw sufficient lead time fr recruitment. A detailed inceptin reprt that utlines a cmplete evaluatin framewrk, methdlgy, data cllectin methds, wrk plan and rles and respnsibilities shuld be prepared and shared by the evaluatin team. The evaluatin SC reviews the draft and, if necessary, slicits feedback frm external UNDAF evaluatin experts t ensure that the inceptin reprt is cmplete and well frmulated. It is als useful t ensure that evaluatin is grunded n a review f relevant dcuments. The evaluatin team shuld be supprted at all levels in their data cllectin effrt as per the agreements reached during the inceptin phase. They are encuraged t triangulate infrmatin frm varius surces and t use a mix f quantitative and qualitative methds. A meeting invlving the evaluatin team and selected stakehlders shuld be rganized fr presentatin and feedback n draft findings and recmmendatins f the evaluatin, and is suggested t be dne prir t the finalisatin f the draft reprt. The draft evaluatin reprt shuld be reviewed by the SC and, if necessary, by evaluatin experts t ensure its cmpletin and sundness. The draft is t be shared with selected stakehlders fr triangulatin and t build wnership. The final reprt shuld fllw UNEG recmmended standards fr evaluatin reprting and includes a well frmulated executive summary. It is particularly imprtant t check that the findings are evidence-based and the recmmendatins are funded upn such findings and use clearly frmulated actin language. 34. A strng mnitring and evaluatin (M&E) system that supprts UNDAF implementatin is an underlying determinant f evaluatin quality. Quality assurance respnsibilities thrughut the UNDAF evaluatin prcess need t be specified and reflected with adequate time allcated in the verall planning. Frequently Asked Questins fr UNDAF Evaluatins 13

14 Sectin 3: Using UNDAF Evaluatin 1. Wh are the key users f the evaluatin? Use by UNCT and the gvernment in develping the new UNDAF: The evaluatin needs t be timed t ensure that it can feed int the prcess and the evaluatin reprt needs t be made available t all stakehlders invlved in the develpment f the new UNDAF (see Q4 n learning frm evaluatin). Use shuld als be ensured thrugh the mandatry management respnse system (see Q3 belw n management respnses). Use by ther UNCTs: UNDOCO disseminates evaluatin reprts widely s that ther UNCTs can learn frm an evaluatin (see Q5 belw n disseminatin). It will als synthesize lessns acrss UNDAFs t facilitate learning and use. Use by stakehlders: Gverning bdies f UN agencies, dnr rganizatins, as well as the gvernments and civil scieties in hst cuntries need t be infrmed abut UNDAF perfrmance and its cntributin t natinal develpment results; what has wrked well and nt s well, and s they can thereby influence UNDAF planning. 2. What is the prcess invlved in preparing a management respnse? 35. UNDAF evaluatin requires an explicit respnse by the UNCT, specifically, a respnse t each recmmendatin and a set f time-bund actins t address them. This will take the frm f a frmal management respnse prepared by the UNCT in cnsultatin with natinal gvernment cunterparts. In accrdance with established gd practice, the management respnse shuld be prepared within tw mnths f receiving the final versin f the UNDAF evaluatin reprt and cmpleted accrding t a standard UNDG management respnse template (being develped). The management respnse t the UNDAF evaluatin is, in turn, shared with the Reginal UNDG Team fr infrmatin purpses. Fr transparency and wider infrmatin and use, the management respnses t UNDAF evaluatins shuld be psted n the UNDOCO website. Detailed guidance n preparing a management respnse t UNDAF evaluatin will be available shrtly. 3. Hw t facilitate learning frm evaluatin? 36. The UNEG Gd Practice Guidelines fr Fllw-up t Evaluatins identify three precnditins fr effective fllw-up t evaluatin: First, invlvement f internal stakehlders and, t the extent pssible, relevant external stakehlders thrughut the evaluatin prcess increases the perceived relevance, and stakehlders' wnership, f evaluatins. It is imprtant t strike an apprpriate balance between prmting the wnership f evaluatin findings and recmmendatins withut cmprmising the independence f evaluatin. Secnd, the quality f evaluatin recmmendatins. The recmmendatins shuld lgically fllw frm findings and cnclusins, be based firmly n evidence and analysis, 14 Frequently Asked Questins fr UNDAF Evaluatins

15 and are t be clearly frmulated and presented in a manner that is easily understd by target audiences. Bth strategic and mre peratinal recmmendatins are expected t be able t be implemented. Third, the evaluatin's credibility is a factr affecting the utility f the evaluatin. Credibility, in turn, depends n independence, impartiality, transparency, quality and the apprpriateness f the methds used. Reprting lines and different structures f the evaluatin units are key factrs influencing the independence, credibility and, hence, the utility f evaluatins. 4. Hw t disseminate evaluatin reprts? 37. A key principle t guide the develpment f evaluatin disseminatin plan is that the reprt and findings shuld be made as accessible and barrier-free as pssible. All UNDAF evaluatins, tgether with management respnses, shuld be made public and upladed t the UNDG M&E webpage. Als, evaluatin reprt and management respnse shuld be upladed n UNCT website and can be disseminated thugh evaluatin and develpment netwrks, including thse invlving UNCT members. The UNDAF evaluatin reprt shuld be widely disseminated directly within the UNCT, t natinal stakehlders including gvernment, civil sciety rganizatins, the Reginal UN Team and internatinal develpment partners. The reprt shuld be translated int natinal languages if needed. Other prducts such as leaflets f evaluatin summary, e-reprts that are mre user-friendly and tailred fr different audiences can als be develped. Frequently Asked Questins fr UNDAF Evaluatins 15

16 ACRONYMS CCA DaO EMG FAQ M&E NDP NGOs PRSP RCO RBM SC TR TCPR UN UNCT UNDAF UNDG UNDOCO UNEG Cmmn Cuntry Assessment Delivering as One Evaluatin Management Grup Frequently Asked Questins Mnitring and Evaluatin Natinal Develpment Plan Nn-gvernmental Organizatins Pverty Reductin Strategy Paper Resident Crdinatr s Office Results Based Management Steering Cmmittee Terms f Reference Triennial Cmprehensive Plicy Review United Natins United Natins Cuntry Teams United Natins Develpment Assistance Framewrk United Natins Develpment Grup United Natins Develpment Operatins Crdinatin United Natins Evaluatin Grup 16 Frequently Asked Questins fr UNDAF Evaluatins