IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-10510) ON A GRANT IN THE AMOUNT OF US$ MILLION TO THE ISLAMIC REPUBLIC OF PAKISTAN FOR A

Transcription

1 Public Disclosure Authorized Document of The World Bank Report No: ICR Public Disclosure Authorized IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-10510) ON A GRANT IN THE AMOUNT OF US$ MILLION Public Disclosure Authorized TO THE ISLAMIC REPUBLIC OF PAKISTAN FOR A GOVERNANCE SUPPORT PROJECT FOR KP, FATA AND BALOCHISTAN December 22, 2016 Public Disclosure Authorized Governance Global Practice Public Sector and Institutions South Asia Region

2 CURRENCY EQUIVALENTS (Exchange Rate Effective December 10, 2016) Currency Unit = Pakistani Rupee (PKR) 1 PKR = US$ US$ 1.00 = PKR FISCAL YEAR July 1 June 30 ABBREVIATIONS AND ACRONYMS ACE Anti-Corruption Establishment ADP Annual Development Programme AF1 Additional Financing 1 AF2 Additional Financing 2 BDNA Balochistan Development Needs Assessment CPS Country Partnership Strategy CRC Citizen Report Card CSO Civil Society Organisation EPP Emergency Project Paper ESSAF Environment and Social Screening and Assesment Framework FATA Federally Administered Tribal Areas FCS Fragile and Conflict Affected Situations FCV Fragile, Conflict and Violence FHDS Fata Household Development Survey FM Financial Management GIT Governors Inspection Team GoP Government of Pakistan GRM Grievance Redress Mechanism GSP Governance Support Project ICRR Implementation Completion and Results Report ICT Information, Communication and Technology IO Intermediate Outcome ISR Implementation Status and Results Report ISUs Implementation Support Units KP Khyber Pakhtunkhwa M&E Monitoring and Evaluation MDTF Multi-Donor Trust Fund MIS Management Information System MTR Mid-Term Review N/A Not Applicable NGO Non Government Organisation O&M Operations and Maintenance

3 OPCS P&DD PCNA PDO PMS PPF PPP PPRA PRIMA PRAMS RF RRF RTI RTPS/RTS TA WDR Operations Policy and Country Services Planning and Development Department Post Crisis Needs Assessment Project Development Objective Performance Management System Project Preparation Facility Public Private Partnership Public Procurement Regulatory Authority Portfolio and Risk Management Procurement Risk Assessment and Management System Result Framework Rapid Response Facility Right to Information Right to Public Services Technical Assistance World Development Report Senior Global Practice Director: Practice Manager: Project Task Team Leader: ICR Task Team Leader: ICR Co-Team Leader: Deborah L. Wetzel Alexandre Arrobbio Sher Shah Khan Marieta Fall Simon Carl O Meally

4 PAKISTAN Governance Support Project CONTENTS A. Basic Information... i B. Key Dates... i C. Ratings Summary... i D. Sector and Theme Codes... ii E. Bank Staff... ii F. Results Framework Analysis... iii G. Ratings of Project Performance in ISRs... ix H. Restructuring (if any)... x I. Disbursement Profile... x 1. Project Context, Development Objectives and Design Context at Appraisal Original Project Development Objectives (PDO) and Key Indicators Revised PDO and Key Indicators, and Reasons/Justification Main Beneficiaries Original Components Revised Components Other significant changes Key Factors Affecting Implementation and Outcomes Project Preparation, Design and Quality at Entry Implementation Monitoring and Evaluation (M&E) Design, Implementation and Utilization Safeguard and Fiduciary Compliance Post-completion Operation/Next Phase Assessment of Outcomes Relevance of Objectives, Design and Implementation Achievement of Project Development Objectives Efficiency Justification of Overall Outcome Rating Overarching Themes, Other Outcomes and Impacts Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Bank Performance Borrower Performance... 16

5 6. Lessons Learned Comments on Issues Raised by Grantee/Implementing Agencies/Donors Annex 1. Project Cost, Financing, and Inputs Annex 2. Outputs by Component Annex 3. Economic and Financial Analysis Annex 4. Results Frameworks and Indicators Changes Annex 5. Detailed Assessment of Achievements by Outcome Indicator Annex 6. Supporting Program Background Information Annex 7. Grant Preparation and Implementation Support/Supervision Processes. 44 Annex 8. Beneficiary Survey Results Annex 9. Summary of Grantee's ICR Annex 10. Comments of Cofinanciers and Other Partners/Stakeholders Annex 11. List of Individuals Consulted and References... 52

6 A. Basic Information PAKISTAN Governance Support Project DATA SHEET Country: Pakistan Project Name: Project ID: P L/C/TF Number(s): TF ICR Date: 12/15/2016 ICR Type: Core ICR Lending Instrument: SIL Grantee: Original Total Commitment: Revised Amount: USD 12.75M Environmental Category: C Implementing Agencies: FATA Secretariat Government of Balochistan Government of Khyber Pakhtunkhwa KP/FATA Governance Reforms GOVERNMENT OF PAKISTAN USD 6.00M Disbursed Amount: USD 12.75M Cofinanciers and Other External Partners: Multi-Donor Trust Fund for Crisis Affected Areas of KP, FATA and Balochistan B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 05/23/2011 Effectiveness: 11/01/2011 Appraisal: Restructuring(s): 11/30/2015 Approval: 11/01/2011 Mid-term Review: 11/04/ /28/2013 Closing: 06/30/ /29/2016 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Grantee Performance: Moderately Satisfactory Substantial Moderately Satisfactory Moderately Satisfactory i

7 C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Moderately Satisfactory Overall Bank Overall Borrower Moderately Satisfactory Performance: Performance: Moderately Satisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: Yes No Satisfactory Quality at Entry (QEA): Quality of Supervision (QSA): None None Rating D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Other Public Administration Theme Code (as % of total Bank financing) Administrative and civil service reform Managing for development results Other accountability/anti-corruption Other public sector governance Public expenditure, financial management and procurement ii E. Bank Staff Positions At ICR At Approval Vice President: Annette Dixon Isabel M. Guerrero Country Director: Patchamuthu Illangovan Rachid Benmessaoud Practice Manager/Manager: Alexandre Arrobbio Joel Hellman Project Team Leader: Sher Shah Khan Naseer Ahmad Rana ICR Team Leader: ICR Primary Author: Marieta Fall Marieta Fall

8 F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To improve the delivery efficiency of the PCNA program through institutional strengthening and support of KP and FATA, while responding to priority needs of the PCNA governance program Revised Project Development Objectives (as approved by original approving authority) To strengthen the capacity of government departments in KP, FATA and Balochistan to help support efficient delivery of the PCNA program in KP and FATA and related development programs in Balochistan (a) PDO Indicator(s) Indicator Indicator 1 : Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years KP & FATA are able to manage and coordinate implementation of PCNA short and medium term interventions and design longer-term interventions (dropped at Additional Financing 2 AF2) Value quantitative or Qualitative) Not clear (capability assumed to be low relative to size of PCNA and with PCNA implementation only started in 2010) KP & FATA are able to manage and coordinate implementation of PCNA short and medium term interventions and design longer-term interventions KP & FATA are managing and coordinating implementation of PCNA short and medium term interventions and design longer-term interventions Date achieved 11/01/ /30/ /29/2016 Achieved. Main outcomes targeted under this sub-objective were reached: ISUs Comments operational and capacitated; Governments of KP and FATA are managing and (incl. % coordinating implementation of the PCNA. achievement) Indicator 2 : Value quantitative or Qualitative) Improved quality of services through implementation of urgent reforms (dropped at AF2) not clear Implementation of urgent reforms iii Evidence of efficiency, transparency and accountability gains which can contribute to quality of services. Date achieved 11/01/ /30/ /29/2016 Comments See related intermediate outcomes. Given PDO, ICRR focused on assessing improved (incl. % efficiency of services through implementation of priority reforms. achievement)

9 Indicator 3 : NEW (AF2) Average Processing Time for Public Services by FATA Tribunal (Days) Value quantitative or Qualitative) 721 days (5768 hrs) 93 days (750 hrs) 56 days (448 hrs) Date achieved 01/02/ /31/ /29/2016 Comments (incl. % achievement) Achieved. Although target exceeded per last ISR, baseline and target not provided in AF2 RF2. Baseline and targets values are thus per ISR. Indicator 4 : NEW (AF2) - Average processing time for public services (delivery of Domicile Certificate in KP) (Days) Value quantitative or Qualitative) Date achieved 10/11/ /30/ /29/2016 Comments (incl. % Achieved. Mandated processing time for this service is of 10 days. achievement) Indicator 5 : NEW (AF2) - Total number of transactions for the main public service targeted by the project (FATA Tribunal) (Number, per month, supplement to PDO Indicator 3) Value quantitative or Qualitative) Date achieved 01/02/ /31/ /29/2016 Comments (incl. % achievement) Indicator 6 : Achieved. Slightly exceeded. Discrepancy between baseline and target reported between AF2 RF2 and last two ISRs (the ISRs subsequent to AF2). Baselines and target values are reported per ISR. NEW (AF2) -Total number of transactions for the main public service targeted by the project (delivery of Domicile Certificate in KP) (Number, total, Supplement to PDO Indicator 4) Value quantitative or Qualitative) Date achieved 10/18/ /30/ /29/2016 Comments (incl. % achievement) Achieved. Although targets exceeded, other factors are contributing to volume of transactions. Baseline not provided in last ISR but Project's last Implementation Support Mission (ISM) reports 1791 requests in the first report provided by Right To Public Services (RTPS). Indicator 7 : NEW (AF2) -Number of grievances addressed by Balochistan Ombudsman (per month) Value quantitative or Qualitative) Date achieved 09/12/ /31/ /31/2016 iv

10 Comments (incl. % achievement) Indicator 8 : Achieved. Targets exceeded but no baseline provided in AF2 RF (with much lower monthly targets). NEW (AF2) -Number of grievances addressed by KP Ombudsman (per month) Value quantitative or Qualitative) Date achieved 10/11/ /31/ /30/2015 Comments (incl. % Achieved. Targets exceeded. achievement) Indicator 9 : NEW (AF2) -Number of grievances addressed by Peshawar High Court (per month) Value quantitative or Qualitative) 0* Date achieved 10/11/ /31/ /30/2015 Comments (incl. % achievement) Indicator 10 : Achieved. *Pre-project baseline (3224) did exist and is aggregate of grievances , not yearly or monthly baseline. NEW (AF2) -Proportion of funds directed to priority sectors increased based on citizen consultations in Balochistan. Value quantitative or Qualitative) 0 1% increase 1.73% increase Date achieved 09/11/ /31/ /29/2016 Comments (incl. % Achieved. Target exceeded with attribution of increase to citizen consultations. achievement) NEW (AF2) -Reduction of the number of days (average) from proposal to start of Indicator 11 : implementation of programs under the Rapid Response Facility. Value quantitative or Qualitative) Date achieved 10/11/ /31/ /29/2016 Comments (incl. % achievement) Indicator 12 : Partially achieved. Baseline and end data mask years in between (such as in 2013) where average lead time increased. End yearly average data (2015) is 42 days and only represents KP. Lead time target was 15 days maximum per Operations Manual. NEW (AF2) -Investments mobilized in targeted development sectors responding to the PCNA program in KP and FATA (US$ Billion) Value quantitative or Qualitative) v

11 Date achieved 10/11/ /31/ /10/2015 Comments (incl. % Achieved. Targets Exceeded. Other contributing factors at play such as donors commitment. achievement) (b) Intermediate Outcome Indicator(s) Indicator Indicator 1 : Value (quantitative or Qualitative) Baseline Value Original Target Values (from approval documents) Formally Revised Target Values PCNA ISUs in KP and FATA fulfill their mandate (dropped at AF2) n/a Yes PCNA ISU in KP, FATA and ISU Balochistan successfully fulfill their mandate YES Actual Value Achieved at Completion or Target Years Date achieved 11/01/ /30/ /30/ /29/2016 Achieved. PCNA ISUs in KP and FATA set up and operationalized; Comments (incl. % achievement) Coordination meetings with donors, Line Departments/Directorates, civil society and academia; 48 proposals developed for MDTF funding; More than a thousand government officials trained. Indicator 2 : Governance program endorsed by stakeholders (modified at AF 1 and AF2) Value (quantitative or Qualitative) No Yes Governance program for KP and FATA endorsed by stakeholders. Governance reform program in Balochistan submitt ed for approval by stakeholders. See final revised indicator (IO 13) Yes* Date achieved 11/01/ /30/ /30/ /29/2016 Partially achieved. Development and endorsement of 10-year Governance Programs Comments (incl. % achievement) (with action plans) that build on PCNA and BDNA. *Balochistan government wants to revise its Governance Program. Endorsement of non-government stakeholders including donors remains, as targeted by the Project. Indicator 3 : Development Partners Forum convened as needed (modified at AF2) Value (quantitative No Yes Donors/CSOs participating in vi Yes

12 or Qualitative) Development Forums hosted by KP. Date achieved 10/11/ /31/ /30/ /10/2015 Comments (incl. % achievement) Achieved. Targets met. Forums established and convened in KP and FATA. Balochistan participated in a development partner forum in Islamabad hosted by KP but is still in the process of establishing its own. Indicator 4 : Key stakeholders successfully included in consultations (modified at AF2) Value (quantitative or Qualitative) No Yes Number of Participants in consultation activities during project implementation Yes. See New IO for number achieved on revised target. Date achieved 11/01/ /30/ /29/2016 Comments Achieved. (incl. % achievement) Government and Donor consensus on implementation of findings and recommendations Indicator 5 : (dropped at AF1) Value (quantitative or Qualitative) n/a Yes n/a Date achieved 11/01/ /30/2014 Comments (incl. % achievement) Indicator 6 : Not rated. Refers to the PCNA which had been endorsed prior to GSP project implementation. Government's capacity to implement key interventions (in KP and FATA, dropped at AF1) Value (quantitative or Qualitative) None provided. 5 Governance Interventions funded; 2 reform actions implemented; 11 governance funded; 8 reform actions implemented Date achieved 11/01/ /30/ /29/2016 Comments (incl. % Achieved. Demonstrated government capacity with targets under Rapid Response Facility (interventions funded and reforms implemented) exceeded. achievement) Indicator 7 : Improved transparency and accountability of the PCNA Program (dropped at AF2) vii

13 Value (quantitative or Qualitative) None provided. Unclear Improved transparency and accountability of the PCNA program Yes in KP and FATA and of the development progra m in Balochistan Date achieved 06/30/ /30/ /29/2016 Achieved. Although specific targets undefined at design, several transparency and accountability outcomes of the PCNA were achieved notably through the development Comments and publication of PCNA monitoring reports and the strengthening or establishment of a (incl. % number of Grievance Redress Mechanisms (GRM) and institutions under the RRF in achievement) support of the PCNA s priority governance areas such as citizen engagement and oversight, and access to timely and equitable justice. Monitoring systems developed (for PCNA and Governance Programs in Balochistan) Indicator 8 : Value (quantitative or Qualitative) No Yes/2* Yes Date achieved 10/11/ /31/ /10/2015 Comments (incl. % achievement) Indicator 9 : Achieved. Targets met. Based on developing country systems. M&E Directorates of KP and FATA jointly produced two PCNA monitoring reports and undertook an evaluation of a number of PCNA investments on citizens perceptions of services. FATA and Balochistan Directorates piloted geo-referenced third party monitoring of select services. *Targets reported in AF2 RF as a numerical value but in subsequent ISRs as Y or N. NEW (AF2) -Project beneficiaries Value (quantitative or Qualitative) Date achieved 09/11/ /31/ /10/2015 Comments (incl. % achievement) Achieved. Target Exceeded. Project reports at project end as beneficiaries the citizens engaged through the consultations and workshops, the GRMs established or strengthened, the third party monitoring of select services, a Citizen Report Card conducted in KP through the Ombudsman Office and the PCNA Evaluation. This indicator was included as a standard core indicator at AF2 but a project beneficiaries indicator was first added in AF1 RF with Line Departments/Directorates in KP, FATA and Balochistan as the beneficiaries. Indicator 10 : NEW (AF2) -Female beneficiaries Value (quantitative or Qualitative) Date achieved 09/11/ /31/ /10/2015 Comments (incl. % Achieved. Targets exceeded. achievement) viii

14 Indicator 11 : NEW (AF2) -Participants in consultation activities. Value (quantitative or Qualitative) Date achieved 09/11/ /31/ /10/2015 Comments (incl. % Achieved. Targets exceeded. achievement) NEW (AF2) -Participants in consultation activities during project implementation - Indicator 12 : female Value (quantitative or Qualitative) Date achieved 09/11/ /31/ /10/2015 Comments (incl. % Achieved. Targets exceeded. achievement) Governance frameworks to lead reforms developed Indicator 13 : Value (quantitative or Qualitative) 0 2 2* Date achieved 10/11/ /31/ /10/2015 Comments (incl. % achievement) Achieved. KP and FATA developed 10 year governance programs. *The government of Balochistan considers the 10 year governance plan developed for the province needs to be revised and has not endorsed it. G. Ratings of Project Performance in ISRs No. Date ISR Archived DO IP Actual Disbursements (USD millions) 1 07/04/2012 Satisfactory Moderately Satisfactory /14/2013 Satisfactory Satisfactory /14/2013 Satisfactory Satisfactory /29/2014 Satisfactory Satisfactory /12/2014 Satisfactory Satisfactory /13/2015 Moderately Satisfactory Moderately Satisfactory /22/2015 Satisfactory Moderately Satisfactory /13/2016 Satisfactory Satisfactory ix

15 H. Restructuring (if any) Board Restructuring Approved Date(s) PDO Changed ISR Rating at Restructuring DO IP Amount Disbursed at Restructuring in USD millions 06/14/2012 Y S MS /13/2015 N MS MS /19/2015 N MS MS /28/2015 N S MS /30/2015 N S MS I. Disbursement Profile Reasons for Restructuring & Key Changes Made AF-1: To expand Project to Balochistan. PDO and Components revised to add Balochistan AF-2: To scale up a few activities in FATA especially and make RF outcome-oriented. PDO indicators dropped and new indicators included Closing Date Extension for completion of project activities and provide a bridge for new operation Closing Date Extension for completion of project activities and provide a bridge for new operation Closing Date Extension for completion of project activities and provide a bridge for new operation x

16 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. At the time of Appraisal, the Khyber Pakhtunkhwa (KP) and the Federally Administered Tribal Areas (FATA) were among the poorest regions of Pakistan and faced fragility. Both areas lagged behind national averages on many social and economic indicators. Moreover, the regions faced fragility and instability driven by pockets of militancy, with spillover from the conflict in Afghanistan. In early 2009, the Government of Pakistan (GoP) undertook military operations against militants which contributed to significant damage to infrastructure and services, imposed great economic cost and displaced an estimated 3 million people The federal and provincial governments embarked on rapid post-crisis recovery efforts in collaboration with international and national development agencies. A Damage and Needs Assessment and a Post Crisis Needs Assessment (PCNA) were conducted in The PCNA assessed the short and medium term social and economic needs of the regions in responding to the post-crisis situation and provided the underpinning for long-term development and peace building efforts in KP and FATA through four strategic objectives. 2 It became the point of reference for government and donor agencies, targeting investments of US$2.8 billion over a ten-year period. 3. A Multi-Donor Trust Fund (MDTF) was established for the recovery and rehabilitation of the crisis-affected areas in KP, and FATA, and subsequently the province of Balochistan. Based on the PCNA, the MDTF was expected to support the governments efforts in aid management and donor harmonization. The Trust Fund has been operational since August 2010 and brings together 11 donors representing commitments of USD 139 million. The World Bank was requested to administer the MDTF. 4. The MDTF-funded Governance Support Project (GSP) was prepared to support PCNA implementation on governance. The PCNA had identified political, governance and rule-of-law deficits as critical drivers of the crisis. In response, the KP Government and FATA Secretariat put in place the legal framework for establishing PCNA Implementation Support Units (ISUs) in the regions Planning and Development Departments (P&DD), responsible for coordination and liaison with government departments. To expedite preparation and in light of the governments limited capacities, GSP was designed in 5 months under the Bank OP 8:00 Rapid Response to Crises and Emergencies to support capacity building and implementation of PCNA recommendations on governance. 3 The Project was in line with the PCNA s first strategic objective (SO1) of building responsiveness and effectiveness of the State to restore citizen trust. Rationale for Bank Assistance 5. The Bank was well-placed to support the GSP given its cross-sectoral experience in conflictaffected areas and its prior work with the KP and FATA on the PCNA. The Bank participated in the development of the PCNA which allowed to develop in depth knowledge about the Program and the local situation in KP and FATA. Moreover, the cross-sectoral portfolio of the World Bank and its experience gained in other conflict areas served as a base for developing a cross-sectoral and fragility-sensitive governance program. Finally, as the administrator of the MDTF, the Bank was well positioned to support a longer term governance program through which other development partners could provide support and coordinate their assistance. 1

17 6. GSP was also aligned with the Pakistan Country Partnership Strategy (CPS) FY The Project was in support specifically of the Bank Country Partnership Strategy objective of Increased Responsiveness and Effectiveness of the State in Pakistan Original Project Development Objectives (PDO) and Key Indicators 7. The original PDO as per the Emergency Project Paper (EPP) and the Grant Agreement was to improve the delivery efficiency of the PCNA program through institutional strengthening and support of KP and FATA, while responding to priority needs of the PCNA governance program. 8. Progress towards the PDO was to be measured by the Results Framework (RF) outlined in the EPP. Key indicators included two PDO level indicators (on capacity enhancement of government and improved quality of services), 7 associated qualitative intermediate outcome (IO) indicators and 9 output indicators with primarily quantitative targets (e.g., 5 studies completed; 10 workshops organized). See original results framework and indicators in Annex Revised PDO and Key Indicators, and Reasons/Justification 9. In June 2012, the PDO was revised through adaptive Level 1 restructuring and Additional Financing (AF1) of US$2.75 million responding to changes on the ground. This restructuring and the US$2.75 million AF1 5 aimed primarily at expanding the project scope to respond to the request of the province of Balochistan to address the similar governance challenges that it faced. Those challenges were then captured under a Balochistan Development Needs Assessment (BDNA) modeled on the PCNA that provided the framework for the GSP interventions in Balochistan. AF1 also earmarked funds for a Project Preparation Facility (PPF) aimed at further strengthening the ISUs and line departments capacity to prepare a pipeline of interventions under the GSP eligible for MDTF financing. The restructuring also extended the project by one year to June The revised PDO was to strengthen the capacity of government departments in KP, FATA and Balochistan to help support efficient delivery of the PCNA program in KP and FATA and related development programs in Balochistan AF1 revised the Results Framework by expanding key indicators to Balochistan and dropping two intermediate indicators. The intermediate outcome indicators and output indicators were expanded to include and monitor performance in Balochistan. The IO indicators Donor consensus on implementation of findings and recommendations of the PCNA and Government s capacity to implement key interventions enhanced were dropped. Although the elimination of these indicators was not explained in the AF Project paper, government counterparts and Bank project team explained to the ICR team that these indicators were redundant or already captured in other indicators. Annex 4 provides a summary of the main changes in the project RFs. 11. In January 2015, a second Additional Financing (AF2) of US$4 million scaled up GSP activities and significantly revised the RF. The results framework was revised to take into account the scale up of activities in FATA especially and to better reflect the actual results of the Project. All PDO and output indicators but two 7 were dropped as well as many IO indicators, as many of them related primarily to inputs/outputs rather than outcomes. Alternative outcome indicators with baselines were added as PDO indicators. AF2 increased the Project amount to US$ million. Annex 4 also provides a description of the RF/Indicators changes at AF2. 2

18 1.4 Main Beneficiaries 12. The Line Departments/Directorates and the ISUs of KP, FATA and Balochistan were the main beneficiaries of GSP. Project documents indicate that main Project beneficiaries included the PCNA ISUs in KP and FATA, and the Line Departments in KP and Line Directorates in FATA. Following AF1, Balochistan ISU and Line Departments were included as beneficiaries. Citizens accessing services delivered by the targeted government agencies were the secondary, or ultimate, beneficiaries expected to gain from any capacity and service delivery improvements. 1.5 Original Components 13. The original PDO was intended to be achieved through two components (US$ 6 million) aiming at leveraging the large PCNA program. Component 1 (US$2.5 million): Strengthening of the PCNA ISUs, Provision of Technical Assistance and Institutional building of government departments. This component aimed to strengthen ISUs and selected governments departments in core implementation functions. Component 2 (US$3.5 million): Rapid Response Facility (RRF) for demand driven reforms. Activities under the two components included: TA, capacity building, workshops and focusgroup discussions, provision of goods and equipment. See table e in Annex Revised Components 14. With the US$ 2.75 million AF1, Components 1 and 2 were modified to respectively include support to strengthen the Balochistan ISU and line departments and implement the RRF for Balochistan. Component 1 was also revised to provide funds for a PPF for (i) KP and FATA ISUs to prepare sub-projects under the PCNA and (ii) Balochistan ISU to prepare governance reform initiatives. Table e in Annex 1 describes AF1 changes in activities/additions per component. Revised Component 1 (US$4.58 million): Additional support to Balochistan ISU and government departments; Project Preparation Facility for KP and FATA. Revised Component 2 (US$4.17 million): RRF also extended to Balochistan. 1.7 Other significant changes 15. The Project also underwent three additional simple restructurings with exclusive scope to extend the closing date. In June 2015, the Project closing date was extended from June 30, 2015 to October 31, 2015 to complete remaining activities while prepare for new scaled up governance operations in KP, FATA and Balochistan. 8 While a longer extension was requested, the Project could initially be extended by only four months to fall within the MDTF's round I end disbursement date of December 31, Once the MDTF was extended, two further simple Project extensions took place bringing the Project s final closing date to June 29, Table c in Annex 1 summarizes the project s restructurings and key changes Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 16. Project design incorporated lessons learned from conflict-affected environments and was based on previous analytical work. Consistent with Bank s experience in fragile and conflictaffected contexts (FCS) and the 2011 World Development Report (WDR), 10 project design was 3

19 kept simple and flexible with two components including one specifically dedicated to Project s flexibility to a changing context. Other lessons from FCS taken into account included: (i) ensuring a robust understanding of country context achieved through prior analytical work such as the PCNA; (ii) focusing on donor harmonization through enhancing the ISUs coordination role Significant project risks were identified and mitigation measures integrated in the design. The key risks were deemed high and included capacity constraints; institutional sustainability; governance risks (political interference, decision-making bottlenecks, transparency in contract management); and a volatile macroeconomic and security environment. The governments and the Bank team incorporated mitigation measures to attract qualified staff, provide enhanced implementation support, and strengthen fiduciary control mechanisms, particularly for newly established ISUs. 12 Steering Committees headed by the Additional Chief Secretaries of each region were also set up to oversight ISUs decisions and actions. Finally, MDTF Bank-Executed resources were also foreseen to conduct analytical work in support of the project upon request The design and quality at entry were however affected by a broad scope of activity. The PCNA defined 7 broad governance priority areas and a long list of potential interventions under those areas. 14 The limited GSP funds aimed to support all those areas through TA or direct funding (RRF) with the criteria for funding pre-determined by the Steering Committee and arguably too broad for real prioritization to emerge (e.g., one of the main criteria was alignment with PCNA Strategic Objective 1 (SO1), MDTF and GSP objectives; other criteria were general such as: nonduplication; realistic implementation timeline; and a first come, first serve principle of selection). See Annex 6 for list of criteria and long-list of potential governance interventions for GSP. 2.2 Implementation 19. The Project had a strong disbursement record overall, although actual funds utilization was lagging behind especially in KP and Balochistan. Project disbursement reached 24% within five months of implementation and 43 % by mid-term (see GSP disbursement profile in Annex 1). However, actual payments at the project level were much lower at 23% on average especially in Balochistan (11%). The Mid-Term Review (MTR) recommended focus on Balochistan funds utilization and fiduciary management resulting in improvement in utilization rates. By June 2016, disbursement had reached 100% with project-level expenditures close to 100 % in all regions Project implementation was affected by delays in hiring a number of Specialists especially in KP. The project was launched on October 31, 2011 by the Governments of KP and FATA. Project Operational Manuals and the Annual Work Plan were approved by the Steering Committee and the Controller General of Accounts. 16 Meanwhile, delays in hiring of ISUs specialists from the start (FM, Procurement, Communication, M&E) in KP especially and FATA slowed implementation progress. After the mid-term, delegation was recommended to ISU Coordinators for the approval of hiring for any vacancies and for payments. 17 In Balochistan, relying on a parttime ISU coordinator for most of the Project contributed to implementation and payment delays. 21. The Project also experienced delays in procurement that were gradually addressed. Delays in hiring ISUs Procurement and Financial Management Specialists and initial needs for clarifications on the procurement guidelines also resulted in procurement delays. 22. AF1 allowed to expand Project s scope to Balochistan while the MTR and AF2 focused on improving implementation and better capturing the impact of project activities. AF1 4

20 expanded project activities to Balochistan. Meanwhile, the MTR recommended strengthening implementation arrangements and capacity building activities (strengthening ISUs and M&E, and expanding Bank-executed TA). 18 AF2 focused on a significant revision of project indicators. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Design 23. M&E arrangements were devised at the start of the project but its design was impacted by limitations of the results framework. M&E arrangements consisted of the hiring of an M&E Specialist in each ISU responsible for overall project M&E and providing quarterly progress reports. 19 Project M&E was limited by a lack of baseline data partially due to the fragile context and the recentness of the PCNA. Moreover, the original indicators were largely output and input focused rather than outcome or results oriented and some of them ambiguously defined. In addition, the result chain between stated IOs, PDO indicators and the overall PDO was weak for component 2. In particular, the improved quality of services PDO level indicator and related transparency and accountability IO needed to be better linked to the overall PDO of delivery efficiency and unpacked to better determine their achievement. It should be noted that Bank guidance indicates that output indicators can suffice to measure the effectiveness of an emergency operation only where nothing more can be done The Project also aimed at strengthening M&E country systems that could contribute to Project M&E. In a context of overall limited M&E capacity, the Project aimed at contributing to building longer-term country M&E capacity through strengthening the M&E Directorates in the provinces respective P&DDs and supporting the development of government monitoring systems for the PCNA/BDNA. In turn, the M&E Directorates could contribute to the M&E of the Project. Implementation 25. For the most part of the Project, the M&E system was in place with intensive implementation support and capacity building, including partnerships with academia. Regarding M&E at the Project level, data was regularly collected and reported in ISU progress reports and Implementation Status Reports (ISRs). To strengthen the overall M&E apparatus, the Bank emphasized ongoing dialogue between ISUs and the Bank Task Team as well as regular supervision missions. Regarding the broader government M&E system, a comprehensive institutional needs assessment for the M&E Directorates of the three provinces were conducted and despite an initial lag, 21 recommendations implemented to address the capacity gaps identified, mostly in the form of trainings. The M&E Directorates of KP and FATA produced two PCNA Monitoring Reports as well as an evaluation of the impact of information campaigns on several GSP and non-gsp interventions on citizen trust and perceptions of the quality of public services. The M&E Directorates of the regions also linked with local academia and students to undertake pilot field monitoring of a few public services to help overcome the monitoring challenges posed by more remote and conflict-affected areas. 26. AF2 significantly strengthened the outcome-orientation of the RF and added baselines, but this came late in the Project lifespan with a few missing values. The significant revision of the RF through AF2 did not take place until 6 months before the Project s initial closing date. In addition, for at least two indicators, 22 baselines and/or targets were still not provided at AF2. Utilization 5

21 27. The Project Additional Financings indicate use of project monitoring data to inform decision-making and resource allocation. In AF2, based on the M&E data reported and given that expenditures in FATA had already reached 100%, it was decided to direct 80% of AF2 funding to FATA for scaling up of activities such as the FATA Tribunal and Anti-Corruption Hotline. 28. There is also evidence that the data generated through the in-country monitoring systems led to concrete action to improve services. Once the regions M&E directorates were better capacitated and in partnership with academia, they were able to undertake several monitoring activities as mentioned including the development of a pilot of geo-tagged dashboards to monitor selected services in terms of availability and quality, with some evidence of actions taken as a result of the monitoring. 23 The governments of KP and FATA also indicated that the PCNA Progress reports spurred action on delays and under-resourced programs for their correction Safeguard and Fiduciary Compliance Environmental and Social 29. The Project was rated as Category C as per the Bank s Environmental Assessment rating system. The Bank team prepared an Environmental and Social Screening and Assessment Framework (ESSAF) in accordance with OP 8.0 for emergency operations. Given it was assessed that the Project would not cause any negative environmental or social impact, the Bank s environmental and social safeguard policies were not triggered. Fiduciary Compliance 30. Project implementation has overall been compliant with the fiduciary rules defined in the Operation and Financial Manuals with in particular regular progress and financial reports, with Bank supervision contributing to its strengthening. Beyond mere compliance with the original Operational Manual, the fiduciary framework was improved during project implementation. For instance, the deviations in procurement of goods identified early in project life through supervision ex-post reviews were addressed through trainings and the introduction of prior reviews. The MTR also recommended to establish a contract management system and improve record management. Also, it was subsequently recommended ISU payment processes be streamlined and Project Coordinators empowered through enhanced financial delegation 25 to address consultant payment delays due to government procedures and lack of knowledge of Bank fiduciary rules. With capacity building and hand-holding, fiduciary performance was rated Moderately Satisfactory (MS) or Satisfactory (S) throughout implementation and upgraded back to Satisfactory at the end of the Project. 31. GSP also benefited from a Bank integrated fiduciary management approach. The assessment for FM and Procurement was combined, which enabled synergies and a more coordinated approach to addressing issues involving both dimensions such as contract management. 2.5 Post-completion Operation/Next Phase 32. The three Governments have maintained a number of project outcomes and requested a follow-up operation per province currently under preparation. The three Governments have increasingly used their own budget resources to sustain a number of the supported interventions for which GSP funding ended up being a fraction of the total allocation (see details in Section 5 on Government s performance). Three operations (Governance and Policy Program -GPP) have 6

22 been requested by the respective provincial governments and their preparation is well advanced. These operations are focused on revenue mobilization, public investment management and service delivery. On the latter area, the operations are building on GSP by aiming to further strengthen government M&E capacity, engage citizens in monitoring service delivery and increase the effectiveness of GRMs. The programs are expected to utilize GSP institutional arrangements notably through implementation units housed in the respective P&DDs. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Relevance of Objectives Relevance of Original PDO (Sub-Rating: High) 33. The original PDO remains relevant and consistent with Pakistan s and the regions (KP and FATA) current development priorities and the Bank s country strategy. The original PDO is consistent with the GoP development strategy (Vision Document of 2025), which identifies the need for a government focused on service delivery and promoting responsive, inclusive, transparent and accountable system of governance. 26 The KP Integrated Development Strategy (IDS ) prioritizes increased coverage and quality of basic services with the need to embed governance reforms in all government priorities. FATA adopted in 2015 a Return and Rehabilitation Strategy for displaced people with 5 pillars that requires continuous institutional capacity development. 27 As mentioned, GSP objectives are in support of the KP and FATA government priorities as outlined in the 10 year PCNA. The Bank s new Country Partnership Strategy (CPS) FY emphasizes the need for improving public sector accountability, transparency, and performance. The relevance of the original PDO is therefore rated as High. Relevance of Revised PDO (Sub-Rating: High) 34. The Revised PDO did not change in essence and is also relevant to the country s and regions current development priorities and the Bank s country and sectoral assistance. The revised PDO remained focused on delivery efficiency through institutional strengthening and capacity building and only expanded that objective to Balochistan. Balochistan Comprehensive Development Strategy ( ) emphasizes developing the capacity of public officials. With the inclusion of the province, the revised PDO was to directly support the Balochistan Development Needs Assessment (BDNA) which was modeled on the PCNA and highlights similar governance needs and priority areas. 28 The relevance of the revised PDO is therefore rated as high. Relevance of Project Design and Implementation (Sub-Rating: Modest) 35. The project design and implementation was consistent with the PDOs and incorporated relevant approaches for achieving the objectives. As outlined in section 2.1., the Project design was based on background analysis and took into account lessons learned in FCS. The components/sub-activities did not fundamentally change when the PDO was revised other than to undertake similar interventions in Balochistan. The design of Component 1 aimed at operationalizing and strengthening the PCNA ISUs was in line with the PCNA recommendation to establish a governance structure for the Program implementation. Dedicating component 2 to a Rapid Response Facility also responded to the governments expectations for flexible funding for priority or non-planned needs. 7

23 36. Project design was however affected by the ambiguity and lack of relevance of select indicators, weak results chain under component two especially at entry and a focus on output indicators up to six months before initial project closure. The quality of services PDO indicator lacked relevance to the overall PDO while the related transparency and accountability IO, as well a number of output indicators ((studies completed, workshops conducted, interventions funded in a timely manner, etc ) were ambiguously defined. The limitations of the initial results framework affected the ability to track the actual result of project activities and their effect on the PDO. With AF1, the results frameworks remain output oriented. At AF2, six months before the initial project closure, the result framework was revised and became more outcome oriented with baselines added but the indicators were not linked to the specific components. In addition, a couple of baselines and targets were still missing with discrepancies on values reported across project documents as further described in Section Finally, the relevance of project design was affected by a broad scope of activity. The PCNA and BDNA identified 7 governance priority areas ranging from access to timely justice to service delivery to support to institutional strengthening and PFM, which was too broad a scope for a project of US$ million across three provinces. Given the scope, activities undertaken by GSP were aligned with the priorities areas. There was however a need for further selectivity even within the RRF, as exhibited by the greater demand for funding than was available and in order to strengthen the justification for the activities supported. There was not a clear delimitation at design in the criteria for projects/departments that would receive GSP assistance from Component 1 versus Component 2 (RRF). 30 In light of the limitations described and their implications for assessing the results of the Project, the relevance of design and implementation is rated as Modest. 3.2 Achievement of Project Development Objectives Rating: Substantial 38. Given the original and revised PDO did not change in essence with the addition of Balochistan, the ICRR team undertook one combined analysis of the original and revised PDO looking at project results across the three regions in terms of delivery efficiency. Based on current ICRR guidelines, results are to be assessed separately against original and revised PDO. However, given GSP s PDO did not change other than to be expanded to Balochistan and only 11 percent of funds were disbursed when the PDO was revised, the ICRR team undertook only one combined analysis of the project results achieved in KP, FATA and Balochistan by project end. 39. The PDO of delivery efficiency was unpacked into the following two sub-objectives: Improving governments capability to manage and coordinate implementation of PCNA and BDNA Improving implementation effectiveness and efficiency (in terms of timeliness of delivery and responsiveness) of priority governance interventions. Attribution and Result Measurement 40. The analysis to determine PDO achievement is relying on the assessment of the most relevant outcomes and outputs from the result frameworks. Most relevant indicators are the outcomeoriented PDO level indicators from the final RF (AF2) and 2 IOs from the first two RFs (i.e. ISUs fulfilling their mandate; government enhanced capacity to implement governance interventions under the RRF) with related outputs. 31 The ICRR considered only those results that can be 8

24 reasonably and to a considerable extent attributed to project activities. Annex 5 includes a detailed assessment of achievement by each outcome indicator pursued by the Project. 41. The achievement of improving government capability for PCNA and BDNA is assessed as substantial. The substantial rating is based on GSP having achieved its targets for building capacity of ISUs and government departments for PCNA/BDNA in the three regions as follows: a) Improved planning, analytical, and M&E capacity for PCNA/BDNA. The Project strengthened the three regions capacity for the planning and monitoring of the PCNA and BDNA. This included supporting Line Departments/Directorates in developing project proposals aligned with PCNA/BDNA priorities for MDTF and RRF funding; and conducting relevant diagnostics studies such as FATA Development Household Survey (FDHS) which was used notably to better target investments. GSP also focused on developing the capacity of the M&E directorates to undertake PCNA monitoring reports. FATA and Balochistan Directorates have also piloted geo-referenced third party monitoring of select public services. 32 b) Improved capacity on fiscal, fiduciary, and other core functions for PCNA and BDNA. GSP also supported the institutional strengthening of central and line government departments notably to build fiscal and fiduciary management capacity. In KP, GSP provided equipment and TA to the Public Procurement Regulatory Authority to develop rules and regulations, procedures, a capacity building plan, an M&E framework, and a Management Information System (MIS). In FATA, GSP helped strengthen the PFM System through establishing an Internal Audit Wing in the FATA Secretariat which conducted audits of Education, Health and Irrigation Directorates. 33 In Balochistan, the Project supported Finance Department and the newly established Balochistan Revenue Authority (BRA) with collection on the Sales Tax on Services showing promising initial results. 34 Trainings were imparted to government officials in various departments notably on common function areas such as FM and M&E with the ISUs providing cross- support to departments across sectors on FM, procurement, communications. c) Improved donor coordination and mobilization of US$2.6 billion investments for the PCNA. Through their ISUs, FATA and KP Secretariats established in each region a Development Partners Consultative Forum consisting of technical working groups (development partners and civil society). In KP, this engagement resulted in developing a Strategic Development Partnership Framework with donors. Through these donor-government coordination mechanisms, total PCNA investments have reached US$ 2.6 billion against the government target of US $2.8 billion (and a Project target of $1.5 billion by June 2015) with 69% coming from the Government of Pakistan and 31% from its development partners. 35 d) Improved design capacity for long-term interventions. The Project targeted the design of a full 10-Year Governance Program building on the PCNA/BDNA priority governance areas. This Governance Program includes specific proposed interventions, an action plan for implementation, and costing of needs to be funded by PCNA allocations. Both KP and FATA developed long-term Governance Programs reflecting their respective priorities in consultation with a diverse range of stakeholders. 36 A 10-year governance program was also designed in Balochistan still to be formally adopted by the government The achievement of improving implementation effectiveness and efficiency of priority governance interventions is assessed as substantial. The substantial rating is based on two elements: 1) Project RRF exceeding its original output targets in terms of the funding of key 9

25 governance interventions; 2) Impact of implemented interventions on citizens in terms of increased delivery of services, reduced processing times, and resolved grievances. a) Effectiveness of the Rapid Response Facility. The RRF funded 13 governance interventions (6 by KP, 5 by FATA, 2 by Balochistan) while at least 6 were targeted. 10 interventions were implemented (6 in KP, 2 in FATA, 2 in Balochistan) while at least 2 were targeted. The Project also aimed to reduce the RRF lead time (average number of days from proposal to start of implementation, defined as disbursement of funds from the RRF), although the target time was unclear and not consistently defined in project documents. 38 While it took longer for the RRF to process a few interventions 39, most were funded in less than 2 months. See Annex 2 for a list of the interventions funded/implemented in each region and corresponding lead times. b) Reduction in the average processing time for selected public services. In KP, the GSP helped establish a Right to Public Services Commission through the RRF and provided the Commission support to monitor the delivery of 5 services initially through a Performance Management System (PMS) for which delivery standards (mandated processing time) had been established under the first RTPS Act in Pakistan. The 5 services saw a reduction in average processing times as well as high and increasing compliance with the standards. 40 A total of 20 services are now notified under RTPS and compliance rates for a majority are also high. 41 GSP also provided financial and technical support to operationalize the FATA Tribunal (an Appelate Court) 42 and install a case management system while training 58 officers to expedite the processing of cases. An analysis of the decided cases shows a sustained decrease in average time for appellate decisions from 721 days at Project start to 56 days (in 2015) (target of 93). 43 In Balochistan, support was also provided to monitor issuance of domicile certificates but it was done through collecting perception data from users rather than tracking processing times. c) Grievances registered related to delivery of services addressed. The Project supported strengthening Grievance Redress institutions through the RRF including the Peshawar High Court, Anti-Corruption Establishment (ACE) and Ombudsman in KP and Balochistan Ombudsman. The KP ACE received restructuring and re-organization support while an E- Citizen GRM was established at the Peshawar High Court which surpassed its annual target of disposed grievances. 44 The RRF supported the set-up of a public GRM for KP Ombudsman and established an E-Citizen GRM for Balochistan Ombudsman to provide additional channels through which citizens could submit grievances (calls, web, sms). Yearly disposal of KP and Balochistan Ombudsman grievances exceeded monthly and annual disposal targets. There was also a significant decrease in the average time to dispose of grievances from 614 to 169 days. d) Increased delivery for the main public service supported by the project in KP and FATA. In KP, the main service supported through the RRF was the issuance of domicile certificates and in FATA the Tribunal. In both cases, the number of transactions targeted by GSP was surpassed (and largely for KP). While GSP s support to FATA Tribunal can be directly linked to the latter s ability to process cases, the support provided to Domiciles Services was indirect through monitoring of compliance with delivery standards. The functioning of this service was dependent on other resources including Government s own resources. However, the now existence of a RTPS Act and Commission is likely to have contributed to the higher volume of transactions for the certificates (see detailed assessment of achievement in Annex 5). 10

26 43. Although its scope was broad, GSP achieved the objectives set out in terms of both capabilities to implement PCNA/BDNA and implementation effectiveness and efficiency of priority governance interventions. The interventions are demonstrating efficiency and delivering concrete outcomes to citizens, which is especially important in an FCS or FCV context. 3.3 Efficiency Rating: Modest 44. Measuring the efficiency of governance projects can pose distinct challenges related to being able to quantify the impact of interventions and derive precise rates of return. The challenge is magnified in the case of GSP due to issues of data reliability and availability given the conflictaffected context, which affect the overall MDTF portfolio. The team thus took an approach using both qualitative and quantitative analysis and relying on derivations and assumptions for the latter. 45. The partial quantitative analysis estimates benefits for a key service/activity supported by GSP in each region with the results suggesting monetary savings and positive returns. Based on the available data, the services analyzed are the issuance of domicile certificates in KP, the processing of cases at FATA Tribunal and of complaints at Balochistan Ombudsman. However, as non-gsp funding also contributed to the functioning and efficiency gains in those services, actual benefits in light of greater costs are likely to be lower. Given the data limitations, limited scope of the analysis (these services represent 9% of GSP investments), the quantitative analysis is illustrative and to be considered as only one element of the efficiency assessment. The analysis with the estimated benefits and underlying assumptions and data is available in Annex Other factors indicate positive or neutral impacts on efficiency. They include GSP s achievements compared to its targets, the impact of the AFs, the planned vs actual component costs, the impact of the project extensions, and the processing time of RRF proposals. They are: GSP achievements vs targets. Most outputs were exceeded with the available resources. Although a few targets may have been set conservatively, this indicates a degree of efficiency. Impact of Additional Financings. Two AFs were provided to GSP. The bulk of AF1 (73%) went to an additional area of coverage, Balochistan. The bulk (80%) of AF2 went to FATA where all original funding had been disbursed, to support ongoing activities with enhanced expected outcomes, which also suggests efficiency. Planned vs actual component costs. The appraised and actual GSP costs taking into account the AFs align. However, in Balochistan, a reallocation of US$ 530,000 was made from Component 1 to Component 2 due to higher than originally anticipated costs under the RRF. 45 Impact of time extensions. GSP was extended 3 times totaling a year (without additional funds disbursed), so funds could be fully utilized and activities completed, especially in Balochistan which was added to GSP later. However, as many of the outputs targeted were exceeded, the time extensions do not appear to have significantly affected efficiency. Processing time of proposals under the RRF. The RRF was important to the Project, especially in terms of reducing processing times and affecting efficiency. As analyzed when assessing outcomes, the achievement on processing times of the RRF is mixed. 47. Overall, the Project is rated Modest on Efficiency. There were efficiency gains under key activities through the reduced processing times and increased volume of transactions and qualitative indications of efficiency. However, as the quantitative analysis could only cover a small 11

27 share of GSP investments and with limitations of the reliability of the underlying data, the ICRR rates the Project s overall efficiency as Modest based on the available evidence. 48. It is important to note that per IEG guidelines, the Project s overall efficiency (rated Modest) is not to be confused with the achievement of the GSP in terms of improving delivery efficiency in the governments (rated Substantial). While the former refers to whether project resources were used efficiently (cost-effectiveness), the latter refers to the efficacy of the Project on its main objective. 3.4 Justification of Overall Outcome Rating Overall Outcome Rating: Moderately Satisfactory (MS) 49. The overall outcome rating emerges as Moderately Satisfactory. This combines the ratings for Relevance (substantial), Efficacy (substantial), and Efficiency (modest), per OPCS and IEG guidelines. 46 Overall, GSP achieved its objectives with moderate shortcomings in Relevance driven by the design limitations; and Efficiency, given data limitations for cost-benefit analysis. Table 1: Combined Assessment of Outcomes (Against Original PDO and Revised PDO) Relevance Efficacy Efficiency Outcome Objectives Design Sub-objective 1 Sub-objective 2 High Modest Substantial Substantial Moderately Modest Substantial Substantial Satisfactory 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 51. The GSP reports having engaged and reached a total of 108,399 citizens compared to a target of 2,000. This was done through the consultations and workshops, the GRMs established or strengthened, the third party monitoring of select services, a Citizen Report Card conducted in KP through the Ombudsman Office and the PCNA Evaluation. The beneficiaries included 31,119 women (target of 200) or about 29% of beneficiaries. 47 The Project targeted women in its outreach notably through focus groups. This engagement of women is considered by all stakeholders a substantial achievement in light of their low levels of involvement in public affairs in the regions. 52. In addition, recommendations of GSP-supported initiatives are being incorporated in key government policy with a potential to affect poverty. In FATA, the Secretariat has utilized the results of the FDHS conducted with funds from the GSP 48 to develop the new Annual Development Program (ADP) and allocate resources. The Survey was also utilized to request greater allocations per capita from the federal government to the region. The FDIHS is the first in the region with the only socio-economic data previously available for FATA from the 1998 census. (b) Institutional Change/Strengthening 53. The Project s focus on institutional strengthening and especially on developing country systems has yielded both tangible and intangible capacity gains among some of the targeted agencies. The focus on country systems has enabled gains as highlighted in this assessment in terms of government capacity to undertake development interventions and contributed to capacitation of institutions in the longer-term. This is exemplified by the growing capacity of the M&E directorates to conduct more rigorous evaluations. 49 As the Project s objective was to 12

28 improve overall delivery capacity of the governments beyond the ISUs, technical assistance and funding was provided to a total of 17 line departments across the three regions. In light of the various trainings conducted, post-trainings evaluations would have been useful to further assess capacity gains and future needs. In addition to the points mentioned in the assessment of efficacy, there have been indications of behavioral change among government actors such as a more resultsoriented focus among the M&E Directorates and a growing focus on impact on citizens of government interventions in general among the targeted institutions moving forward Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 54. Although no beneficiary surveys were conducted in the context of the ICRR, citizen satisfaction data on select services that received GSP support was collected during project implementation which indicates low levels of citizen awareness but satisfaction among those who have used the services. 88 percent of litigants at the FATA Tribunal expressed satisfaction with the quality of the judgments received. Both KP Ombudsman and Balochistan Ombudsman implemented Citizen Report Cards (CRC) respectively on ten key public services and the effectiveness of the GRM system of the Balochistan Ombudsman. The CRC in Balochistan showed overall low citizen awareness about the Ombudsman s office or how to contact it. The RTS in KP has also collected user satisfaction data on selected notified services which is integrated in its PMS. Both FATA and Balochistan third party monitoring pilots of respectively public assets and the delivery of health, education and domicile certificates was complemented by beneficiary feedback data. There are also interviews of several heads of agencies supported by the Project citing GSP support as instrumental to the establishment or strengthening of their institutions. Several citizens interviewed were also satisfied with the handling of their complaints from KP Ombudsman and RTS and the Anti-Corruption hotline in FATA. 51 The Project has given greater visibility and credibility to government action in a context where it is needed. 4. Assessment of Risk to Development Outcome Risk Rating: Substantial 55. Overall, there is a substantial risk to maintaining the development outcomes made possible through the GSP which the follow-on projects and the government s own commitments can help reduce. These risks mainly pertain to the institutionalization of many of the GSP supported mechanisms that enabled gains and to risk factors outside the scope and control of the project and implementing agencies. The rating is based on the probability of the risk materializing combined with the potential impact, should that risk materialize. Four main risks have been identified and assessed for their potential impact: o Government may not be able to retain ISUs technical and operational staff supporting reforms. There is a risk that a financing and continuity gap would disrupt the technical and operational support for the reforms. This risk has already materialized through loss of ISU staff with institutional memory that are hard to replace. GPP projects will help acquire new staff, but capacity embedded in government own planning and resources will be critical to sustain reforms. o Accountability gains may not be sustained due to a lack of public awareness and institutionalization of engagement mechanisms. GSP-enabled gains on social accountability (judicial apparatus; grievance redress and citizen feedback mechanisms) are impacted by a lack of citizen awareness about these mechanisms. The PCNA evaluation conducted showed that greater 13

29 awareness can improve people s perception of public services. Although the governments have taken up the funding of many of these initiatives, greater institutionalization of citizen outreach and engagement mechanisms would be needed for more sustainable and effective accountability. o Loss of government commitment. Government turnovers including of high level officials are frequent in Pakistan, and new officials might not give the same importance to GSP supported reforms. This risk has not materialized during the project but could be further mitigated by integrating interventions in government planning and budgeting. o The unpredictable security situation could derail any development activity. While the security situation has improved, violent activities continue to pose significant risks in KP, FATA and Balochistan and could potentially disrupt any activity, damage infrastructure and displace civilians in the affected areas. Flexibility in implementation such as through remote or third party mechanisms may contribute to mitigate this risk, and is being applied by the GPP. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Sub-Rating: Moderately Satisfactory (MS) Strengths 56. The Project had significant relevance to addressing the regions priorities, because it directly supported needs identified by the regions. GSP supported the implementation of the emergency reform plans in KP, FATA and Balochistan that identified their respective short and medium term development needs. GSP s focus on building government capacity was critical to these programs objectives of building responsiveness and effectiveness of the State in order to restore citizen trust. GSP was also underpinned by extensive analysis related to developing these reform plans. The PCNA in particular is based on extensive post-conflict assessments led by the governments. 57. The Bank took into account lessons from project implementation in conflict-affected and post conflict environments through a simple design. The Project design at entry was kept: simple (two components and a few capacity building activities); flexible (the RRF was to allow to fund government priority governance initiatives as they emerged); and focused on process interventions to address critical institutional strengthening needs (e.g., the operationalization of the ISUs to managing the implementation of the PCNA and BDNA). Weaknesses 58. The initial Project design and results frameworks focused on realizing outputs versus outcomes, lacked baselines and defined indicators ambiguously. While acknowledging GSP was swiftly developed in a crisis affected context, the initial key performance indicators were output oriented and not sufficiently explicit, and the results framework lacked baselines. Given the context, there was no precedent or pre-existing value for a number of indicators. However, the situation in terms of governance actions and reforms undertaken prior to GSP could have been estimated. Also, the RF set arguably low targets given most of them were exceeded, and ambiguously defined indicators. This led to RF revisions during implementation. 14

30 59. In addition, the design could have benefited from a more focused scope for interventions. Although the RRF was designed to respond to priorities as they emerged, the criteria for funding was too broad for true prioritization to emerge. Greater selectivity would have allowed to better leverage GSP to implement activities aimed at solving specific problems and deemed most urgent. 60. Quality at entry is thus rated Moderately Satisfactory. Although GSP initial design would have benefited from greater focus in scope and more defined and outcome based indicators, the design took into account the contextual challenges, incorporated lessons from similar contexts, benefited from extensive analysis, and took a comprehensive approach to identifying risks and designing related mitigating measures. (b) Quality of Supervision Sub-Rating: Moderately Satisfactory (MS) Strengths 61. Supervision was frequent and hands-on in spite of the limitations imposed by operating in a crisis affected environment. Intensive support was provided to mitigate the capacity challenges. This included regular Project Coordination meetings in Islamabad and formal implementation support missions when possible (every 6 months on average). When the security situation prevented the team from visiting the Project, alternate arrangements were put in place to ensure adequate supervision. For example, Project teams were invited to Islamabad and Karachi; or meetings were held through Video-Conferencing. Close supervision especially in the second half of the Project helped accelerate funds disbursement and utilization. 62. The Bank team demonstrated proactivity notably through two main restructurings that allowed to focus the PDO and target the results framework on outcomes. These revisions were necessary to better capture the actual scope of the project (capacity building) and assess the impact of interventions. The team also incorporated delivery baselines for the services being supported at AF2. In addition, the team was proactive and successful in responding to the government requests for additional financing and including Balochistan. 63. The Bank worked closely with the three governments to maintain GSP supported institutional foundation through the preparation of follow-up projects. Through three new projects in KP, FATA and Balochistan that build on the GSP, the Bank supports a transition to maintain capacity that will also contribute to the new projects. The GSP was notably extended to provide a bridge for the GPP and retain ISU Specialists who actively participated in GPP design. Weaknesses 64. However, the significant revision of the RF at AF2 came too late to influence project activities and results and lacked consistency. The significant revision of the RF aimed to better capture the actual results of the project by focusing on outcomes. However, indicators selected lacked consistency (e.g., the indicator on increased funds directed to priority sectors based on citizen consultations was included for Balochistan but not FATA where budget consultations also took place; support to monitor the performance of select services such as domicile certificates on a pilot basis was also provided for Balochistan but not captured in the RF). Also, the revision of the RF took place more than 3 years into implementation and 6 months before the planned Project closure at the time, which limited its impact on redirecting activities if needed. 15

31 65. There were also consistency challenges in documentation and reporting. Indicator values (baselines and targets) for the final RF especially were not always reported consistently between AF2 and the subsequent ISRs (See detailed assessment of outcomes in Annex 5). 66. Long hiring times and payment delays, especially in KP, remained an issue throughout the Project that the Bank tried to proactively address but could not always resolve. The Bank was proactive in identifying the issues and proposed solutions to expedite hiring and streamline payments processes, but recommended measures were not taken on board or had limited impact. 67. Quality of supervision is thus rated Moderately Satisfactory. The Bank team provided frequent supervision and demonstrated proactivity in addressing potential issues. However, the mitigation measures either did not have the desired impact, or in the case of the second restructuring, came late and lacked consistency. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 68. The overall rating for Bank performance is Moderately Satisfactory based on the combined ratings of quality of entry and supervision. Although opportunities to strengthen design were missed and delayed, Bank close supervision and strong involvement, despite the limitations posed by this conflict affected context, significantly contributed to the achievement of outcomes and secured two additional financings. 5.2 Borrower Performance (a) Government Performance Sub-Rating: Satisfactory (S) 69. The grant recipient was the Government of Pakistan who then provided the funds to the Governments of KP and Balochistan, and FATA. The Governments of KP, Balochistan and FATA Secretariats had primary responsibility for reporting and overseeing project implementation in compliance with the Bank s guidelines and procedures including fiduciary aspects. 70. The three governments had strong ownership throughout the project which helped achieve project targets and contributed to sustainability. Government ownership in KP, FATA and Balochistan remained high throughout the Project, even following contentious elections and a change in Government in Balochistan. The governments also contributed to the sustainability and scope of impact by allocating government own sources to GSP-funded activities. GSP allocations have been a fraction of government allocations, with the project funding share for some interventions decreasing from 100% to less than 10%. 52 The Governments were proactive in requesting Project extensions to allow for additional time to complete Project activities and for a transition for the GPP in preparation. 71. The governments of KP and FATA especially built capacity to manage and coordinate donor support or their long-term development priorities. The governments of KP and FATA effectively established a Development Partners Forum providing a framework for consistent participation of donors to respond to government priorities and enhance mutual accountability. 53 Balochistan would need further support to institutionalize its donor coordination mechanisms. All three governments were able to design 10-Year Governance Programs which provide a government-led framework for coordinated donor support with the Balochistan one being revised. 16

32 Weaknesses 72. The effectiveness of the Steering Committees to oversee the ISUs and provide strategic management could have been enhanced. The Steering Committees were conceived to endorse ISU decisions and strengthen their mandate vis-à-vis Line Departments. They were mandated to meet at least twice a year in KP and quarterly in FATA and expected to be informed of project progress on a regular basis. In practice, however, these Committees provided only a minimum level of review and were not utilized much. 73. Government performance is thus rated Satisfactory. Although supervision of the Steering Committees could have been more frequent and Balochistan lagged on donor coordination, the governments demonstrated strong commitment and contributed to the sustainability of initiatives. (b) Implementing Agency or Agencies Performance Sub-Rating: Moderately Satisfactory 74. The PCNA ISUs of KP and FATA and the BDNA ISU of Balochistan were responsible for the overall management and implementation of the Project. They were housed and overseen by their respective P&DDs. They were responsible for day-to-day project implementation including FM, procurement, disbursement, and M&E arrangements. Line Departments in KP and Balochistan and Line Directorates in FATA were implementing partners especially for activities under the RRF. The ISUs had ultimate sign-off on requests for project funds. 54 Strengths 75. The Project was overall adequately managed by the ISUs despite operating in a high risk environment. Despite initial delays, implementation progressed steadily overall. The ISUs further demonstrated their strengthened capacity by engaging with more line departments, managing increased PCNA investments in KP and FATA, and training more than a thousand government officials. Fiduciary performance was assessed as moderately satisfactory or satisfactory during project implementation and upgraded to satisfactory during the last six months. Fiduciary arrangements were deemed satisfactory, and the ISUs remained generally compliant with the prescribed internal control framework, even in a high-risk and crisis-affected context. The last PRIMA and PRAMS assessments rated fiduciary performance satisfactory. 76. Although M&E capacity was initially limited, the ISUs helped build the capacity of country M&E systems while developing their own. Each ISU was staffed with an M&E Specialist who received additional training and produced regular progress reports. The Specialists had to rely on the M&E Directorates to report on governance activities supported by the GSP, e.g., the RRF projects. This tension and the context of low institutional M&E capacity initially led to unavailability of adequate data for evidence-based decision-making, recognized at the MTR. However, following a comprehensive institutional needs assessment, M&E Directorates received several trainings, which enhanced their capacity. Weaknesses 77. However, a few issues impeded performance and caused delays. For instance, Project start-up was delayed due to lack of knowledge about account opening procedures at the National Bank of Pakistan. 55 It took almost a year after Project effectiveness for KP and FATA to comply with two legal covenants, the recruitment of an internal auditor (due 3 months after effectiveness), and 17

33 getting the Project live in PIFRA (National Financial Management Information System). Annual work plans were at times delayed in all three regions and impeded performance and delayed AF In addition, staffing and payment delays were a problem in all three regions, in particular in KP, and affected budget utilization. 25% of ISU staff still remained to be hired 9 months into Project implementation including key fiduciary positions in KP (FM, Procurement, M&E). In KP and FATA, the hiring delays cannot be all attributed to the ISUs as they were partly due to additional internal departmental (P&DD) clearances, which affected the effectiveness of the implementation arrangements. 57 To address this bottleneck, the Bank recommended the ISUs of KP and FATA empower the Coordinators to hire after 7 days as per the Bank s procurement policy and maintain a roster of qualified candidates. In FATA where delegation to the Coordinator was in effect, there were less hiring and payment delays. In KP, this recommendation was not taken on board. In Balochistan, key ISU positions were vacant on average 60% of the time 58 and lack of a full-time Coordinator for most of the Project led to delays. Delays manifested themselves in lags in expenditures, especially in KP and Balochistan, and slowed implementation. However, with regular supervision, the Project was able to ultimately disburse and utilize funds fully. 79. The country context and GSP and other MDTF projects being the first that the Governments of KP and FATA have had with the World Bank in part explain the initial delays. Staffing is a systemic issue in Pakistan affecting projects across the board and especially in FCS areas. The overall initially steep learning curve for the implementing agencies and partners especially contributed to the initial start-up delays and slower utilization of funds as they familiarized themselves with the Bank procedures and were building their capacity The performance of the implementing agencies is rated Moderately Satisfactory. The performance of the ISUs was overall adequate, taking into account the limited capacity and high fragility context. There were delays especially at start up, but the agencies were able to ultimately put in place the operational apparatus for the Project and utilize the project funds as intended. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 81. Overall Borrower s performance is rated Moderately Satisfactory. This rating combines the Government s (S) and implementing agency s performance ratings (MS). The government s actors at the higher level and the ISUs demonstrated commitment, and progressed in the implementation by learning by doing. However, there were moderate shortcomings especially in the setting up delays and maintaining of the implementation arrangements and needed capacity. 6. Lessons Learned 82. Designing projects in fragile contexts needs an adequate balance between incorporating flexibility, and having well defined objectives, scope, and outcome measurement. While adaptability and flexibility is critical for projects in fragile contexts given capacity limits in particular and a changing environment, institutional strengthening in such contexts also need: In-depth context understanding combined with strong team field/supervision footprint; Analytics and problem identification to generate defined objectives and result frameworks with clear indicators and targets; Simple design with focused scope, limited number of components, clear implementation arrangements, and flexible mechanisms to ensure swift restructuring and implementation support. 18

34 83. Projects in fragile contexts could still aim to be oriented on development outcomes and results. It can be harder to develop a robust, linear results framework in a fluid, fragile environment and less risky to focus on processes and outputs than aim for actual more tangible outcomes. However, in those contexts, it is also important that citizens can experience investments in government capacity in terms of concrete improvements in service delivery. A balance could be sought between delivering tangible and visible results to populations and less visible capacity and institutional strengthening actions. 84. Bank-executed technical assistance can be leveraged for intense implementation support in fragile areas. The Bank team highlighted how with the limited capacity in KP, FATA and Balochistan, the Project required just-in time and continuous technical support of national and international consultants in particular to design reform solutions. This was made possible through Bank-executed analytical support, which is more easily and quickly deployable. The three province respective GPPs under preparation have incorporated this lesson in their design. 85. Citizen awareness and institutionalization is essential to the effectiveness of social accountability initiatives. As illustrated by this assessment, the effectiveness of the social accountability initiatives undertaken by the GSP can be enhanced by increasing and institutionalizing government engagement with CSOs and citizens so they can take advantage of available channels to reach government for better services. 60 In this regard, a more systematic reliance on CSOs, which engagement was not always sustained per the governments, may generate greater citizens awareness about the various initiatives as well. 86. Focused scope and sustainability arrangements are necessary for intermediary/cross cutting institutions providing capacity building to line departments. Within a context of scarce resources, capacity building through entities with a cross-cutting mandate embedded in government programs and supporting line departments, such as the PCNA and BDNA ISUs can generate synergies and have a leveraging effect. Meanwhile, the mandate of these entities needs to be clear and focused to avoid overburdening and substitution to line departments Three separate Projects would have enabled more tailoring to specific needs in terms of scope at design and addressing bottlenecks during implementation. Although it supported crosslearning between the 3 regions, 62 the implementation of the GSP as one project, especially when it came to supervision and reporting, masked differences in performance between regions, making it more challenging to address specific problems. One project per province would have enabled tailoring to the specific needs of each and simplified transaction and coordination costs with the various counterparts. The GPPs have incorporated this lesson with three separate projects. 7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors 88. The ICRR was shared with the three governments on December 12, 2016, and FATA and Balochistan provided comments. The Government of FATA requested to include two additional GSP outputs in the list of Outputs by Component (Annex 2), which the team did. They consist of the development of an Accelerated Development Strategy ( ), and a study on revenue sharing. Balochistan government indicated that the ICRR appeared to have scoped up all the relevant information and feedback. As of December 23, 2016, no comments were received from the Government of KP. 19

35 Annex 1. Project Cost, Financing, and Inputs (a) Project Cost by Component (in USD Million equivalent) Components 1. Strengthening of the PCNA ISUs, provision of TA and Institutional Building of government departments / directorates a. Goods, Consultancy, Training and Operating Costs b. Civil Works physical infrastructure (b) Financing (in USD Million equivalent) Source of Funds MDTF for Crisis Affected Areas of NWFP/FATA/Balochistan Appraisal Estimates Type of Cofinancing Revised at AF1 - June 2012 Appraisal Estimate (c) Summary of GSP restructurings and key changes Revised at AF1 Revised at AF2 - January 2015 Revised at AF2 Actual/ Latest Estimate Actual/ Latest Estimate Percentage of Appraisal % Percenta ge of Apprais al % % N/A N/A N/A N/A 2. Rapid Response Facility % c. Goods, Consultancy, Training % and Operating Costs d. Civil Works physical N/A N/A N/A N/A infrastructure Total Project Costs % Type of Milestone / Restructuring Level and Key Changes Made Date of Action Amount Original / Revised Closing Date Beneficiary Original Project October 31, 2011 US$ 6 million June 30, 2014 KP / FATA AF-1 (Level 1): PDO revised Component 1 revised AF-2 (Level 2): Additional indicators included Closing Date Extension (Level 2): June 14, 2012 US$ 2.75 million June 30, 2015 KP / FATA/Balochistan January 13, 2015 US$ 4 million June 30, 2015 KP / FATA June 19, 2015 n/a October 31, KP / FATA / End date extension for 2015 Balochistan completion of project activities Closing Date Extension (Level 2): October 28, 2015 n/ a Dec 12, 2015 KP / FATA / Balochistan 20

36 End date extension for completion of project activities Closing Date Extension (Level 2): End date extension for completion of project activities November 30, 2015 n/a June 29, 2016 KP / FATA / Balochistan (d) GSP Disbursements over project lifetime e) Governance Support Project Components and Activities Original Project Components and Activities Component I Strengthening of the PCNA Implementation Support Units, Provision of Technical Assistance and Institutional Building of government departments (total estimated cost US$ 2.50 million, 42%). Main Activities: Operationalization of ISUs through provision of staff & consultants, training & capacity building, goods & equipment TA and capacity building to ISUs and Line Departments/Directorates in KP and FATA to develop systems for coordinating and managing PCNA, establish pipeline of projects, and establish Development Partners Forum. Component II Rapid Response Facility (RRF) (total estimated cost US$ 3.50 million, 58%). Main Activities: Provision of goods, equipment, services with consultations, workshop and focus-group discussions to set up and manage a funding mechanism for governance activities to be identified and implemented by the Line Departments/ Directorates or other government agencies of KP and FATA that fall under PCNA. Revised Project Components and Activities (AF1) Revised Component I: Strengthening of the ISU of KP, FATA and Balochistan, Provision of Technical Assistance and Institutional Building of government departments (total estimated cost US$ 4.58 million, 52%). Revisions/Additions: (US$ 2.08 million) Full operationalization of Balochistan ISU TA and capacity building to Balochistan ISU and Line Departments to develop systems for coordinating and managing governance programs, establish pipeline of projects, and establish Development Partners Consultative Forum, Project Preparation Facility for Balochistan for governance programs ; Project Preparation Facility for KP and FATA to expedite preparation of projects under PCNA. Revised Component II: Rapid Response Facility (RRF) (total estimated cost US$ 4.17 million, 48%). Revisions/Additions ( US$ 0.67 million) Provision of goods, equipment, services with consultations, workshop and focus-group discussions to set up and manage a funding mechanism for fast-track implementation of specific governance interventions in Balochistan as identified by Line Departments. 21

37 Annex 2. Outputs by Component (a) List of Outputs by Component Activity Outputs at Project close (June 2016) KP: Component 1 - Strengthening of the PCNA ISUs, provision of TA and Institutional Building of government departments PCNA ISUs set-up and operationalization Project development Establishment of monitoring systems for PCNA ISU staffed and equipped Budget disbursed and spent Coordination meetings held with donors, Line Departments, civil society representatives (2) and academia (3) MDTF coordination committee comprising of ISU team and MDTF project s PDs meeting monthly 24 proposals developed by the Line Departments with 16 approved by MDTF Steering Committee: Strengthening of Judicial Academy, Ombudsman, Anti-Corruption Establishment, Directorate of M&E, Foreign Aid Section of P&DD, and Reform Implementation Cell for Service Delivery Performance Management; Support to establishing Public Procurement Regulatory Authority (PPRA), establishment of E-Citizens Grievance and of Case Management system, Redress System at Peshawar High Court, Support to RTI and RTS Commissions, TA for Land Record Computerization and Service Delivery Centre Software, Operationalization of Public-Private Partnerships (PPP) Unit, Development of a 10 years Governance Program, TA Implementation Readiness for the Southern Area Development Programme, Diagnostic Survey for KP Directorate of Higher Education PCNA Evaluation conducted by M&E Directorates of KP and FATA in collaboration with the Georgia State University and University of Peshawar Two PCNA implementation monitoring reports published and disseminated by M&E Directorates Implementation of key analytical work and studies Technical Assistance and institutional strengthening Institutional Needs Assessment for KP M&E Directorate 10 year Governance Plan for KP developed Citizen Report Card on 10 public services by KP Ombudsman Six (6) capacity building trainings for KP M&E Directorate (144 officials trained including 13 women) Twenty-five (25) workshops with 464 officials trained (including 53 women) notably on FM, Community-Driven Development and public participation and accountability (in the budget process) Establishment of Public Procurement Regulatory Authority Capacity Enhancement of Foreign Aid Section at P&DD Strengthening Reform Implementation Cell for service delivery performance management Donor coordination KP: Component 2 Rapid Response Facility Funding of priority Governance interventions Establishment of Development Partners Forum 21 coordination meetings held with donors 6 Governance interventions funded in KP: Operationalization of the KP Judicial Academy, strengthening of the Anti-Corruption Establishment, Strengthening of KP Provincial Ombudsman, Establishment of E-Citizen 22

38 GRM at Peshawar High Court, Support to the 2013 implementation of the RTI Law, and the establishment of the RTS Commission Implementation of 6 reform actions implemented: the 6 governance interventions funded above have reform actions been implemented. FATA: Component 1 - Strengthening of the PCNA ISUs, provision of TA and Institutional Building of government departments / directorates PCNA ISUs set-up and operationalization Project development Establishment of monitoring systems for PCNA Implementation of key analytical work and studies Technical Assistance and capacity building Donor coordination Citizen pre-budget consultations ISU staffed and equipped Budget disbursed and spent Coordination meetings held with donors (20), Line Directorates, civil society representatives (2) MDTF coordination committee comprising of ISU team and MDTF project s PDs meeting monthly 24 proposals developed by the Line Departments with 15 approved by MDTF Steering Committee: Fata Tribunal Phase-1, Directorate of M&E, FATA Tribunal Phase II, FATA Development Indicators Household Survey ( ), Donor Coordination Steering Committee, Anti-Corruption Hotline, Strengthening Finance Department, Pilot Testing of Internal Audit, Ten years Governance Program, Data Management Unit for FATA Development Authority, Mineral Department FATA Development Authority, FATA Reforms Commission, Operationalization of Project Management Unit for FATA Emergency Rural Roads Project, Study for extension of laws to FATA, Community Driven Development. PCNA Evaluation conducted by M&E Directorates of KP and FATA in collaboration with the Georgia State University and University of Peshawar Two PCNA implementation monitoring reports published and disseminated by M&E Directorates 10 year Governance Plan Extension of Laws in Reference to Draft Local Government Regulation 2012 Institutional Needs Assessment of M&E Directorate FATA Results Oriented Analyses and Technical Assistance to stimulate dialogue among policy makers for reforming the national systems for community driven development delivery in FATA FATA Public Expenditure and Financial Accountability Assessment FATA Development Indicators and Household Survey (FDIHS) PCNA Evaluation FATA accelerated Development Strategy 2016 to 2025 Revenue sharing for FATA study (A case for fair & symmetric application of equalization scheme) Twenty (20) capacity building trainings for FATA M&E Directorate (300 officials trained including 30 women) Thirty Seven (37) workshops with 591 officials trained (including 45 women) notably on M&E, impact assessment, FM, CDD and social accountability. Pilot Testing of Internal Audit with FATA Secretariat Establishment of Development Partners Forum 20 coordination meetings held with donors 565 citizens (with 31 women ) involved in citizen pre-budget consultations 23

39 FATA: Component 2 - Rapid Response Facility Funding of priority 5 Governance interventions funded: Governance Operationalization of FATA Tribunal (Phase I and II), Establishment of Anti-corruption interventions hotline for the Governor s Inspection Team, Establishment of FATA Reforms Commission and facilitation for delivery of reforms report, Strengthening of PFM systems in FATA and Piloting of internal audit at FATA Secretariat Implementation of reform actions 2 reform actions implemented: Operationalization of FATA Tribunal Phase 1 and Phase 2, Establishment of Anti-Corruption Hotline at Governor s Inspection team Phase 1 Balochistan: Component 1- Strengthening of the PCNA ISUs, provision of TA and Institutional Building of government departments / directorates BDNA ISUs setup and operationalization Project development ISU staffed and equipped Budget disbursed and spent Coordination meetings held with donors (4), Line Departments MDTF coordination committee comprising of ISU team and MDTF project s PDs meeting monthly 9 proposals developed by the Line Departments with 6 approved by MDTF Steering Committee: Institutional Strengthening of Finance Department, Performance Management Cell at Chief Secretary Office, Strengthening of Ombudsman, of P&D Department, of Anti- Corruption Establishment,10 years governance Plan Establishment of monitoring systems for BDNA Third-party (youth) monitoring reports of health, education and domicile services in five districts (covering 773 service delivery units) Performance Management Cell established with strengthening of M&E Directorate Implementation of key analytical work and studies Technical Assistance and capacity building Institutional Needs Assessment of M&E Directorate Balochistan Citizen Consultations on Budget Allocations Five (5) capacity building trainings for Balochistan M&E Directorate (39 officials) Fourteen (14) workshops with 225 officials trained (including 14 women) including on procurement, contract management, performance management, etc Institutional strengthening of Finance Department and Balochistan Revenue Authority Performance Management Cell at Chief Secretary Office 4 meetings held with donors. Donor coordination Citizen budget Consultations Balochistan: Component 2- Rapid Response Facility Funding of priority Governance interventions 511 citizens (including 61 women) involved in consultations by the Government of Balochistan for the FY budget. 2 Governance interventions funded in Balochistan through RRF: Establishment of e-grmis (E-Grievance Redressal Management Information System) at Provincial Ombudsman Office (facilitated by Toll Free Call Numbers for complaints registration and redressal using an online automated system); Establishment of Complaints Handling System at Anti-Corruption Establishment Balochistan (facilitated by Toll Free Call Numbers and online complaints registration web portal using an online automated system) 24

40 Implementation of reform actions 2 reform actions implemented: Strengthening institutional capacity and establishing E-Citizen Grievance Redressal System at Ombudsman, Strengthening of Anti- Corruption Establishment (b) Rapid Response Facility: Lead time from submission to implementation (disbursement) (b.1) KP Proposal Submitted Date of NOL Lead Time 1 Support to operationalization of Judicial Academy Phase I 2 Strengthening the Khyber Pakhtunkhwa provincial Ombudsman office, Phase-1 May 01, 2012 June 1, days May 20, 2012 July 17, days 3 Strengthening restructuring & re-organization of Anti-Corruption Establishment (ACE) Khyber Pakhtunkhwa October 9, 2012 November 19, days 4 Peshawar High Court: Establishment of e- Citizen Grievance Redressal System 5 Support to implementation of Khyber Pakhtunkhwa Right to Information Law Establishment of Khyber Pakhtunkhwa Right to Public Services Commission Source: KP ISU February 11, 2013 October 15, 2013 March 12, 2014 February 26, 2013 December 13, 2013 April 7, days 58 days 24 days (b.2) FATA Proposal CN reviewed by ISU Date of NOL Lead Time (days) Planned Disbursement (PKR million) 1 Operationalization of FATA Tribunal Phase I (Completed 23 Jan12 6 Feb Establishment of Anti- Corruption Hotline at GIT 17 Oct12 30 Jan Strengthening of FATA Tribunal Phase II 15 Jun12 30 Mar FATA Reforms Commissions 1 May May Strengthening of PFM System (Phase-1) 6 Pilot Testing of Internal Audit Source: FATA Borrower s ICR. 4 Jan Jan Mar13 18Apri (b.3) Balochistan Proposal 1 Strengthening Institutional Capacity and Establishment of E- Citizens GRM - Balochistan Ombudsman CN reviewed by ISU Date of NOL Dec 2013 Jan 2014 Lead Time (months*) Planned Disbursement (PKR million) 1 month

41 2 Strengthening the Anti- Corruption Establishment (ACE) April 2014 June months Source: Balochistan Borrower s ICR; *data not provided in days. (b.4) Average Lead Time (days) Calendar Year KP FATA Balochistan GSP Overall Overall Source: Implementation Support Mission Aide-Memoire, February

42 c) Compliance regarding delivery of notified services in KP under RTS (January December 2015) TYPE OF SERVICE (1) Number of Citizens who applied for Services (2) Services Provided to Citizens Within Stipulated Time (3) Services Provided to Citizens Beyond Stipulated Time (4) Percentage (5) 1 Registration of FIR in Thanas 36,952 35,135 1, % 2 Issuance of FARD by Patwari 5,150 4, % 3 Issuance of Domicile Certificate 99,958 91,782 8, % 4 Death & Birth Certificates 66,724 58,833 7, % 5 Approval of Building Plans % 6 Release of Funds from district committee to local committee % 7 Jahez % 8 Educational Scholarships % 9 Disbursement of Zakat Funds to Mustahiqeen % 10 Water Connection % 11 Disposal of Garbage and Solid Waste % 12 Issuance of Learner 5,769 5, % 13 Issuance of Driving License 3,817 3, % 14 Renewal of Driving Licence % 15 Examination in OPD 1,479 1, % 16 Examination in Emergency 2,384 2, % 17 Issuance of Drug License % 18 Issuance of Arms License 11,171 8,568 2, % 19 Processing of Arms License 2,348 2, % 20 Verification of the Applicant (Arms) 2,974 2, % Grand Total 241, ,844 22, % Source: Right to Public Services Commission, KP. 27

43 Annex 3. Economic and Financial Analysis The ICRR team undertook a partial quantitative analysis to estimate benefits for a key service/activity supported by GSP in each region. For KP, they are the potential earning gains for applicants from reduced processing times of domicile certificates under the RTS; for FATA, the potential savings for litigants resulting from the speedier processing of Tribunal appeals; and for Balochistan, the productivity returns from the increase in the disposal of grievances. The analysis covers only select services of GSP that lent themselves to such analysis and for which relevant data was available (representing an estimated 9% of GSP investments). The results suggest monetary savings and positive returns. However, as non-gsp funding also contributed to the functioning and the efficiency gains in those services, the actual benefits resulting from the interventions is likely to be lower. In light of the data limitations, limited scope of the analysis and the other contributing factors, the quantitative analysis is illustrative and to be considered as only one element of the efficiency assessment. KP Estimated benefits: Issuance of domicile certificates Under the KP RTS supported by GSP, the average processing time for the issuance of domicile certificates was reduced from an average of 14 days to 9 days by 2015 (with a mandated service standard of 10 days). Domicile certificates are needed in Pakistan to apply for public schools and government jobs. For citizens who are applying for certificates so they can apply for government jobs, assuming a minimum wage rate of Rs. 15,000 per month time saved (5 days) gives a benefit of Rs. 2,500 per applicant in terms of potential wages from the government jobs they could have applied for and gotten sooner. Assuming just a minimum of 10% of applicants are seeking government jobs in 2015, (10% of 92,000 applicants) and 89% compliance over the given time period, there can be estimated savings of Rs million. With the cost of an application for domicile certificate estimated at Rs. 300 per person (form, transportation), the total cost for the successful applicants is estimated at Rs. 2.4 million. The funding provided to the RTS Commission by the GSP was Rs. 66 million (focused on supporting the establishment of the Commission and the monitoring of 5 services). However, GSP support to the RTS Commission, although forming the bulk of the TA and funding received by the Commission, is just a fraction of the overall costs involved in the functioning of the 5 services having resulted in reduced processing times. Actual benefits in relation to the costs involved are thus likely much more modest. Issuance of domicile certificates Cost Benefit (2015) Benefit/Cost Ratio* GSP: Rs. 66 million/5 services = Rs. 13,2 million Cost for applicants: Rs. 2.4 million Rs million (US$ 195,000) Source of underlying data (estimated wages, application costs) from KP ISU and the web. *Data not available to utilize net present values for costs and benefits. FATA 28 Comments 1.3 Benefit/cost ratio likely lower in light of significant costs covered by other parties Estimated benefits: Processing of cases at FATA Tribunal FATA Tribunal is an Appellate Court put in place given the existing Subordinate Court operating on a parttime basis and with a great backlog of cases). GSP provided initial financial and technical support (estimated at Rs 11.5 million) to the Tribunal to operationalize it as well as have it function in a first phase and until resources were allocated by the FATA Secretariat from other sources. Lack of economic data on FATA is a challenge for forecasting the benefits but assumptions can help in drawing the impact of reduced processing times. The lawyers fee for litigants ranges between Rs 25,000-75,000 per case. The minimum monthly wage rate notified by the Government of Pakistan is Rs 15,000 per month. On average a case in Subordinate court

44 takes 400 days and an appeal in the Tribunal takes 92 days. Using the minimum wage rate, transportation, lodging and average of lawyers fee, each litigant is likely to incur a loss of Rs 300,000 on his/her earnings over the litigation period 1 (or Rs 610 per day). With a reduction in the average time for appellate decisions to 56 days (in 2015), there is a potential saving of Rs 21,906 from the reduction in litigation period for each litigant appealing. Based on the existing number of cases/appeals disposed of in the Tribunal in 2015, that is a potential total saving of Rs 4.75 million in Cost Benefit (2015) Benefit/Cost Ratio* Comments FATA Tribunal: GSP: Rs million (68% of initial resources available to Tribunal)/4 (years): Rs million (2015) Potential savings: Rs 21,906 per litigant Rs 4.75 million total (US$ 45,000) 1.6 Other sources contributed to setting up and functioning of FATA Tribunal so benefit/cost ratio likely lower Source of underlying data (estimated wages, transportation and application costs) from FATA ISUs and the web. *Data not available to utilize net present values for costs and benefits for ratio calculation. Balochistan Productivity gains: Support to Balochistan Ombudsman For the Balochistan Ombudsman, based on the type of data available, the team aimed to quantify the productivity gains from the disposal of cases. There was an annual increase in disposal from 1296 cases (in 2012) to 1395 (in 2015), which suggests greater efficiency assuming fixed investments (including a fixed number of 103 Ombudsman Officers). 2 That is a productivity gain of 7.6% (7.6% more cases processed). The case disposal in 2012 and 2015 corresponds respectively to a disposal of 108 cases and cases per month and 1.05 and 1.13 cases per employee per month. With the monthly wage of an Ombudsman employee estimated at Rs 56,067.18, we have respectively the cost of each case at Rs 53,397 (2012) and Rs. 49,617 (2015), with the gain in productivity (reduced cost of processing a case) resulting in estimated savings of 3,780 Rs per case of Rs million for Cost/Output (2012) Cost/Output (2015) Productivity gains Balochistan Ombudsman Wages: Rs 7.14 million Cases: 1296 Wages: Rs million Cases: % per employee Rs 3,780 per case Rs million (2015) (US$ 50,000) Source of underlying data (estimated wages, number of Ombudsman employees) from Balochistan ISU. 1 Calculation/estimated losses from FATA Borrower s ICRR. 2 For years in between (2013, 2014), there were even more cases disposed of, in relation to While there is a slight decrease in disposal in 2015 i.e. 1395; however this can be attributed to a decline in backlog of cases (i.e in 2012 to 2071 in 2015). 29

45 Annex 4. Results Frameworks and Indicators Changes Results Framework And Monitoring (Original PDO) Program Development Objective: Program Development Objective is to Improve the delivery efficiency of the PCNA program through institutional strengthening and support of the Khyber Pakhtunkhwa and FATA, while responding to priority needs of the PCNA governance program. Component Activities Outputs Intermediate Outcome Indicators 1. Capacity Enhancement Outcome: Khyber Pakhtunkhwa & FATA are able to manage and coordinate implementation of PCNA short and medium term interventions and design longerterm interventions 2. Rapid Response Funding window Outcome: Improved quality of services through implementation of urgent reforms 1. PCNA Implementation Support Unit in Khyber Pakhtunkhwa and FATA set up and operationalized 2. Overall 10 year Governance Program designed 3. Five Project concept notes developed for interventions 4. Development Partners Forum established 5. Systems established to monitor progress of PCNA program 6. Five studies completed and disseminated 7. Ten workshops conducted 1. At least 5 Governance interventions funded in a timely manner 2. At least 2 reform actions implemented 1. PCNA ISU in Khyber Pakhtunkhwa and FATA successfully fulfill their mandate 2. Governance program endorsed by stakeholders 3. Development Partners Forum convened as needed 4. Key stakeholders successfully included in consultations 5. Government and Donor consensus on implementation of findings and recommendations of PCNA 1. Government s capacity to implement key interventions enhanced 2. Improved transparency and accountability of the PCNA Program Original and Revised PDO and Output Indicators Original Indicators Revised Indicators (AF 1) Added/Dropped/Revised PDO Indicators (AF 2) Component 1 1. Capacity Enhancement (KP & No change* Dropped. FATA are able to manage and coordinate implementation of PCNA short and medium term interventions and design longer-term interventions) 2. PCNA Implementation Support Unit in KP and FATA Set up and operationalized No Change Indicator dropped 3. Overall 10-year Governance Program designed 4. Five Concept Notes developed for interventions 5. Development Partners Forum established 6. Systems to monitor progress of PCNA program established in KP & FATA Overall 10-year Governance Program designed in collaboration with stakeholders, completed and approved for KP and FATA, and submitted to stakeholders for Balochistan. At least one pilot intervention per region completed. Project Preparation: Eight concept notes developed for interventions in KP and FATA, two Project concept notes for interventions in Balochistan developed and submitted to the MDTF Steering Committee Development Partners Forum established in KP and FATA Systems established to monitor progress of PCNA program in KP & FATA, and governance program in Balochistan Indicator revised and changed to Intermediate Outcome Indicator (see below) Indicator dropped Indicator dropped in light of IO on Forum convening (revised, see below) Indicator changed to Intermediate Indicator (see below) 30

46 7. Five Studies completed and No Change* Indicator dropped disseminated 8. Ten Workshops Conducted Training workshops * Indicator dropped 9. New indicators 10. Balochistan ISU established* Indicator dropped Component 2 (RRF) 1. Rapid Response Facility (Improved quality of services through implementation of urgent reforms) 2. Five Governance interventions funded in a timely manner 3. Two reform actions implemented (in KP and FATA) No change* One Governance enhancing activity funded in a timely manner in Balochistan, five governance enhancing activities in KP and FATA No change New indicators 1. Number of completed KP, FATA and Balochistan pilot projects (at least one per region)* Dropped Indicator dropped Indicator dropped Indicator dropped 2. Two diagnostic studies in Balochistan Indicator dropped 3. Project prepared for financing and Indicator dropped implementation (at least 4)* New Indicators (PDO level) 1. Number of grievances addressed by Peshawar High Court 2. Number of grievances addressed by KP Ombudsman 3. Number of grievances addressed by Balochistan Ombudsman 4. Average processing time for selected public services in KP 5. Total number of transactions for the main public service targeted by the project in KP 6. Average processing time for selected public services in FATA 7. Total number of transactions for the main public service targeted by the project in FATA 8. Proportion of funds directed to priority sectors increased based on citizen consultations in Balochistan 31

47 9. Reduction of the average number of days from proposal to start of implementation of programs under the Rapid Response Facility 10. Investments mobilized in targeted development sectors responding to the PCNA program in KP and FATA Sr. # Original Intermediate Indicators 1. PCNA ISU in Khyber Pakhtunkhwa and FATA successfully fulfill their mandate 2. Governance program endorsed by stakeholders Original and Revised Intermediate Outcome Indicators Added/ /Dropped/Revised Intermediate Indicators (AF 1) PCNA ISU in KP, FATA and ISU Balochistan successfully fulfill their mandate Governance program for KP and FATA endorsed by stakeholders. Governance reform program in Balochistan submitted for approval by stakeholders Added/ /Dropped/Revised Intermediate Indicators (AF 2) Indicator dropped as it was deemed new PDO and IO level indicators in this table better capture its dimensions Governance frameworks to lead reforms developed 3. Development Partners Forum convened as needed 4. Key stakeholders successfully included in consultations 5. Government and Donor consensus on implementation of findings and recommendations 6. Government s capacity to implement key interventions enhanced No change No change Indicator dropped Indicator dropped Number of donors and CSOs participating in development forums hosted by KP Participants in consultation activities during project implementation (number) Participants in consultation activities during project implementation female 7. Improved transparency and accountability of the PCNA Program Improved transparency and accountability of the PCNA program in KP and FATA and of the development program in Balochistan 8. Project beneficiaries Of which female (beneficiaries) Indicator dropped to simplify RF and as captured in some of the new PDO indicators Beneficiaries: i) Number of beneficiaries (citizens) ii) Number of Female beneficiaries 9. Number of monitoring systems developed *Indicator not mentioned in AF1 RF but appears in ISRs subsequent to addition of Balochistan (ISRs 3, 4, 5, 6 between AF1 and AF2). 32

48 Annex 5. Detailed Assessment of Achievements by Outcome Indicator a) Summary of Achievement by Indicator (Original and AF1 RFs)* Indicator Baseline Target (2015) Target Achieved (at closure) Improved delivery capability of the governments for PCNA and BDNA 1. PCNA ISUs in KP and FATA and BDNA ISU fulfill their mandate 2. Governance program endorsed by stakeholders 3. Development Partners Forum convened as needed 4. Key stakeholders successfully included in consultations Indicator Status Comments N/A Yes Yes Achieved. The mandate of the PCNA and BDNA ISUs is to coordinate assistance and provide support to Line Departments and Directorates in the regions on the programs implementation. The GSP directly financed the set-up and operationalization of the three ISUs, which were staffed, equipped with O&M budgets executed. Throughout the life of the Project, they organized coordination meetings with donors, civil society representatives and academia. They assisted Line Departments/Directorates in implementing PCNA/BDNA through support in FM, procurement, communication, and M&E as well as in the development of proposals for MDTF and GSP funding and relevant studies and organized trainings for government officials. Annex 2 provides details on these outputs. No Yes Yes* Partially Achieved. This indicator assessed the design of a 10-Year Governance Program for each region coordinated by the ISUs and their endorsement by the relevant stakeholders. Both the governments of KP and FATA developed Governance Programs reflecting their respective priorities in consultation with a diverse range of stakeholders. The KP Governance Program was approved by the Working Committee chaired by the Additional Chief Secretary and published. The FATA Governance Program was approved by the senior management of FATA Secretariat and published. *Balochistan Governance Program has been developed but needs revision and has not been endorsed. The endorsement of other relevant stakeholders might be needed such as other donors at the Development Partners Forum as targeted by the Project when designed. No Yes Yes Achieved. Through the coordinating support of the ISUs, the FATA and KP Secretariats established in each province a Development Partners Consultative Forum consisting of technical working groups (development partners and civil society) that meets at least twice a year. During the Project, 5 such forums were held in KP and 6 in FATA. In KP, this engagement has resulted in the development of a Strategic Development Partnership Framework and the KP IDS. Balochistan participated in a Development Partners Forum in Islamabad hosted by KP and will need further support to.establish its Development Partners Forum. No Yes Yes Achieved. GSP defined key stakeholders as CSO, local NGO, parliamentarians, civil servants, communities and successfully as their active participation and views reflected in the analytical work & recommendations. Consultations took 33

49 5. Government and Donor consensus on implementation of findings and recommendations place during the preparation of the 10-Year Governance Programs. In KP, 2,851 citizens (482 women) were reported included in consultations. In FATA, 1,458 citizens were included (109 women). In Balochistan, 2,744 citizens (644 women) were included. In FATA and Balochistan, pre-budget citizen consultations took place with evidence of citizen preferences affecting the budget allocations made in Balochistan (see specific indicator on pre-budget consultations in Balochistan below). N/A N/A N/A Not rated This indicator refers to consensus on the implementation of PCNA recommendations. It is not rated, as it was achieved prior to project start and therefore dropped at AF1. Improved quality of services through implementation of urgent reforms 6. Government s capacity to implement key interventions enhanced None provided 7. Improved transparency and accountability of the PCNA Program None provided 6 Governance Interventions funded; 2 reform actions implemented; 13 Governance Interventions funded; 10 reform actions implemented; Achieved. GSP provided implementation support to PCNA/BDNA priority governance interventions identified by Line Ministries through the ISU-managed RRF (approving and funding interventions meeting the criteria of the Steering Committee). The RRF funded 13 governance interventions (6 by KP, 5 by FATA, 2 by Balochistan) while at least 6 were targeted. 10 interventions were implemented (6 in KP, 2 in FATA, 2 in Balochistan) while at least 2 were targeted. See list of interventions in Annex 2 (Outputs by Component). Unclear Achieved GSP aimed to promote transparency and accountability of the PCNA program through RRF-funded activities. Although several activities contributed to this objective, the original RF did not provide indicators to measure this IO. RRF supported the implementation and strengthening of GRMs such as the Anti- Corruption Hotline under the Governor s Inspection Team in FATA and the Right to Information Law and Ombudsman in KP. GSP support also allowed KP and FATA M&E Directorates to develop and disseminate PCNA implementation monitoring reports and an evaluation of citizens perceptions of public service delivery. The latter showed that public awareness campaigns about GSP facilitated interventions had a positive impact on citizens perceptions of the quality of public services, especially those exposed to violence in KP and FATA. In addition, the GSP helped build fiduciary management capacity in KP and FATA that contribute to the accountability outcomes. * Although most the original indicators were dropped at AF2 to focus on outcome oriented indicators, the ISUs continued to collect data on the various outputs underlying these indicators including for Balochistan that the Project still aimed to produce and that contributed to the Project s overall outcomes. 34

50 b) Summary of Achievement by Indicator (Final RF AF2) Indicator Baseline Target (2015) Target Achieved (at closure) Reduction in the Average Processing Time for selected Public Services 1. Average Processing Time for Public Services by FATA Tribunal (Days) 2. Average processing time for public services (delivery of Domicile Certificate in KP) (Days) 721 (5,768 hours) Indicator Status Comments Achieved. FATA Tribunal is an Appellate Court that the Project helped established in light of existing Subordinate Court operating on a part-time basis with a great backlog of cases. GSP provided financial and technical support to operationalize the FATA Tribunal and install a case management system while training 58 officers to expedite the processing of cases. An analysis of the decided cases shows a sustained decrease in average time for appellate decisions from 721 days at Project start to 56 days (in 2015) (target of 93) Achieved. GSP helped establish the RTS Commission and provided support to monitor the delivery of 5 services initially for which delivery standards (mandated processing time) had been established under the RTPS Act (with penalties imposed on government officers for non-compliance). GSP also helped set up a Performance Management System dashboard. The 5 services saw a reduction in average processing times as well as high and increasing compliance with the standards. A total of 20 services are now notified under RTPS and compliance rates for a majority are also high. It is also reasonable to infer that prior to the RTPS Act and Commission which GSP helped establish, processing times were higher and compliance rates lower, thus the need for RTS. Total number of transactions for the main public service targeted by the project in KP and FATA 3. Total number of transactions for the main public service targeted by the project (Number, per month) (FATA Tribunal) 4. Total number of transactions for the main public service targeted by the project (Number, total) (delivery of Domicile Certificate in KP) Grievances registered related to delivery of project benefits addressed Achieved. Slightly exceeded, but discrepancy between baseline and target reported between AF2 RF2 and last two ISRs (the ISRs subsequent to AF2). 1,791 55,000 92,030 Achieved. Exceeded, but other factors contributing to volume of transactions. The support provided to Domiciles Services in KP was indirect through monitoring of compliance with delivery standards (mandated processing time for this service is 10 days). The functioning of this service was dependent on other resources including Government s own resources which could be a major contributing factor affecting the ability to process transactions and their volume. The baseline and target were not clear in the AF 2 RF and the baseline not provided in the last ISR, but the Project s last ISM reports 1,791 requests in the first report provided by RTPS. It can be inferred the much higher volume of transactions experienced from first report is due to the now existence of a RTPS Act and Commission providing and monitoring a compliance framework for the delivery of this service (citizens more encouraged to apply). 35

51 5. Number of grievances addressed by Balochistan Ombudsman (per month) 6. Number of grievances addressed by KP Ombudsman (per month) 7. Number of grievances addressed by Peshawar High Court (per month) Achieved. GSP supported strengthening of Grievance Redress institutions through the RRF including the Balochistan Ombudsman. For the latter, GSP supported strengthening institutional capacity and setting up Citizen Facilitation Desks in districts as well as establishing an E-Citizen GRM to provide additional channels through which citizens could submit grievances (calls, online, sms). The yearly disposal of Balochistan Ombudsman grievances exceeded the initial monthly and annual disposal targets (although it is not clear whether the targets were set appropriately at AF2, as there was initially no baseline and they largely surpassed AF2 targets). There was a significant decrease in the average time of grievance disposal from 614 to 169 days Achieved. In KP, equipment and capacity building was funded for the Ombudsman Office to set up a public GRM. The yearly disposal of KP Ombudsman grievances exceeded the initial monthly and annual disposal targets. 3,224* Achieved. An E-Citizen GRM was established at the Peshawar High Court which exceeded its annual target of disposed grievances (from 2013 when the Peshawar High Court started receiving GSP support). *Prior to 2013, only a manual GRM existed with a total of 3,224 addressed between 2009 and The pre-project baseline is therefore an aggregate of grievances over a 4-year period, not a yearly or monthly baseline (KP ISU). Proportion of funds directed to priority sectors increased based on citizen consultations in Balochistan 8. Proportion of funds directed to priority sectors increased based on citizen consultations in Balochistan % Achieved. Following GSP-supported citizen pre-budget consultations in FY14-15 in Balochistan (with 1804 citizens consulted including 589 women), an increased share of Annual Development Plan funds was allocated to priority sectors, especially education (4.4%), health (1.2%) and irrigations (5.6%). An attribution analysis estimated the increase of allocations resulting from the consultations to be 1.73% (target of 1%). Balochistan plans to institutionalize the consultations. Reduction in the average number of days from proposal to start of implementation of programs under the Rapid Response Facility 9. Reduction of the number of days (average) from proposal to start of implementation of programs under the Rapid Response Facility Partially Achieved. Investments mobilized in targeted development sectors responding to the PCNA program in KP and FATA Under the original PDO, GSP initially aimed to reduce the lead time (average number of days from proposal to start of implementation) of RRF funded programs for 5 governance interventions (with initial target lead time of 15 days per Operations Manual). Under the final RF, this indicator was to assess the average yearly lead time across all approved RRF programs (with a target of 65 or less days in 2015). Although the reported average lead time remained under that target and decreased from 52 (2012) to 42 days (2015), this masks years in between (such as in 2013) when the average increased from the year before as well as differences between regions. While the average lead time in KP remained below the target throughout the Project, it was above the target in FATA in 2013 (there were no proposals in 2014 and 2015). Data only available for one year in Balochistan (2014) due to its later addition. Averages thus skewed to reflect KP data primarily. See table b.4 in Annex 2. 36

52 10. Investments mobilized in targeted development sectors responding to the PCNA program in KP and FATA (US$ Billion) Achieved. Under this indicator, the GSP aimed to help mobilize investments for the implementation of the PCNA program through support to the ISUs to manage and coordinate the PCNA and liaise with donors. Since the launch of the PCNA in 2010 and through the government/ donors partnerships frameworks established, total investments have reached US$ 2.6 billion against the government target of US$ 2.8 billion (and a Project target of US$1.5 billion by June 2015), with 69% coming from the Government of Pakistan and 31% from its development partners. Source: GSP ISR, June * Last reported dates for achieved values range from June 2015 to May

53 Annex 6. Supporting Program Background Information a) Selection process (criteria/scoring form) for projects supported by GSP (TA and RRF) Criteria Ownership/ Sustainability: Productivity: Relevance: To Beneficiary Department To Development Objectives Duration of project and aptness of Deliverables: Innovative and nonduplicating: Participation Fairness and equity: Gender responsiveness: Project Costing and Eligibility: Description PCN reflect Government s ownership (For instance, reflected in the Needs Assessment exercise or CDS of GoKPK or PCNA) Proposed project is aligned with the Government & development partners supported prioritization & sequencing. Proposed project will introduce new systems/strengthen business processes/ enhance the capacity of HR of client institution Project addresses the needs identified by the client and is relevant to its objectives Proposed project is aligned with PCNA SO, MDTF strategy and GSP Objectives The project implementation time is realistic and practical. (duly taking into account various administrative approvals & procurement processes) Are deliverables appropriately identified; is the sequencing of deliverables correct The proposed project does not duplicate similar initiatives undertaken by other donor funded project/program (Refer to Development Coordination matrix, Table 7 of OM). Proposed project is demand driven and has been developed adopting a participatory approach. Evidence of stakeholders commitment demonstrable in CN Proposed project does not have a skewed geographical coverage or focuses on a particular official/group /department etc For Training Projects: The individual nominated has not received more than 2 short term (up to one month) trainings & skill development initiatives in the past 1 year. Exceptions shall be justified in writing & approved by the ACS. The project reaches out to male and female beneficiaries equitably Total Project Cost is within stipulated limit i.e. resources are so allocated that 5 projects can be financed from GSP funds amongst other TA support Appropriate costing has been carried out taking into account all anticipated costs for TA The project does not include ineligible items The project expenditure complies with categories/components stipulated in EPP Score 1= Lowest 4= highest First come first The project follows first come first serve basis criterion Serve Total Score Percentage Score (Points scored by the Project/total score, multiplied by 100) % Source: Governance Support Project KP and FATA, Project Operations Manual, March

54 b) Long list of potential Governance interventions identified by PCNA Component 1 Khyber Pakhtunkhwa o Assessment of existing PFM at Khyber Pakhtunkhwa using Public Expenditure and Financial Accountability (PEFA) Framework and implementing reform strategies, in light of PEFA identified risks and mitigation action plan. o Design of robust and accountable PFM to ensure efficient and transparent delivery of public services through design & pilot implementation of financial management systems within the national systems for participatory and community based planning, budgeting, spending, accounting and auditing suitable to address PCNA sectors specific needs. o Design of a Gender Responsive Budget Strategy in Khyber Pakhtunkhwa. o Reforms to boost capabilities of the administrative system and address identified risks to peace building including: (a) setting the public service delivery baseline through design of citizen services report cards (b) first round of citizen report cards prepared for key five public services and sharing survey results with line departments and civil society followed by preparation and publication of improvement of services action plan (c) develop citizen charters in conflict affected districts/agencies with special focus on women participation (d) develop mechanisms to institutionalize citizen feedback client survey, legal aid clinics and para-legal network (e) roll out of the implementation of pilots program across Khyber Pakhtunkhwa where citizen-local government liaison is strengthened, Citizen charters prepared and displayed. o Adopting a community driven development approach under 'local government act', amongst other including to: (a) introduce mechanisms for participatory and community based planning, budgeting, spending, accounting and auditing in the conflict context of Khyber Pakhtunkhwa to transform perceptions of poor responsiveness and trust by engaging the civil society in processes for transparency and financial oversight through providing technical support to the Local Government Department of Khyber Pakhtunkhwa to develop mandatory participation mechanism; (b) involve communities in development work by introducing small community based spending schemes to instill sense of ownership and possibly counter radicalization; (c) design of civil society led rural support programmes that involve jirgas and CBOs and roll out of civil society led rural support programme in conflict affected areas involving jirgas and CBOs in at least 50% of the Khyber Pakhtunkhwa districts and frontier regions. o Strengthening frameworks for access to information. o Conduct comprehensive study on formal and informal accountability and anti-corruption institutions, including assessing capacity, complementarities, shortcomings and gaps in Khyber Pakhtunkhwa. o Assessing Anti-Corruption reform programmes in Khyber Pakhtunkhwa, followed by development and implementation of formal and informal institutions based reform program for Anti-Corruption Establishment (ACE) in Khyber Pakhtunkhwa. o Assessing State needs to strengthen the representation and inclusion of civil society in the identification, planning, targeting, tendering, recruitment, procurement, implementation, and monitoring process of projects. o Identifying activities that also empower communities to be aware of and demand their rights, and to hold authorities and implementing partners to account. o Conducting rapid assessment of capacities needed to support a large PCNA program in Khyber Pakhtunkhwa at different levels of administration and develop a functional capability in Khyber Pakhtunkhwa Government to support peace-building and reconciliation efforts. The assessment, to be conducted in the first 6 months of PCNA implementation phase, will feed into a capacity development strategy. 39

55 o o o o Preparatory work for Institutional and Civil Service Reform, amongst others including: (a) criteria for ensuring merit-based and transparent selection of key positions within the public service should be developed, initially for the crisis affected areas; (b) recruitment procedures must include gender sensitivity and ensure women are involved at all levels of civil service; (c) review of the adequacy of civil service salaries; (d) initiating a program to recruit civil servants on incentive contracts into priority functional areas. It could initially be piloted in the crisis affected areas; (e) capacities of public administration and civil society must be enhanced in the short term for basic service delivery and stabilization activities in Khyber Pakhtunkhwa; (f) reforms to ensure citizens voice is heard, in addition to participation in planning, transparency and accountability (g) identify and test mechanisms as well as internal processes for participation, to ensure that community contributions are considered, and rejection explained where not, in addition to planning for internalization of tested effective mechanisms and processes over the 30 months horizon. Adoption of a far-reaching e-governance system, including to install e-governance at the district levels. Conducting rapid assessment of IT, automation and other operational systems specific capacities needed to support a PCNA program in Honorable PHC and develop a functional capability in Honorable PHC to support peace-building and reconciliation efforts. The assessment will feed into a capacity development strategy. Assessing further strengthening operational capacity needs of the Honorable PHC and Courts at District and Sub-Divisional Headquarters in Khyber Pakhtunkhwa, in particular the crisis affected areas and provide support to address the needs for PCNA specific activities. Commissioning a comprehensive fact-finding report and the preparation of TORs of staff for the following purposes: (a) assess and determine the various activities and work load of the Record Rooms, in coordination with the Honorable District and Sessions Judges concerned; (b) determine the schedule of activities, and cost-estimation related to automated indexing, registration and scanning of documents consigned in the said Record Room; and (c) identify the relevant software to support the automation of Honorable PHC and other Courts at the District and Sub-Divisional Headquarters. FATA o Assessment of existing PFM at FATA using Public Expenditure and Financial Accountability (PEFA) Framework and implementing reform strategies, in light of PEFA identified risks and mitigation action plan. o Identifying institutional and human capacity development needs of the PA set up in FATA agencies. o Assessment and develop of a strategy addressing the following issues amongst others: (a) adopt a community driven development approach to local government in FATA; (b) strengthen the representation and inclusion of civil society in the identification, planning, targeting, tendering, recruitment, procurement, implementation, and monitoring process of projects. (c) assist legislators to better understand their role and responsibilities in peace building and enhance their understanding of the need for more civic engagement and oversight as an integral element of local government; (c) transform perceptions of poor responsiveness and trust by engaging the civil society in processes for transparency and financial oversight; (d) involve communities in development work by introducing small community-based spending schemes to instill sense of ownership and possibly counter radicalization; (e) clarify the role, responsibility, function and capacity for elected councilors in FATA, in addition to their capacity development to assume the roles and responsibilities, effectively; (f) identify mechanisms to re-establish the credibility and trust in traditional decision making bodies like jirgas as they are likely to remain an important interface between the administration and the public; (g) capacity development for 40

56 o o o o o o o civil society to equip it to liaise between communities and government; and (h) facilitate institutionalizing civil society representation and community participation mechanism at community, union council, agency and FATA Secretariat levels, so as to link policy-making to community level activities. Capacity gap analysis and measures for enhancing capacities of civil society in the short and medium term for basic service delivery and stabilization activities in FATA. Creation of a meaningful space for dialogue, participation and oversight, in parallel to the improvements in service delivery and systemic reform. Strengthening the framework for citizen s access to information. Designing mechanisms to: (a) strengthen the representation and inclusion of civil society in the identification, planning, targeting, tendering, recruitment, procurement, implementation, and monitoring process of projects; (b) identify and promote Community Driven Development (CDD) interventions that complements community empowerment, to be aware of and demand their rights, and to hold authorities and implementing partners to account. Building local capacity through programs on: (a) legal literacy which raise awareness around key rule-of-law concerns; and (b) PFM which facilitates and disseminates social audits, especially of PCNA related interventions. Conducting rapid assessment of skills and capacities needed to support a large PCNA program in FATA at different levels of administration and develop a functional capability in FATA to support peace-building and reconciliation efforts. The assessment will feed into a phased capacity development strategy and include amongst others, capacity gap analysis and measures for enhancing capacities of public administration in the short, medium and long term for basic service delivery and stabilization activities in FATA. Assessing and developing a strategy addressing, amongst other the following issues: (a) SAFRON administrative functions delegation to FATA Secretariat; (b) adoption of a far reaching e-governance system; (c) review of the adequacy of civil service salaries; (d) initiate a program to recruit civil servants on incentive contracts into priority functional areas, initially to be piloted in the crisis affected areas; (e) reforms in civil service to ensure citizens voice is heard, in addition to participation in planning, transparency and accountability; (f) support FATA Secretariat in identifying and testing mechanisms and internal processes for community participation and to ensure that community contributions are considered, and rejection explained where not. Component 2 Khyber Pakhtunkhwa o Adoption of a far reaching e-governance system, including installing e-governance in Honorable Courts at District and Sub-Divisional Headquarters in Khyber Pakhtunkhwa. o Enabling online access to Statutory and case laws, and the computerization of records of judicial cases at the prioritized Judicial Rooms, as a pilot initiative. o Develop and test fast track funds flow mechanism including cash grants, community driven development (CDD) interventions, block grants and any others, for pilot testing and aiming to be internalized into the national PFM systems within the PCNA 30 months time horizon. o Introduce mechanisms for participatory and community based planning, budgeting, spending, accounting and auditing in the conflict context of Khyber Pakhtunkhwa. Such mechanisms are to be tested in the short term for roll out and finally internalization within the national systems over the 30 months period. o Initiate peace, security and stabilization dialogue engaging the community, political representatives and leadership, civil and military officials and cross section of the civil society at all levels, including dialogue on the role of jirgas and their interface with formal governance structures. 41

57 o o o o o o o o o Assist legislators to better understand their role and responsibilities in peace building and enhance their understanding of the need for more civic engagement and oversight as an integral element of local government. Establishment of the Office of the Khyber Pakhtunkhwa Ombudsman, including: (a) develop plan to fully operationalize the Ombudsman s Office; (b) develop plan to establish citizen complaint centres (CCC) and provide technical support to the offices for roll out at district level; (c) provide technical assistance to design district and agency-wise Ombudsman Offices. Establish effective public grievance redress (PGR) mechanisms, supported by systems at community level which provide direct feedback to Khyber Pakhtunkhwa Secretariat about the quality of service delivery, and help to re-establish trust and confidence in the state. Map legislative committees and identify areas for capacity development including amongst others: (a) to assist legislators to better understand their role and responsibilities in peace building and enhance their understanding of the need for more civic engagement and oversight as an integral element of particularly the local government through support program for the Provincial Assembly; (b) training of Legislative Committees on public account, budget and local governments; (c) strengthening of the Assembly Secretariat to strengthen legislative committees; (d) support Provincial Assembly Secretariat to issue budget documents reflecting the results of audit and corrective allocations and measures; (e) support the Public Accounts Committee for enhanced oversight and legislative scrutiny. Strengthening and empowerment of the Chief Minister s Inspection Team to undertake investigations and audits of all districts including PATA and frontier regions Strengthening the Auditor General of Pakistan s Khyber Pakhtunkhwa specific external audit functions to perform effective audit functions throughout the province and particularly the crisis affected areas. Convene all stakeholders conference to agree on a Khyber Pakhtunkhwa Anti-Corruption Strategy that would also outline how to implement the Right to Information Act to increase transparency and accountability. Job specific training and generic skill development across the Khyber Pakhtunkhwa Secretariat and the field formations, particularly in the crisis affected districts and frontier regions through: (a) relevant country study visits for the senior management (b) project design, contracting, monitoring, oversight and reconstruction policy management oriented skill development for the middle as well as senior management (c) out of country need specific trainings for senior and middle management with the commitment to serve Khyber Pakhtunkhwa Secretariat and / or the field formations for a minimum of three years (d) fee re-imbursement and required skill specific allowances for all in Khyber Pakhtunkhwa Secretariat and field formations, encouraging own skill development in the required skills sets. The initiative will include full time, part time, online and distance learning programs with the commitment to serve Khyber Pakhtunkhwa Secretariat and / or the field formations for a minimum of three years; (e) strengthening operational capacity of the Khyber Pakhtunkhwa Secretariat and the field formations through provision of required support tools, including computers, printers, scanners, photocopiers and other IT support; (f) initiating a program to recruit skilled civil servants on incentive contracts into priority functional areas, including consultants, in the short term to support senior managers on PCNA specific issues, and till the civil services compensation rules regulations are rationalized. FATA o Initiate program to: (a) recruit skilled civil servants on incentive contracts into priority functional areas, including consultants, in the short term to support FATA senior managers on PCNA specific issues, and till the civil services compensation rules regulations are rationalized (b) develop criteria for ensuring merit-based and transparent selection of key positions within the 42

58 o o o o o o o o o o o public service in FATA. Recruitment procedures must include gender sensitivity and ensure women are involved at all levels of civil service. Planning, developing and pilot testing mechanisms for : (a) fast track funds flow including cash grants, community driven development (CDD) interventions, block grants and any others, for pilot testing with a view to be internalized into the national PFM systems within the PCNA 30 months time horizon. participatory planning, budgeting, spending, accounting and auditing, in the conflict context of FATA to transform perceptions of poor responsiveness and trust by engaging the civil society in processes for transparency and financial oversight. Involve communities in development work by introducing small community-based spending schemes to instill sense of ownership and possibly counter radicalization. Strengthen the representation and inclusion of civil society in the identification, planning, targeting, tendering, recruitment, procurement, implementation, and monitoring process of projects. Establish effective public grievance redress (PGR) mechanisms, supported by systems at community level which provide direct feedback to the FATA Secretariat about the quality of service delivery, and help to re-establish trust and confidence in the state. The development of mechanism to incorporate this feedback into planning, and the communication of this back to communities is critical. Facilitate creation of awareness and existence of avenues for dialogue, combined with citizens' ability to contribute usefully and the government's interest and openness in hearing them. Strengthen and further empower the Governor s Inspection Team to undertake investigations and audits of all tribal agencies. Assess regulatory reform and facilitating mechanisms to extend the jurisdiction of the Federal Ombudsman to FATA along with public information campaign to explain its functions and effectiveness. Convene all stakeholders conference to: (a) agree on FATA Anti-Corruption Strategy; and (b) outline how to implement the Right to Information Act to increase transparency and accountability. Initiate consultations and facilitate discussions and dialogue with the PAs, FATA Secretariat, FATA Parliamentarians and FATA representative civil society individuals and organization on multiple reform issues, amongst others including: (a) lifting restriction on political parties activity (b) build consensus on further FCR reform and status issues; (c) review the PA's power of arrest, detention, right to bail and exclusion of women and minors from collective responsibility under the law. Job specific and generic skill development across the FATA Secretariat and the field formations in the agencies through: (a) relevant country study visits for the senior management; (b) project design, contracting, monitoring, oversight and reconstruction policy management oriented skill development for the middle and senior management; (c) need based, out of country trainings for senior and middle management with the commitment to serve FATA Secretariat and / or the field formations for a minimum of three years; (d) fee re-imbursement and required skill specific allowances for all in FATA Secretariat and field formations, encouraging own skill development in the required skills sets. The initiative will include full time, part time, online and distance learning programmes with the commitment to serve FATA Secretariat and / or the field formations for a minimum of three years; (e) strengthening operational capacity of the FATA Secretariat and the field formations through provision of required support tools, including computers, printers, scanners, photocopiers and other IT support; and (f) identify and address the institutional capacity development needs of the Political Agents Office (policy, regulations and human capacity) for medium term reform 43

59 Annex 7. Grant Preparation and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Lending/Preparation Naseer Rana Lead Operations Officer, TTL GCFKE Sher Shah Khan Senior Governance Specialist SASGP Mahwush Mushtaq Malik Counsel LEGES Martin Serrano Senior Counsel LEGES Puteri Watson Governance Officer SASGP Rehan Hyder Senior Procurement Specialist SARPS Robert Bou Jaoude Program Manager - MDTF SASPK Satyendra Prasad Senior Governance Specialist SASGP Shafqat Manzoor Mirza Program Assistant SASEP Waseem Kazmi Financial Management Specialist SARFM Supervision/ICR Sher Shah Khan Senior Public Sector Specialist, TTL GGO18 Christian Berg M&E Specialist NON BANK STAFF Christine I. Wallich Senior Advisor OPSFN Clelia Helena Rontoyanni Senior Public Sector Specialist GGO18 Maha Ahmed Monitoring and Evaluation Officer SACPK Mehreen Saeed Communications Officer SACPK Rehan Hyder Senior Procurement Specialist SARPS Roland Lomme Senior Governance Specialist SASGP Syed Waseem Abbas Kazmi Financial Management Specialist SARFM Immanuel Frank Steinhilper Public Sector Specialist GGO18 Mirza Imran Baig Program Assistant GGODR Muhammad Omer Lodhi Team Assistant GGODR Irum Touqeer Extended Term Consultant SASGP Jean Mazurelle Consultant SACPA Hamid Khan Consultant GGODR Nosheen Razzaq Gill Consultant GGODR Marieta Fall Public Sector Specialist GG018 Simon Carl O Meally Senior Public Sector Management Specialist GG017 Ayesha Ahmad Consultant, ICR GGODR Annie Vincent Consultant, ICR GGODR (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD (including travel and No. of staff weeks consultant costs) Lending (Total) 2.15 $57, Supervision/ICR (Total) $ 1,181,

60 Annex 8. Beneficiary Survey Results None conducted in context of ICRR but several implemented during Project implementation. See section 3.6 Summary of Findings from Beneficiary Surveys. 45

61 Annex 9. Summary of Grantee's ICR Khyber Pakhtunkhwa Achievement of objectives KP government has adopted a number of GSP supported reforms. Key among these are legislations on access to information rights and the right to public service 3, the institutionalization of evaluation in planning systems for effective public expenditures and continuance of the Ombudsman office. This adoption of Project initiatives generates a high likelihood of their sustainability. During the project implementation stage, an evaluation of selected GSP interventions concluded positive impact for governance communication on citizens perceived quality of public service delivery. 4 Currently, the pace for reforms in the governance sector has picked up steam with more services being added to the list of public services offered at tehsil (sub-district) level. 5 GSP s flagship activities in KP operationalization and strengthening of Right to Information Commission (RTIC), Right to Public Service Commission (RTPC), e-citizen Grievance Redressal System (e-cgrs) have not only created awareness amongst the citizens of KP, but have created demand for the continuity of governance sector reforms. Support to KP Secretariat in strengthening PFM, establishing the Anti-Corruption Establishment, strengthening the ombudsman office, and support in improving M&E of public sector programs can prove to be catalysts for curbing corruption and prudent public investment management. GSP managed to achieve all the targets set for its outputs and outcomes. It should be acknowledged that GSP commenced in a testing time when governance in KP was perceived to be resting on quick sand (and not responsive to citizen needs).the performance of GSP in terms of assessment against the indicators stipulated in the first results framework (EPP 2011) has been remarkable. Against certain indicators there was 100 percent achievement, while in other cases the target set for GSP was surpassed. A citizen perception survey, designed with support from Georgia State University, was conducted from end of 2014 to March The sample includes 3,741 unique respondents drawn from 34 villages in KP for face to face interviews. The final sample includes data from 30,473 individuals drawn from 46 distinct tehsils across KP. The results show generally increased levels of satisfaction with public services- more profound for individuals exposed to conflict than those not exposed to conflict. The results also show an increase in perceived improvement in justice system and system of governance for people exposed to conflict. Moreover, the results also suggest increased confidence in local institutions such as district courts and municipality. Efficiency The comparison of Program targets with achievements offers evidence on program efficiency. The Program targeted 10 concept notes, 5 studies and 10 workshops in the stipulated resources but the ISU managed to complete 24 concept notes, 25 workshops with 11 governance interventions rolled out. As intended, the Program has produced some positive changes in the form of improved skills and knowledge 3 The KP Right to Information Act 2013 was enacted on 5 th November, 2013, and the Khyber Pakhtunkhwa Right to Public Service Act 2104 was enacted on 15 th January, Under this study, the impact of public awareness messaging campaigns for RTI, RTPS, Anti-Corruption Hotline (ACH), and e-citizen Grievance Redressal System (e-cgrs) was gauged. 5 Under the Right to Public Services Act 2014, initially, public service delivery for two services (Domicile and Fard) was initiated at two Districts. Now the services have been extended to 19 districts and also include issuance of birth/death certificates, Building Plan registration and registration of First Information Report (FIR). 46

62 among civil servants. An unanticipated impact (positive externality), already materialized, is that the Program has created visibility and positive reputation within the KP secretariat at both senior management and civil servant levels. Risks In overall terms, the risk to the development outcome is assessed to be substantial, although it could be substantially reduced with timely, appropriately designed and institutionally integrated remedial actions. Post-conflict implementation environment notwithstanding, several aspects lead to optimistic prospects for maintaining the Project s development outcomes, in the short term. The Project has gained ownership of the KP Secretariat and other implementation agencies, who have continually engaged with the Project, taking forward many of its initiatives toward mainstream adoption. Some of the initiatives undertaken are moving toward institutionalization and likely to be maintained as part of government s systems. However, four main risks remain: i) the risk that the ISU modality of high functioning systems may not be institutionalized unless specific steps are taken in the subsequent project; ii) the risk that GSP gains in evidence based policy and planning may not sustain over time; iii) the risk that people s engagement with public sector systems may not receive continued importance; iv) the risk that gains in citizen trust may dissipate if expectations of higher quality services are not met and the inertia of reforms in service delivery is not maintained. Lessons Learnt Important lessons were derived from the citizen report card baseline survey results: (a) efforts at development and service delivery should be enhanced; (b) communication efforts should be increased to provide information on government efforts to citizens to build positive perceptions leading to citizen trust in state. Delay in opening of assignment account with the National Bank of Pakistan was probed by World Bank and recommended remedial measures that could be undertaken to ensure timely opening of accounts. The findings and recommendations of the citizen perception survey offer lessons that would benefit the subsequent projects in timely start up. Experience of project staff in following the Operations Manual developed for GSP elevated the importance of effective training of the project staff on its use with the overall aim of inducing greater efficiency in the program operations. GSP highlighted the importance of robust management decision support system and its benefits to the program. The Project s management could adopt the working backwards problem-solving strategy involving the finance team to alert the management on parked funds and the matter could be taken up with the procurement and program teams to identify the causes of delays. A collaborated effort in this regard would result in instituting a reporting framework that would support the decision making process and enhance management s oversight. GSP implementation experience shows that follow up projects will benefit from instituting a mechanism for assessment of the impact of its development interventions. For e.g. pre and post evaluation of the training workshops would allow to address the gaps in the training related initiatives and at the same time offers an opportunity to obtain feedback of the beneficiaries for subsequent or related training requirements. 47

63 FATA Achievement of objectives FATA Secretariat has adopted a number of GSP initiatives by incorporation into government policy. Key among these is acceptance of position paper on National Finance Commission for tabling at the commission sessions, institutionalization of evaluation in planning systems for effective public expenditures and continuance of FATA Tribunal. GSP s flagship activity in FATA the operationalization and strengthening of the FATA tribunal has not only created awareness amongst the citizens of FATA but have created demand for the continuity of the Tribunal, through its performance in expediting the long drawn out litigations. GSP achieved important milestones in implementing initiatives like legal literacy to create higher impact of FATA Tribunal and public engagement with GIT. Additionally, the justice sector reforms, have not only reduced the litigation costs with timely decisions but will contribute towards economic productivity. The research paper on the revenue sharing mechanism is an advocacy tool that created awareness amongst the policy makers with regard to the increased share in revenues for FATA. Support to FATA secretariat in strengthening PFM, the establishment of anti-corruption hotline and support in improving M&E of public sector programs can prove to be catalysts for curbing corruption and prudent public investment management. The assessment of GSP achievements is meaningful when viewed in the perspective of prevailing conditions in FATA. GSP managed to achieve all the targets set for its outputs and outcomes against the first RF. It also needs to be acknowledged that GSP took place in circumstances where the governance framework has been continually debated and questioned, the overall governance framework that is neither representative nor transparent or responsive to citizen needs. Therefore, the achievement of the GSP overall objective should not be correlated with to all-is-now-well in FATA. Against certain indicators there was 100 percent achievement while in other cases, the target set for GSP was surpassed. A citizen perception survey, designed with support from Georgia State University, was conducted from end of 2014 to March The sample includes 1,814 unique respondents drawn from 19 villages in FATA for face to face interviews. The results of the survey show that education is positively correlated with desiring democratic local governments. It is also correlated with desiring political rights for citizens. Similarly, younger age is correlated with desiring local government in place of indirect administration. These findings are important as they light up the way forward for institutional reforms in FATA. They also highlight that the cause of peacebuilding will be served by assuaging people s desires in achieving representative local government and institutions that guarantee rights for the people. Efficiency The comparison of Program targets with achievements offers evidence on program efficiency. The Program targeted 10 concept notes, 5 studies and 10 workshops and 23 concept notes, 34 workshops and 6 governance interventions were rolled out. As a result, the Program has produced some positive changes (directly and intended) in the form of improved skills and knowledge among civil servants. An unanticipated impact, already materialized, is that the Program has created visibility and positive reputation within the FATA secretariat at both senior management and civil servant levels. Risks The risk to maintaining the outcomes is assessed to be substantial, although they could be mitigated with the creation and consolidation of institutional practices as well as a structured and organized institutional development plan through GPP. Despite the post-conflict implementation environment, several pointers lead to optimistic prospects for maintaining the Project s development 48

64 outcomes, in the short term. The FATA Secretariat and other implementation agencies have continued engagement with the Project, taken forward many of its initiatives toward mainstream adoption and actively collaborated for commissioning a subsequent phase of the Project. The risks in the medium term are however substantial and the same as in KP in addition to the risks linked to maintaining the impetus for institutional innovations and internalizing the multi-sectoral approach espoused by GSP. With change in leadership of FATA Secretariat and slide in priorities, the risks to outcomes become substantial. Implementation of governance reform may be impacted by the much-discussed FATA reforms. If political organization of FATA is folded into the KP Province, it may result in major reorganization of FATA Secretariat and its systems. Lessons learnt Important lessons derived from the citizen survey results are: (a) meaningful reforms should be implemented to institute representative system of governance at the agency level; (b) system of justice should be reformed in line with people s preferences to assuage their demands for rights; (c) efforts at development and service delivery should be enhanced to engage citizens with the state; and (d) communication efforts should be increased to provide information on government efforts to citizens to build positive perceptions leading to citizen trust in state. In addition to the lessons similar to the ones drawn in KP in terms of GSP implementation (need for a robust management decision-making system, for an institutionalized mechanism to assess the impact of development interventions, etc ), the GSP encountered the following issues during implementation which need to be avoided in the GPP: micro-management by the Bank in day to day running of the project, burdensome requirement of NOL for each and every activity and subsequent delays in granting it and insufficient capacity building support both to the FATA Secretariat and ISU team. Balochistan Achievement of Objectives The Government of Balochistan (GoB) has initiated a number of GSP supported reforms and has been committing funds out of its own Public Sector Development Programmes (PSDPs) to supplement GSP funding. This commitment generates a high likelihood of sustainability of these initiatives, even though over the short to medium term, these may require continued support through the follow up project. GSP achievement is meaningful, keeping in perspective the existent multidimensional political and security environment in the province. The inherent flexibility of the Program provided the GoB an opportunity to increase its outreach to priority areas considered most important for restoring citizen s trust in the state and also improving its capacity. The performance targets of GSP in terms of assessment against the first results framework (EPP 2011) were achieved on most indicators and not achieved on two. Against certain indicators there was 100 percent achievement while it fell short on the development of a 10 year Governance Program (needs to be redrafted and has not been endorsed) and the establishment of a Development Partners Forum that convenes as needed. GSP. Support to Balochistan secretariat in strengthening PFM, establishment of the ACE, strengthening the ombudsman office and support in improving M&E of public sector programs can prove to be catalysts for curbing corruption and prudent public investment management. However, for the reforms to self-perpetuate, strategic communication with beneficiaries is crucial. A Citizen Report Card survey was designed and conducted by the Institute of Management Science (IMS) in eleven districts of Balochistan to get insight about the service delivery of the Ombudsman office. The survey included a representative sample of around 5,500 individuals (4,911 49

65 respondents) randomly selected from 11 districts throughout the province. The results show generally low level of penetration of strategic communication regarding the services provided by the office of Provincial Ombusman Balochistan as most of the citizens were unaware of the office and the services it provided. Moreover, the majority of the small proportion of aware citizens, did not know how to contact the ombudsman office. Seen in this backdrop, one can see why it is important to follow up governance improvements with the solid information dissemination campaigns for making sure that citizens (including marginalized groups) are aware of the existence of these services and know how to access them. The results of CRC are important in their own right for evidence-based evaluations and planning. Efficiency The comparison of Program targets with achievements offers evidence on program efficiency. Within the stipulated resources, the Program targeted: 10 concept notes which included 2 concept notes for Balochistan against which 6 concept notes were rolled out by the ISU Balochistan; 5 studies, in aggregate for the whole GSP against which 3 alone were completed by the ISU Balochistan; and 10 workshops, in total for the GSP against which 36 alone were conducted by ISU. As intended, the Program has produced some positive changes in the form of improved skills and knowledge among civil servants. An unanticipated impact (positive externality), already materialized, is that the Program has created visibility and positive reputation within the Balochistan secretariat at both senior management and civil servant levels. Risks The risk to maintaining the project outcomes is assessed to be substantial. The Project has gained ownership of the Balochistan Secretariat and other implementation agencies, who have continually engaged with it, taking forward many of its initiatives toward mainstream adoption. Moreover, these stakeholders have actively collaborated for commissioning a subsequent phase of the Project. The Project also carried out communication campaigns to highlight its governance and transparency initiatives for evaluation as well as for outreach purposes, but their effectiveness needs to be enhanced. The risks identified in Balochistan are similar to those in KP. They could be mitigated with creation and consolidation of the institutional practices as well as a structured and organized institutional development plan through GPP. Lessons learnt They include further training project staff on the use and utility of the Operations Manual in order to induce greater efficiency in the program operations, and as in the other regions, instituting a mechanism for assessment of the impact of the development interventions for follow-on projects. Another important lesson of implementation in Balochistan is that the program can benefit immensely with key management positions filled on a full time basis. Continuous engagement of the ISU coordinator (not just on a part-time basis) with implementation related activities is of utmost importance for robust management decision support and adoption of timely remedial measures. The Balochistan ISU also shared the following lessons and needs with the ICRR mission: Future interventions and likely outcomes to be more strategic; Effective phase-out strategy for HR and Resource Allocation; Translation of outcomes into policies, strategies and sector plans; Effective communication strategy for proper propagation of programs and enhanced citizens engagement; Exhaustive Mid Term Reviews ensuring improved alignment with BDNA and overall governance priorities; Ensuring follow up projects other than from project resources. 50

66 Annex 10. Comments of Cofinanciers and Other Partners/Stakeholders None. 51

67 Annex 11. List of Individuals Consulted and References a) Individuals Consulted World Bank 1. Mr. Anthony Cholst Operations Adviser, SACPK 2. Mr. Ousman Jah Program Coordinator (MDTF), SACPK 3. Mr. Sher Shah Khan Senior Public Sector Specialist, GGO18 4. Mr. Naseer A. Rana Lead Operations Officer, GCFKE 5. Mr. Syed Waseem Kazmi Senior Financial Management Specialist, GGO24 6. Mr. Rehan Hyder Senior Procurement Specialist, GGO06 7. Ms. Clelia Rontoyanni Senior Public Sector Specialist, GGO18 8. Ms. Mehreen Saeed Communications Officer, SAREC 9. Ms. Maha Ahmed Rural Development Specialist, GFA06 (Former M&E Officer) 10. Mr. Hamid Khan Consultant, GGO18 Government of Pakistan 11. Mr. Eazaz A. Dar Joint Secretary, Economic Affairs Division Government of FATA 12. Mr. Mian Zaki ullah Coordinator, GIS Directorate of M&E 13. Mr. Hisbullah Program Officer, Directorate of Projects FATA 14. Mr. Muhammad Sajjad Registrar, FATA Tribunal 15. Mr. Sang e Marjan Chairman, FATA Tribunal 16. Muhammad Atif Nazir Member, FATA Tribunal 17. Mr. Muhammad Zahoor Coordinator, ISU FATA 18. Mr. Muhammad Amir Contract Administration Specialist, ISU 19. Mr. Munem Khattak Financial Management Specialist, ISU Government of Khyber Pakhtunkhwa 20. Mr. Azmat Hanif Orakzai Chief Commissioner, Right To Services Commission 21. Mr. Israr Khan Director General, M&E Directorate Mr. Muhammad Ayaz Chief Economist, Planning & Development Department. KP Secretariat Mr. Khalid Ilyas Coordinator, ISU KP 24. Mr. Faisal Shahzad Financial Management Specialist, IU 25. Mr. Pir Imran Monitoring and Evaluation Specialist, ISU Government of Balochistan 26. Mr. Muhammad Wasay Tareen Provincial Ombudsman Balochistan 27. Mr. Manzoor Sarparah Director General, Monitoring & Evaluation Directorate Mr. Sheryar Taj Secretary, Planning & Development Department, Balochistan Civil Secretariat Mr. Mujeeb ur Rehman Coordinator, ISU 30. Mr. Dawran Baseer Financial Management Specialist, ISU 31. Mr. Yaser Sultan Procurement Specialist, ISU 52

68 b) End Notes/References 1 Estimated GDP growth for KP declined to around 3% in 2007/8, as compared to Pakistan s GDP growth rate of 4.2% for the year 2007/8, with consistently far higher rates of unemployment than the rest of Pakistan ( particularly high among young men aged 15-30, the main resource pool for militant recruitment). FATA does not have the legal status of a province and while the region was not included in the surveys then due to the instability, socio-economic indicators were likely worse there and the lowest in the country. Governance Support Project, Emergency Project Paper (EPP), PCNA had the following four objectives through 9 sectors; i) build responsiveness and effectiveness of the State to restore citizen trust; (ii) stimulate employment and livelihood opportunities; iii) ensure the delivery of basic services; and, iv) counter radicalization and foster reconciliation. Post-Crisis Needs Assessment (PCNA) The 9 sectors were Good Governance, Rule of Law, Agriculture and Natural Resources, Non-Farm Economic Development, Education, Health, Social Protection and Strategic Communications. 3 The PCNA governance priorities emphasized a reformed judicial infrastructure, anti-corruption and more responsive service delivery. The PCNA recommendations, when implemented fully would require 2.7 billion USD to be channeled to KP and FATA. Capacity building of Government line departments was needed to ensure the effective management of these finances in the regions and implement the recommendations. 4 Outcome area 4.2 under Pillar 4 Improving Security and Reducing the Risk of Conflict of CPS FY Original amount US$ 6 Million was increased to US$ 8.75 Million (US$ 2.75 AF-1). The project had disbursed US$ 1.45 Million, 24% of the total amount of US$ 6 Million within five months of implementation. 6 Project Paper AF 1, September 6, Two output indicators were revised and re-introduced as IO indicators: Overall 10 year Governance Program designed and Systems to monitor progress of PCNA program established in KP & FATA. 8 Preparation of separate successor Governance and Policy Projects (GPP) in KP, FATA and Balochistan under MDTF Round II. 9 Project restructurings info could not be entered directly in the portal datasheet because effectiveness date was not recorded in the portal (and could not be entered after closing), so team manually entered info in annex GSP Emergency Project Paper (EPP) August Ibid. 12 For e.g. an internal auditor was required, which went beyond the usual Bank requirement for governance projects, but was deemed necessary in light of the fiduciary capacity and risks. 13 GSP EPP, The PCNA priority governance areas are: Political and Legal Reform, Civic Engagement and Oversight, Access to Affordable, Timely and Equitable Justice, Anti-Corruption, State Provision of Public Security Services, Strengthening Local Government Institutions, Service Delivery and Social Inclusion Measures 15 GSP Implementation Status Report, June 2016; PCNA ISU Progress Report, GSP KP, June 2016; FATA Borrower ICR, November All these actions were Legal Covenants with two outstanding ones for each region by the time of the first Project Implementation Status Report (ISR), GSP ISR 1, June GSP ISR 5, June MTR Aide-Memoire, November GSP Emergency Project Paper (EPP), August Ensuring a Results Orientation During Project Preparation Under OP/BP 8.00 Rapid Response to Crises and Emergencies, Guidance Note for Staff. World Bank, April The recommendations had not yet been fully implemented at MTR, MTR Aide-Memoire, November Average Processing Time for Public Services by FATA Tribunal and Number of grievances addressed at Balochistan Ombudsman. 23 Reports generated from these dashboards have been shared with the relevant government departments for action and in Baluchistan, such monitoring led to the re-opening of three schools marked as close in a report. 24 End of Project Evaluation, GSP FATA, Implementation Support Mission Aide-Mémoire, June Government of Pakistan Vision Document The pillars are: (i) rehabilitating physical infrastructure, (ii) strengthening law and order, (iii) expanding government service delivery, (iv) re-activating and strengthening economy, and (v) strengthening social cohesion and peace building (with financing of PKR 62 billion for two years), FATA Sustainable Return and Rehabilitation Strategy, March

69 28 At the request of the government of Balochistan, the BDNA was developed by Bank in Especially between the RF at AF2 and subsequent ISRs (baselines and targets). 30 Projects under both components chosen under same selection criteria in Annex 6. It appears if proposals were involving Line Departments and Directorates per se strengthening, they were funded through Component Although most indicators from the first two RFs were dropped, the ISUs continued to track a number of related outputs that contribute to the overall PDO. 32 FATA has done a pilot mapping of public assets of water, sanitation and livestock facilities complemented by beneficiary feedback data. Balochistan has implemented third party monitoring (youth) of the delivery of health, education and domicile certificates with the perception of their quality collected from residents of a few districts 33 Voices of the Governance Support Project, KP and FATA Secretariats and ISUs. 34 The BRA received support from GSP and other donors and since its inception in late 2015, has already been able to collect significant amounts in tax revenues. Prior, the province relied on the federal government for the provincial collection of taxes with very low levels of collection. The GPP will provide more targeted support to the BRA. 35 Most funded sectors include Education (2471 % above target), Health (951%) and Infrastructure (10%) whereas least funded sectors include Social Protection (87% below target), Strategic Communications (85%), Agriculture and Natural Resource Management (73%), governance( 50%), Non-Farm Economic Development (47%) and Rule of Law (37%). ISM Aide-Memoire January Including the youth in light of their critical role in the peace and stability in the region. 37 Interviews with Balochistan government stakeholders during ICRR mission. 38 Although the Project Operations Manual suggests a prescribed lead time of 15 days from submission of proposal/concept note to decision/funding, in actuality, the Project pursued a reduced implementation time from year to year. By that measure, the yearly average lead time went from 52 (in 2012 when the RRF was first implemented) to 42 days (in 2015 when it last approved interventions (2015), but this masks years in between (such as in 2013) when the average lead time increased from the year before. 39 For e.g. in FATA, the government decries the time taken for the proposal for the establishment of the Anti-Corruption Hotline (106 days) and the strengthening of FATA Tribunal (110 days). In contrast, the operationalization of the Tribunal proposal was processed in 15 days. FATA Borrower s ICR. 40 In the case of the fifth service, FIR registration, a lesser change was seen in the overall average processing time (one day) with no change in the compliance rate of 96%. 41 RTS compliance data across all notified services obtained from KP Right to Public Services Commission for 2015 from which rates were calculated. See table c in Annex 2. It is reasonable to infer that prior to the RTPS which the GSP contributed to setting up, overall average processing time for the notified services was higher and compliance rates lower and thus the need for the RTPS which led to the reduced processing times. 42 The FATA Tribunal is an appellate Court instituted to complement the existing Court that operated part-time with a great backlog of cases. FATA Borrower s ICR/ ISU Progress Report 43 Or 750 hours Per Project s last ISR. 44 From 2013 when the Peshawar High Court started receiving GSP support. Prior to 2013, only a manual GRM existed with a total of 3224 addressed between 2009 and ISM Aide-Memoire January 2016 and KP USU. 45 As explained by the Project. However, ultimately the RRF in Balochistan funded 2 main interventions with 3 more interventions approved through the same process but under Component 1 with the delineation not clear. 46 Based on the Independent Evaluation Group (IEG) guidelines combination of these specific Relevance, Efficacy and Efficiency ratings. 47 Implementation Support Mission Aide-Memoire, February Conducted in partnership with the Bureau of Statistics FATA and the Institute of Development Economics of Pakistan. 49 Such as moving from input monitoring to output /outcome and results monitoring, and Impact evaluations. As mentioned, KP and FATA M&E Directorates in partnership with academia conducted a first impact evaluation of awareness campaigns on perceptions of PCNA interventions. 50 Interviews of stakeholders during ICRR mission. 51 GSP Implementation Support Mission, Aide-Memoire, January 2016; Voices of the Governance Support Project, GSP KP and FATA Secretariats and ISUs; Balochistan Borrower s ICR, June In KP for e.g. the financing of the PPRA, RTI and RTS Commission entities has been gradually brought onto the province s regular budget. For instance, the KP PPRA was financed from GSP resources for its first year of operation (FY2013/14), and in the following year the Government provided 71 percent of the entity s budget. In FATA, the Government has taken up the costs of operating FATA Tribunal. 53 GSP ISR, December GSP EPP,

70 55 It was a mandatory requirement of the Ministry of Finance for project offices to open assignment accounts at the National Bank of Pakistan. 56 Implementation Support Mission Aide-memoire, May For e.g. the Secretary of P&DD had to in practice, sanction all the ISU related hirings and payments. 58 Balochistan Borrower s ICR, June For example, Line Departments and Directorates not attaching the necessary documentation contributed to payment and procurement delays. 60 This is also a recommendation of the joint annual review of the MDTF portfolio in Pakistan led by the Departmnet for International Development (DFID) of Also recommended at MTR and the MDTF annual reviews. 62 KP, FATA and Balochistan learnt from each other during implementation such as in the development of the procurement plans or of their respective ICT-based monitoring dashboards, instead of reinventing the wheel. 55

71 56