Afghanistan Third Party Monitoring

Transcription

1 Afghanistan Third Party Monitoring THE WORLD BANK February

2 The Afghanistan Program The Context: The Afghanistan program consists of both IDA and trust fund resources; Total active portfolio: $3.2 bn The Afghanistan Reconstruction Trust Fund (ARTF) started in 2002 to provide recurrent cost financing for the Afghan Government s operating budget, funding non-security wage bill and O&M; ARTF: $7.9 bn paid in since 2002 by 33 donors, ARTF: $7.4 bn committed, $6.3 bn disbursed to gov for recurrent and investments; active portfolio $2.4 bn ; IDA: $2.7 bn committed, $2.3 bn disbursed; active portfolio $767m Current active portfolio: 31 projects (11 IDA, 15 ARTF and 5 co-financed). 2

3 Why TPM? Afghanistan faces several challenges, including: Low capacity environment; Difficult access; Challenging security context; Rough terrain, complex geological structure, disperse and remote project sites; A large trust fund program with high donor scrutiny and pressure for accountability. Need to strengthen supervision, access data from the field, report to donors. The ARTF Supervisory Agent key characteristics: Carries out asset verification, quality assurance, and data mapping of national infrastructure projects; Has conducted a total of 11,239 site visits since the program start in 2001; Provides the Bank with strengthened outreach to all 34 provinces of Afghanistan, including highly insecure areas; Focused on infrastructure quality and safeguards issues in education, rural roads, irrigation and community development; Monitored 4 projects worth a total of US$2.25 billion; Increasing economies of scale/program efficiencies: Cost per site visit: Year I: $4600, Year IV: $2060 Developed feedback systems to ensure deviations are rectified and construction quality is improved, resulting in increased ministry appreciation of monitoring value; Collaborative efforts with the ministries helped to improve program performance and foster a 3 mutual push for better projects.

4 To Consider before/during Preparation? Be clear on objective but ensure room for flexibility: Afghanistan SA started off with infrastructure, but now branching into other areas such as safeguards and FM. Initial focus was on reporting to donors but shifting the focus towards quality assurance during construction and beneficiary input; TPM by theme: The objective, focus and use of the reporting differs for infrastructure, financial management, livelihoods, social and env. Safeguards etc. Different objectives: TPM monitors social outcomes in Health. Audit vs??: Normally TPM refers to an audit agency focusing on fiduciary aspects. Afghanistan focuses on quality assurance, verification and access to data different skills needed, different contracts; 4

5 Continued What agency to hire?: The credibility of the third party is critical but will often depend on context. In International contractor was needed to ensure donor buy-in; Government more comfortable with local counterparts; The right set-up, sophisticated systems were needed; in Afghanistan compromise with international contractor, expat oversight and local counterparts Contracting: The Bank has very limited experience with Bank contracting directly this takes preparation; TPM ownership Gov/Donors/Bank?: Ideally we want gov and Bank to co-own to ensure oversight and buy-in and therefore sustainability of investment; Donors would like to own but sensitive - we need balance in what can be shared. 5

6 The SA - Monitoring Approach(es) The database, CMORE, is central to everything: data collection, monitoring, security tracking, and information processing; Smartphone technology allows for engineers to work in austere and dangerous areas; Risk management and mitigation is done through multiple tiers of control including GPS verification of staff locations and projects; Information is published in an online database and mapping application searchable catalogue that geospatially displays data; Detailed inspection reports, analysis of identified patterns, recommended actions shared with Bank and ministries; Actions to address deviations are agreed and correct systems to prevent similar problems from reoccuring and increase impact on the ground. Two Methodologies: Third Party Monitoring SA field engineers travel to sites and complete comprehensive reports. Pros: High level of detail, skilled engineers on site, complete coverage of the project including social and other data,; Cons: Cost to travel to site, sites are seen once or twice per year. Local Monitoring Community member selected to send daily information to SA/ministry in Kabul. Engineer provides oversight; Pros: Low cost, increases community buy-in and support, information comes in near real-time, deviations and rectifications can be seen immediately. 6 Cons: Monitoring is fairly simple as the members are not engineers, limited to projects in or near the community.

7 How it works? 7

8 The Process in Pictures 8

9 Lessons Learned Cooperation and collaboration with the GOA ministries is essential and includes monthly meetings, weekly reporting, joint field visits, and joint work plans. New to both Bank and ministries, learning curve to be expected. Key challenge to develop systems; Critical to convince ministries of program value should not be seen as policing function but rather input for collaboration. The TPM contractor key in this regard to strike right balance and communicate well. TPM allows the Bank to access data from all over the country, benefitting monitoring of both quality of infrastructure, output/outcome results and project effectiveness; Multifaceted dialogue builds awareness within ministries on issues related to program monitoring and infrastructure quality assurance; Helps build capacity within the government by raising the bar on monitoring and reporting standards; Provides donors, the Bank and the government with a credible system of data collection and verification which contributes to effective policy making and ARTF credibility; Addresses individual needs within CDD, transport, irrigation, livelihoods, education and health; Economy of Scale works, more project sites lower cost per site visit. 9

10 Challenges Challenges: Expensive investment. Cost decreasing over time though due to efficiencies. Working towards cost sustainability: introducing new component for community based monitoring; Significant potential risk if data is not used by bank and ministry teams; A flexible contractor is important to ensure we can respond to emerging needs safeguards an example in ARTF program. TPM to carry on in coming years but will continue to evolve in response to needs in the program and context will have to align with government s monitoring vision as it evolves; Management of relationship with government technical team, tendency for being defensive, sometimes misunderstood as policing.

11 TPM in Transport Projects Projects: NERAP- US$232million, closed Dec 2013, covered 32/34 provinces, over 200/368 districts, over 500 contracts; ARAP-US$332 million, ongoing, covers 33/34 provinces, over 250/368 districts; over 600 rehab. contracts and over 500 maintenance contracts; Justifications for TPM: Dispersed and remote project sites, impossible for Bank team to monitor all project sites; Sub-projects in almost every province including in secure provinces, not cleared for Bank staff to visit; Expensive investment, ensure the right quality and scope; Complex engineering structures require special expertise to monitor; Feasible for such type of contract, economy of scale; TPM Scope: Visual quality evaluation of the roads, bridges and structures against defined technical specifications; Evaluation of the construction methodologies used by the contractor; Assessment of the contractor s resources on site; Review of the engineering designs against site and identifications of design faults for improvement; Visual check of the quality of construction material; Evaluation of contractor s performance; Other ad-hoc check and data collection as per the task team request. 11

12 TPM in Transport Continued.. Advantages: Over 1266 site visits by TPM engineers since inception, which otherwise not possible with traditional bank supervision; Significant Improvement in overall sub-projects quality; Each rehabilitation sub-project gets at least one visit ( 100% representation); Reliable and accessible data from the site; Opportunity for remedial actions before too late; Knowledge sharing between TPM-Consultant and government engineers; Better control of problem sub-projects and prospect for timely decision making; Flexibility for ad-hoc and special data collection from site; Screening contractors and identifying the frequent bad performers; Feasibility of special analysis of data and preparation of analytical reports; Weekly technical meetings to discuss deficiencies, deviation and agree on remedial actions 12

13 TPM in Transport Continued.. Challenges: Security risk still an issue; Cost of monitoring, sustainability; Availability of sites for monitoring; always need for a bunch of project to justify the cost; Sometimes communication between TPM and government engineers becomes an issue; culture of being defensive; Ownership of the government, not yet fully accepted and recognized; Quality of reports and findings highly depend on capacity of the monitoring engineers on site; counter check has been challenging; TPM services not mature in the country, limited number of qualified consultants, local consultants not yet developed as a result high cost for service Cost per sites is not a comprehensive indicator; should be linked to the size of investment; linked to performance of the project ( quality, disbursement, no. of deviations, on time completion of sub-projects), etc. 13

14 TPM in Education Sector Project: Education Quality Improvement Project (EQUIP) II is a USD 460 million basic/secondary education project. Coverage is nationwide (insecure and secure provinces). Project is implemented entirely using government systems right up to community level. School shuras are responsible for the majority of school construction. TPM Scope Three functions: implementation support; capacity building; external supervision. Works with clients to monitor progress with meeting outputs/objectives; Builds capacity within governments by working closely with government engineering departments; Provides due diligence in project implementation (compliance with technical standards/physical audit/safeguards compliance etc.) Justification for TPM: Country context : highly insecure, governance-challenged environment Bank capacity: limited ability to conduct field visits Community capacity: responsible for school construction but quality can be mixed; Government Capacity: lack of qualified engineers, weak monitoring systems, weak or non-existent technical standards (e.g., DRM) 14

15 TPM in Education continued.. Advantages: Complements Bank supervision and Government monitoring; Raises the bar by having high monitoring standards; Brings in new technology (satellite imagery, mobile phone apps) as well as innovations in monitoring (community monitoring). Captures knowledge across the country as well as across the Bank portfolio. Allows client/bank to analyze systemic deficiencies/trends and put in place mitigation measures. TPM can be contracted to conduct monitoring of physical assets (IRD) and/or social indicators (Johns Hopkins - health service delivery. Provides independent verification of government systems/findings ( ghost schools ); Direct and indirect capacity building of counterpart agencies. 15

16 TPM in Education continued.. Challenges: Linking physical/fiduciary audits would make system more effective; Payment to TPM (per site) is much higher than what government or Bank teams have for monitoring; Institutional model of TPM varies and some are more effective than others; Contractual accountability: TPM in the long term should be contracted by client and accountable to the client. TPM is an independent monitor but is dependent on government counterparts for information, access to project sites, etc. TPM is only as relevant as the quality of information/cooperation they get from government. Regulatory environment of the country can influence the effectiveness of TPM 16

17 TPM in Rural Development Sector Project Third National Solidarity Project: sept 2016: (US$1.5 Billion) to build, strengthen and maintain Community Development Councils (CDCs) as effective institutions for local governance and socio-economic development. Est. over 41,000 elected Community Development Councils (over 90% of rural villages) constructed over 85,000 sub-projects (US$1.5 in block grants over 10 years) TPM Scope and advantages Sample size: Total of 800 TPM sites this year (<1%) Provide assurance: 34 ARTF donors, complaints handling tool for WB team, Evolved from physical verification after completion towards enhancing quality construction during the 4 stages of construction Results: changed technical engineering designs, learning during site visits for communities and project engineers, corrections in poor construction etc.); 17

18 TPM in Rural Development continued.. Monitor the MIS High reliance on NSP s database (by WB, Donors, GoA) Serve as a means to verify the integrity and accuracy of NSPs own data collection and MIS by the FPs and NSP monitors; High risk areas (for WB, NSP monitors and FP engineers) No govt. presence in district (No District Governor & No District Police Chief, functions from other nearby village or provincial center), Located on insurgency corridor; CDC members can t disclose their identity in other villages or officials; Travel to district center is extremely risky; Travel with project documents is impossible; Permanent AGEs check posts; Regular military operations) Fiduciary monitoring A first pilot (100%); Move the focus away from communities towards the systems at the Province; FM gives greater autonomy to NSP; Contribute to greater fiduciary assurance and identify gaps for NSP IV design 18

19 TPM in Rural Development continued.. Challenges and lessons learnt: Requires full commitment from the WB Task team to manage it (provide data, context, manage relationships, regular coordination etc) Reporting needs to be backed up by local context knowledge (case of latrines, construction material, safeguards, grading for CDD type of sub-projects); Local paid monitors and the community voluntary model; Standard approach to TPM changed to suite individual project needs (Not all ministries need capacity building); Building trust and a good working relationship between the Project team and TPM agent takes time 19

20 Social Safeguards in TPM Gradual inclusion of safeguards and gender Safeguards monitoring needs: Original contract for TPM did not include safeguards and gender Development of project specific questions regarding safeguards and gender Close cooperation btw IRD and project staff to develop questions IRD s engineers were trained on social safeguards to understand issues. The key safeguards questions/output indicators: Availability and implementation status of safeguards instruments at sub-projects site, (i.e. ESMP, abbreviated RAP with strip plan) Records of consultation/training sessions with communities, including PAFs (i.e. number of consultation and training sessions to date, etc.) Functionality of grievances handling (i.e. number of grievances submitted/handled, key issues, etc.) Gender data (i.e. consultation meeting with project affected women, girls enrollment ratio, female staff, female CDC, etc.) Environmental Safeguards (i.e. safety measures, safety equipments, quarry/borrow pit management, etc. )

21 Benefits and use of data Follow up of IRD TPM findings: TPM findings Project level GRC is in place-but no grievances were recorded. No sub-project had evidence of women being consulted during planning. Land for a number of schools through community donation were not proper location. The donated land were low-lying and or flood plain areas, which are prone to flooding. Information on affected land/assets (i.e. size with number of PAFs) and land disputes information. Follow up actions Public awareness program among the affected communities regarding GRM services. (ARAP about 187 grievances submitted- Last mission findings). Recruitment of female staff to conduct consultation with project affected women. Site selection criteria developed and are being applied. Those safeguards figures, which are being reported is very helpful. In addition; Project staff will report follow up actions on TMP to WB in monthly format TPM findings help us to discuss shortcomings with project implementers during WB supervision implementation mission. The implementing partners safeguards staff are now more accountable to communities, particularly to affected people with regards to land issues.

22 Challenges & opportunities Challenges: The PMUs staff was initially a bit defensive, and hesitant to accept findings of TPM related to social safeguards. Strengthening relational btw the implementing agencies and the TMP is crucial for success. And the implementing partners shall provide response and feedback to TPM reports. Limited capacity of TPM engineers on social safeguards. Only reporting on availability of reporting data but no qualitative information on process. Opportunities: TPM has strengthened safeguards performances, and it has helped PMUs safeguards team to be more accountable to communities and WB. The TPM experiences, particularly use of smart phone has proved very useful in fragile situation.