VOLUME 2: KEY FINDINGS OF THE FIELD VISITS AND E-SURVEY

Transcription

1 PUBLIC-PRIVATE INFRASTRUCTURE ADVISORY FACILITY STRATEGIC REVIEW 2008 March 2009 Final Report VOLUME 2: KEY FINDINGS OF THE FIELD VISITS AND E-SURVEY Submitted by: Cambridge Economic Policy Associates Ltd

2 CONTENTS 1. Introduction Objectives and scope of the field visits Objectives and scope of the e-survey Structure of the report Key findings from the field visits Background information Relevance Effectiveness Additionality Impact Other issues Main results from the E-Survey Introduction Approach and methodology Respondent profile Main findings

3 1. INTRODUCTION In this second volume of the Final Report for the PPIAF Strategic Review 2008, we present the key findings and observations from the field visits to select PPIAF portfolio countries and the e-survey. The purpose of the report is to provide details on the information collated from the field visits and the e-survey, with the overall assessments and recommendations based on the views of all PPIAF stakeholders being provided in the main Final Report. Both of the tools of analysis discussed here the field visits and the e-survey had a number of objectives. To a greater or lesser extent both have sought to obtain the voice of the customer, that is, solicit the views of beneficiary client governments on the continued relevance, effectiveness, outcomes and impact of PPIAF activities. In particular, the e-survey has been a useful way of validating the views on these issues expressed by the Program Council, PMU and others earlier on in the study, by seeking the views of private sector consultants who have previously provided consultancy advice on PPIAF projects, as well as those of beneficiaries. Given the nature of surveys there is, however, typically a limit to the depth to which it is possible to investigate issues. In contrast, the field visits provided an opportunity to explore a number of issues in more depth, albeit with a narrower range of respondents. The main challenge faced by the case studies was to try and understand the extent to which PPIAF has contributed to the promotion of PPI in a given country, particularly as regards ultimate impacts, such as the delivery of new infrastructure or improved services. In addition, where appropriate, we have also consulted with other stakeholders such as donor agencies, civil society, etc during the field visits. In this introductory section, we provide a brief overview of the objectives and scope of the field visits and the e-survey, followed by a structure for this report Objectives and scope of the field visits Figure 1.1 below summarises the key objectives, areas of analysis and scope of the field visits. 1 1 These were also approved by the PPIAF Programme Council at the meeting in November 2008 in London. 3

4 Figure 1.1: Objectives and scope of the field visits Objective To identify the role of PPIAF interventions in shaping/ developing PPI in the country Areas of analysis Understand the timeline and process of PPI development in the country Assess role of PPIAF and counterfactual Scope Kenya, Uganda, India and Vietnam Interviews with Government, civil society, private sector consultants/ engineers, donors, etc Visits to four countries Kenya and Uganda in Sub Saharan Africa (SSA), India in the South Asia Region (SAR) and Vietnam in East Asia and the Pacific (EAP) were undertaken during November 2008 and January The countries chosen offered different examples of size, client and PPI market maturity, with significant clustering of completed and ongoing PPIAF projects. Two were in Africa and two in Asia; two were also PPIAF regional office locations. More details on our methodology for country selection is provided in our Inception Report. Each of the field visits involved the full participation and support of the PPIAF regional co-ordination offices. The majority of the consultations took place face to face but were supplemented by telephone and communications. The consultations lists in the four country reports add to 79 persons, of whom 44, or 58%, represent government or other public beneficiaries; PPIAF consultants added another 13 (16%) and the residual 22 (28 %), were split evenly between donors and NGOs /others Objectives and scope of the e-survey The e-survey was used as additional tool of analysis to obtain the views of a wider stakeholder community. Several observations on the relevance, effectiveness and impact of PPIAF activities were tested through the e-survey questionnaire and the responses have provided confirmations on a number of key observations. The key objectives and scope of the e-survey is summarised in Figure 1.2 below. 4

5 Figure 1.2: Objectives and scope of the e-survey Objective Additional tool of analysis to test key observations on relevancy, appropriateness and impact of PPIAF amongst the wider stakeholder community Areas of analysis Scope General questions related to testing key hypothesis of Inception findings Specific questions for country beneficiaries and consultants to understand their experience of PPIAF Specifically targeted at beneficiaries, PPIAF consultants & civil society across the focus regions/ countries The e-survey was sent out to 277 PPIAF beneficiaries and consultants, based on an list prepared by the PMU. 2 The survey was launched on the 5 th December 2008, with a deadline provided until the 31 st of December The e-survey was completed by 51 people (25.6% of total of s sent) Structure of the report Following this brief introduction, the rest of this report is structured as follows: Section 2 summarises the main observations from the field visits with summary reports for each of the four countries appended at the end of the section. Section 3 provides the main results from the e-survey, with the aggregated response numbers appended at the end of the section. These sections aim to bring out the main findings and observations from the field visits and the e-survey, with the overall conclusions and recommendations being provided in Volume 1, where these results have been assessed together with those obtained in Phase 1 of the assignment (where the immediate stakeholders PPIAF donors, the PMU, World Bank task managers, etc were consulted). The report is also supplemented by four annexes on the detailed country field visit reports and a final annex on the aggregation of responses from the e-survey. 2 The survey was uploaded to an online professional survey management tool. For further information please see surveymonkey.com 3 Initially the deadline to complete the survey was 19 th December, but the deadline was extended. 5

6 2. KEY FINDINGS FROM THE FIELD VISITS In this section, we present the key findings from the field visits to Kenya, Uganda, India and Vietnam. We first summarise some portfolio parameters for the four countries, followed by the overall feedback on the relevance, effectiveness, additionality and impact of PPIAF across the four countries. Summary field visit reports are provided as an annex to this section. More detailed field visit reports are also provided in a separate annex Background information There are a total of 79 PPIAF projects in the four field visit countries, of which 56 (71%) are completed. The sector coverage varies between countries, but in aggregate is fairly evenly spread between transport, energy, water and sanitation and multi-sector; none of the four country portfolios included any ICT projects. The breakdown of the 79 total projects is as follows: India 36; Vietnam 22; Kenya 14; and Uganda 7. By value, the country portfolios added to US$15.8m with the following breakdown: India US$6.3m; Vietnam US$4.2m; Kenya US$3.7m; and Uganda US$1.5m Relevance In all four counties the PPIAF portfolio was found to be relevant and fully aligned with local economic development strategies. Outside the water and sanitation sector, the main envisaged contribution was through enabling and supporting higher levels of growth. Beneficiaries reported favourably on how baseline studies helped shape policy reform options and consensus, followed by more specific and pragmatic projects that helped advance the PPI enabling environment. The budget size, scope and sector coverage was judged mostly appropriate, with a mix of sector and multi-sector interventions valued for independence and innovation. There was unanimous feedback that the size of fast-track quick approval projects should be increased to US$125K to reflect inflation and secure high quality consulting advice. Most beneficiaries saw the value in selective PPIAF support for high development intensity or pioneering transactions but considered this to be best provided for through a case by case approach. Retention of the enabling environment upstream focus was supported but with perhaps greater consideration given to pre transaction or PPI process support, as well as traditional policy and regulatory interventions. The need to have a real time road map to phase or sequence interventions that advance the probability of investment was underlined. In India there was a strong response to widen the sector coverage to include health and education; in Kenya and Uganda, the potential of irrigation and agro-industrial led opportunities was argued. In all four countries the decision not to commit PPIAF grant resources to the ICT sector appeared fully justified Effectiveness Some of the main findings from the field visits on the effectiveness of PPIAF operations are discussed below. 6

7 There was a marked differential capacity to implement PPI enabling reform across the four countries, with Uganda and Kenya being the weaker performers. Our observation of the outcomes of PPIAF projects in the four countries reveals that there has been quite a significant difference in the speed, degree and capacity to implement PPI enabling reform. Moreover, the field visits have also revealed that there are also substantial differences in the ease of achieving financial close in PPI transactions as well as difficulty in transactions post financial close. More details on the outcomes of specific projects in each of the countries is provided in the country visit annexes below. The benefits of moving to a country framework, in partnership with various levels of government and regional development banks/ bilateral donors, were seen as being strongest in India. In Vietnam it was thought that this was too inflexible and would increase the likelihood of capture by vested interests. In Kenya the need to remain opportunistic as well as focused on steps to gain infrastructure investment on the ground was apparent. In all four countries the need to scale up and systematise PPI across sectors to decrease transaction costs, was recognised. A common response was to move to create a central PPP unit or cell; this exists in India, has been approved in Kenya and is under review in Uganda and Vietnam. Beneficiaries and other observers all valued the quick response and non-bureaucratic delivery of independent, high quality consultancy advice. This was more effective when emerging out of continuous dialogue with a cluster of key infrastructure clients; a timely response was often needed to access discrete windows of opportunity. The Vietnam portfolio in particular confirmed the productivity of linking pilots and replicating across pioneering or lead clients. The benefits of sequencing a series of interventions, often small but highly specific, were also apparent. Indeed more generally, a cluster of tailored interventions over a period seems more effective than a larger but standalone intervention. In the case of India the biggest portfolio the question of scale and critical mass was raised. Task management by the World Bank dominates in all four countries. 4 Good task managers tended to be associated with good project outputs and thereafter better outcomes and impacts. There was however significant variation in the quality, accessibility and expertise of Bank TTLs, a resource that was shrinking rather than expanding. Some projects have significant delays and slippage. Moves to widen task management to include regional development banks, the IFC and the PMU are mainly recent and are generally supported in country. Beneficiaries in all four countries would welcome an increase in task management by the PMU although it was recognised that this may require additional nonadministrative capacity. The PPIAF PMU was broadly seen as a centre of excellence and knowledge for PPI that complemented that which existed in the Bank. In some cases PPIAF was seen as levelling the playing field in terms of the influence of the World Bank Group. 4 In Kenya 1 project was undertaken by WSP and 1 conference by the PMU but all other 14 projects by Bank TTLs; in Uganda 6 of the 7 were under the Bank but the most recent and ongoing transport intervention is locally executed; in Vietnam only 1 of the 22 is task managed outside the World Bank, in this case the ADB; and India, were the IFC and WSP both managed 2 as did the PPIAF regional PMU, but the residual 30 remained with the Bank. 7

8 Not surprisingly, where countries also host regional offices the importance of PPIAF regional projects increases and supplements the direct portfolio. Interviews also suggested that PPIAF presence and positioning was stronger in both countries which had a regional office. Over the four countries, co-financing added to US$6.75m, or 43% of the PPIAF total project value. The proportion however varied from 86% in high performing Vietnam to 40% in India and down to 11% to 15% in Uganda and Kenya. Clearly the potential differs by region and individual country and suggests different targets. PPIAF activities in the sample four countries all seem to be well co-ordinated and enhance aid effectiveness. Finally, the SNTA window provides a complimentary window and instrument to involve local capital markets and sub-national tiers of government Additionality In the absence of PPIAF, the general view was that projects would probably have gone ahead but have been delayed and typically less effective as windows of opportunity would have been lost or independence partially compromised. In Vietnam it was also suggested that the ability or willingness to undertake innovative pilots or attempt replication between or within countries would be compromised. Other Trust Funds or programmes did not have the same all year accessibility or flexibility or dedicated focus to upstream PPI in infrastructure or were tied to use of nationals or future transactions / lending operations. Problems of attribution were strongly evident in all four countries but at the same time the PPIAF intervention was typically highly valued and seen by beneficiaries as making a significant contribution. PPIAF clearly benefited from and also contributes to the influence and positioning of the World Bank in country. The adjustments in PPIAF strategy and operation after 2004 have clearly helped additionality and branding; this is particularly evident from the positive response associated with regionalisation Impact Evidence on development impact in terms of increased investment, improved access and service quality or fiscal gains is only just beginning to emerge. The strongest evidence is in energy, urban transport and the water/ sanitation sectors in Vietnam. The weakest is in Uganda and Kenya. India is somewhere in the middle with substantial outcomes and probable impacts but attribution is very difficult. This regional dimension is significant. All countries clearly valued the results of PPIAF projects and recognised the influence on policy and regulation but translating this to infrastructure on the ground is essentially work in progress. Partly this reflects the gestation periods associated with infrastructure project cycles but also project development / structuring, finance and affordability issues. PPIAF has clearly had an impact on government awareness and understanding but the capacity to procure and negotiate PPI remains fragmented and thin even in countries like Vietnam. 8

9 2.6. Other issues Not surprisingly there was strong and consistent support for beneficiary governments to be represented in PPIAF governance. With the exception of India, the awareness and use of PPIAF knowledge products was very limited and not actively promoted by TTLs or indeed the PMU. The main users sampled appear to be consultants. Key clients need to be more engaged in use of hard and soft copy materials to improve the application of existing PPI toolkits and knowledge on a more systematic basis. In Kenya and Uganda unsolicited bids are a major infrastructure problem and adversely impact the emerging PPI market. The problems associated with dealing with few or single bidders will accelerate with global recession and increased risk aversion by developers and financiers. The recession will probably increase the importance of PPI as an alternative to public funding and underpin the need to be highly competitive in bid structuring and regulation. Loss of momentum will have substantial knock on effects on pipelines. It will prove more difficult to close or refinance deals; rail and road concessions in Uganda and Kenya look even more fragile. In Vietnam the slow down offers an opportunity to lock in and deepen reforms, particularly in energy. In India, the PPI component of the ambitious five year infrastructure plan will probably become more emphasised. PPIAF s role in diversify funding sources and improving access to local capital markets will therefore become even more important in recession. 9

10 3. MAIN RESULTS FROM THE E-SURVEY This section summarises the main results from the e-survey. Responses are first summarised by issue in the first few sections of this annex, followed by a more detailed breakdown of the percentage of responses by question Introduction The main purpose of the e-survey was to gain a greater understanding of the views of the PPIAF beneficiaries as well as the private sector consultants who have worked on PPIAF assignments. In particular, the e-survey aims to assess and validate key strategic issues identified in the first phase of the assignment on the continued relevance, effectiveness and impact of PPIAF activities Approach and methodology The e-survey was compiled on the basis of the key issues emerging at the end of Phase 1 of the assignment (which culminated with the submission of the Inception Report), on the core areas of questioning for the review, namely the relevance of PPIAF, its operations and governance, and the impact of its activities. The survey was sent out to 277 PPIAF beneficiaries and consultants, based on an list prepared by the PMU. 5 The survey was launched 5 December 2008, with a deadline provided until 31 of December The e-survey was completed by 51 people (18.4% of total of s sent) Respondent profile Out of the 51 respondents to the e-survey, 23 (or 45%) were representatives from beneficiary country governments (BCG) and 28 (or 55%) were private sector consultants (PSC). 7 Respondents in general were mainly from mid-to-high level positions in the government and private sector consultancies. Main BCG respondents were officials working at various ministries, regulatory agencies, PPP units, privatisation commissions and public state companies. Representatives from African country governments account for 42% of total BCG respondents, reflecting the weight of the region in the PPIAF portfolio, followed by representatives of the EAP region with 17%. 8 5 The survey was uploaded to an online professional survey management tool. For further information please see surveymonkey.com 6 Initially the deadline to complete the survey was 19 th December, but the deadline was extended. 7 Note that 2 of the respondents were also consultants to the World Bank and 1 is currently working with the World Bank. 8 The remaining of the BCG representatives are distributed as follows: 13% in the LAC region, 13% in the ECA region, 10% in the MENA region and 5% in the SAR region. 10

11 Over 80% of the respondents 9 have been users or beneficiaries of PPIAF products and services. Surveyed PSCs have in general enjoyed more interaction with PPIAF over 85% 10 of them have been involved in two or more PPIAF activities compared with surveyed BCG officials, who in 56% 11 of the cases have been involved in one PPIAF activity only. With regards to the project size, over 75% of e-survey respondents have been involved in projects up to US$250,000 and only 10% of them being involved in projects with a budget over US$500,000. Finally, 46% 12 of respondents have participated in multi-sector activities reflecting the large proportion of these activities in the PPIAF portfolio. On the other extreme, only 6% of respondents have been involved in PPIAF activities in the telecom sector Main findings Continued relevance of PPIAF 98% of all respondents are of the view that PPIAF initiatives are significant and appropriate given the context of the overall aid architecture. The overall view is that PPIAF fits well in the current Private Sector Participation (PSP) development context and responds to the market needs. The box below provides a quote from one of the respondents on PPIAF. PPIAF continues to be a thought leader, resource, and promoter of relevant PPP solutions and related regulatory policy. Without PPIAF's support to the enabling environments for PPP in partner countries, governments would continue to struggle to deal on an ad hoc basis, project by project. Geographic focus Over 40% 14 of respondents suggested that PPIAF should focus its activities on least developing countries, independent of their location, while 36% considered that PPIAF s main geographic focus should be Africa. Respondents supporting an Africa focus recognised that this is the least developed region in the world, where infrastructure needs (in particular energy and transport) are also the greatest. A quote from one of the respondents towards this end is provided in the box below. The PPIAF should focus in the Africa region as it continues to be underserved in PPP framework assistance. The EBRD provides adequate support to European countries; ADB to Asian countries. The AfDB is still in the process of determining which PPP related services it will support, in the medium term it will not be a major player 9 41 respondents out of 50 that replied to the specific question. 10 Or 22 out of the 25 respondents to the specific question. 11 Or 13 out of the 23 BCG respondents to the specific question. 12 Or 22 out of the 48 respondents to the specific question. 13 For more detail please refer to the annex below in this section out of 42 respondents to this question. 11

12 The remainder 21% of the respondents believe that PPIAF s focus should not be in the African region, citing the reasons that PPIAF should focus where: i) transactions are; ii) government willingness to reform is greatest; iii) poorest in Asia. Sector Most respondents believe that the PPIAF traditional infrastructure sector focus is still relevant as it includes the infrastructure backbone critical to the economic development of a country. Despite that, 91% 15 of all respondents believe that PPIAF should expand its focus beyond its core sectors and move towards funding projects in other sectors such as social infrastructure (29 respondents), housing (19 respondents) and agri- infrastructure (15 respondents). Size A third of the respondents consider that small grants (US$75,000) may be suitable for simple assignments as they are quick to implement. However, they are concerned that small grants do not provide high quality consulting advice and are too small given the complexity of PPI issues. One PSC respondent suggested that PPIAF increase the upper limit of small grants to US$125,000. Activity type 74% 16 of the respondents consider that the PPIAF initiative is crucial to support the development of the PPI enabling environment in developing countries, particularly in Africa, where the enabling environment in many countries is very weak. A PSC respondent strongly highlighted the fact that given global economic circumstances, establishing a transparent environment that strikes a balance between investor requirements and government objectives will be a key factor to attract private investment in the near future. PPI policy/strategy, institution building and legal/regulatory/competition framework are the enabling activities that respondents believe to be more significant. On the other hand, the balance 26% of the respondents felt strongly that PPIAF should increase its support towards the delivery of specific transactions. This is an area with huge financial and human capital gaps. However, they recognise that in order to this, PPIAF would have to increase its budget considerably. Knowledge products Approximately 61% 17 of respondents are familiar with PPIAF knowledge products and they are generally satisfied with their content and quality. Of the respondents that are 15 Or 41 out of 45 that responded to the specific question. 16 Or 35 out of 47 that responded to the specific question. 17 Or 30 out of 49 that responded to the specific question. 12

13 unfamiliar with PPIAF knowledge products, majority are BCG 63% of the respondents who have said that they are unfamiliar with PPIAF knowledge products are BCGs. The most popular knowledge products are the PPIAF handbooks, toolkits and best practice guides followed by PPIAF working papers. The PPI database is consulted only by PSC respondents PPIAF operations and governance PPIAF operations Beneficiaries A few results on the efficiency and effectiveness of PPIAF operations as viewed by the BCG respondents are as follows: 83% 18 are satisfied with the quality of task management of PPIAF projects. With regards to the application process, 75% 19 of government officials are satisfied with the way the application process was managed. 72% of them applied for a PPIAF grant through a World Bank task manager. Overall, 93% 20 of them would consider requesting for PPIAF technical assistance in the near future. Some options provided for alternative sources of funds include other development partners (UNEP, regional development banks, World Bank etc) or use their own sources to fund the activity. Private sector consultants A few results on the efficiency and effectiveness of PPIAF operations as viewed by the PSC respondents are as follows: PSC respondents are fully or mostly satisfied with the way PPIAF manages projects and 96% 21 of them would consider bidding for PPIAF projects in the future. PSC respondents believe that the PPIAF procurement process is quick, efficient, appropriately timed and fair. Moreover, their view is that the ToRs are clear and well defined. Finally, the general view is that the PMU is mostly responsive and accessible to their requests for support (see quote in box below). However in few cases, consultants do contact the World Bank task manager instead of the PMU when they need assistance during the assignment. 18 Or 15 out of 18 that replied to the question. 19 Or 15 out of 20 that replied to the specific question. 20 Or 20 out of 21 that replied to the specific question. 21 Or 23 out of 24 that replied to the specific question. 13

14 The PPIAF Manila Office has been accessible and provided invaluable support. They are also responsive and ready to work out solutions PPIAF governance 65% 22 of respondents consider that PPIAF governance arrangements should be more inclusive and should incorporate participation from developing countries governments as a way to reflect beneficiaries country priorities. Few respondents have highlighted the fact that it would improve the ownership and sustainability of the projects and it would increase self-reliance of the beneficiary government. On the other hand, 35% of respondents believe that introducing developing countries in the PPIAF governance scheme would reduce the efficiency of the organisation. With regards to the issue of including regional development banks into the PPIAF governance structure, over 64% 23 of respondents believe that it would be a positive development for PPIAF. This will allow better harmonization and coordination of interventions, avoid duplication of activities and more importantly it would facilitate project cycle related activities as regional banks are important co-financiers of PPP projects Impact of PPIAF projects 24 According to various PSCs, PPIAF has been instrumental in raising awareness and engaging stakeholders in PPP dialog in the Africa region. In addition PPIAF is seen by a BCG respondent as an excellent and reliable source funding for PPI activities. Some specific examples on positive impacts described by some respondents include a transactions development project in Mumbai, where PPIAF successfully offered local policy makers much needed advice to develop better management approaches in vital and challenging areas; and the introduction of competitive tendering in transport projects in Vietnam, which has much scope for replication Recommendations arising from the survey The survey also provided for a general open-ended question, wherein respondents were asked to provide useful recommendations for PPIAF on any other issue not covered so far. Some of the responses include: Bank capture. PPIAF should become less World bank-centric and establish itself independent presence more strongly. This could be achieved by specific representation at conferences, "advertising" through channels such as devex.com, approaching the potential beneficiary governments directly as well as engaging with other groups that are doing similar work to exchange experiences. 22 Or 28 out of 43 that replied to the specific question. 23 Or 28 out of the 44 that replied to the specific question. 24 In this section of the survey, we have only asked qualitative questions in relation to the impact of PPIAF interventions given the broad and diverse target audience. 14

15 Strengthen the collaboration with IFC on transaction design and implementation activities. Clustering a series of PPIAF interventions (awareness raising, enabling environment) around the objective of an actual IFC-led transaction could be good way of developing the enabling environment and implementing effective transactions. Support government in policy implementation of PPIAF activities. Assist governments not only in providing them with best policy advice but also with policy implementation. Increase PPIAF dissemination activity. Improve the dissemination of PPIAF knowledge products especially among African governments and regional institutions. Moreover, it would be interesting to increase the dissemination of reports from PPIAF projects, as they are of high quality and interest. Create frameworks or 'help desk' support which would enable faster drawdown of high quality input. One of the key issues for a civil servant running a PPP programme is to be able to access high quality practical support when it is needed (advice on a technical issue during the project cycle), not a lengthy report delivered several months later. It may only be a half hour telephone call or but hugely significant in terms of impact. Build local capacity. PPIAF should start partnering with the local institutions in order to support the PPP initiatives and also helping to build local capacity. 15