A~~~~A. sclosure Authorized. Public Disclosure Authorized. Public Disclosure Authorized. Public Disclosure Authorized Z-4~~~~~~~~~~~~~~~~~~~~~~~. r.

Transcription

1 Public Disclosure Authorized FI r.7 CL a2 Public Disclosure Authorized A-~~~~~~~~A A~~~~A PIP-~~~~~~~~~~~~~~~~~l q a > t - s.,i N 2t c: 3 - z~~~~'i~ Public Disclosure Authorized ls s X f S n z~~~~~~~~~~~ Wi.-C 0 Z-4~~~~~~~~~~~~~~~~~~~~~~~. sclosure Authorized

2 RECENT WORLD BANK TECHNICAL PAPERS No. 250 Rangeley, Thiam, Andersen, and Lyle, International River Basin Organizations in Sub-Saharan Africa No. 251 Sharma, Rietbergen, Heimo, and Patel, A Strategyfor tile Forest Sector in Sub-Saharan Africa No. 252 The World Bank/FAO/UNIDO/Industry Fertilizer Working Group, World and Regional Supply and Demand Balancesfor Nitrogen, Phiosphate, and Potashi, 1992/ /99 No. 253 Jensen and Malter, Protected Agriculture: A Global Review No. 254 Frischtak, Governance Capacity and Economic Reform in Developing Countries No. 255 Mohan, editor, Bibliography of Publications: Technrical Department, Africa Region, July 1987 to April 1994 No. 256 Campbell, Design and Operation of Smaliholder Irrigation in South Asia No. 258 De Geyndt, Managing the Quality of Health Care in Developing Countries No. 259 Chaudry, Reid, and Malik, editors, Civil Service Reform in Latin America and the Caribbean: Proceedings of a Conference No. 260 Humphrey, Payment Systems: Principles, Practice, and Improvements No. 261 Lynch, Provision for Children with Special Educational Needs in the Asia Region No. 262 Lee and Bobadilla, Healthi Statisticsfor the Americas No. 263 Le Moigne, Subramanian, Xie, and Giltner, editors, A Guide to the Formulation of Water Resources Strategy No. 264 Miller and Jones, Organic and Compost-Based Growing Mediafor Tree Seedling Nurseries No. 265 Viswanath, Building Partnerships for Poverty Reductioni: The Participatory Project Planning Approachi of the Women's Enterprise Management Training Outreach Program (WEMTOP) No. 266 Hill and Bender, Developing the Regulatory Environmentfor Competitive Agricultural Markets No. 267 Valdes and Schaeffer, Surveillance of Agricultural Prices and Trade: A Handbookfor the Dominican Republic No. 268 Vald6s and Schaeffer, Surveillance of Agricultural Prices and Trade: A Handbookfor Colombia No. 269 Scheierling, Overcoming Agricultural Pollution of Water: The Cliallenge of Integrating Agricultural and Environmental Policies in tihe European Union No. 270 Banerjee, Rehabilitationi of Degraded Forests in Asia No. 271 Ahmed, Technological Development and Pollution Abatement: A Study of How Enterprises Are Finding Alternatives to Chlorofluorocarbons No. 272 Greaney and Kellaghan, Equity Issues in Public Examinations in Developing Couintries No. 273 Grimshaw and Helfer, editors, Vetiver Grassfor Soil and Water Conservation, Land Rehabilitation, and Embankment Stabilization: A Collection of Papers and Newsletters Compiled by the Vetiver Network No. 274 Govindaraj, Murray, and Chellaraj, Health Expenditires in Latin America No. 275 Heggie, Managemenit and Financinig of Roads: An Agenda for Refor,n No. 276 Johnson, Quality Review Schemesfor Auditors: Their Potentialfor Sub-Saharan Africa No. 277 Convery, Applying Environmental Economics in Africa No. 278 Wijetilleke and Karunaratne, Air Quality Management: Considerationsfor Developing Countries No. 279 Anderson and Ahmed, The Casefor Solar Energy Investments No. 280 Rowat, Malik, and Dakolias, Judicial Reform in Latin America and the Caribbean: Proceedings of a World Bank Conference No. 281 Shen and Contreras-Hermosilla, Environmental and Economic Issues in Forestry: Selected Case Studies in Asia No. 282 Kirn and Benton, Cost-Benefit Analysis of the Onchocerciasis Control Program (OCP) No. 283 Jacobsen, Scobie and Duncan, Statutory Intervention in Agricultural Marketing: A New Zealand Perspective No. 284 Valdes and Schaeffer in collaboration with Roldos and Chiara, Surveillance of Agricultural Price and Trade Policies: A Handbookfor Uruguay No. 285 Brehrn and Castro, The Marketfor Water Rights in Chile: Major Issues No. 286 Tavoulareas and Charpentier, Clean Coal Technologies for Developing Countries No. 287 Gillham, Bell, Arin, Matthews, Rumeur, and Hearn, Cotton Production Prospects for the Next Decade (List continues on the inside back cover)

3 WORLD BANK TECHNICAL PAPER NO. 325 Energy Series Estimating Construction Costs and Schedules Experience with Power Generation Projects in Developing Countries Robert W Bacon, John E. Besant-Jones, andjamshidheidarian The World Bank Washington, D. C.

4 Copyright 1996 The Intemational Bank for Reconstruction and Development/THE WORLD BANK 1818 H Street, N.W. Washington, D.C , U.S.A. All rights reserved Manufactured in the United States of America First printing August 1996 Technical Papers are published to communicate the results of the Bank's work to the development community with the least possible delay. The typescript of this paper therefore has not been prepared in accordance with the procedures appropriate to formal printed texts, and the World Bank accepts no responsibility for errors. Some sources cited in this paper may be informal documents that are not readily available. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations, or to members of its Board of Executive Directors or the countries they represent. The World Bank does not guarantee the accuracy of the data included in this publication and accepts no responsibility whatsoever for any consequence of their use. The boundaries, colors, denominations, and other information shown on any map in this volume do not imply on the part of the World Bank Group any judgment on the legal status of any territory or the endorsement or acceptance of such boundaries. The material in this publication is copyrighted. Requests for permission to reproduce portions of it should be sent to the Office of the Publisher at the address shown in the copyright notice above. The World Bank encourages dissemination of its work and will normally give permission promptly and, when the reproduction is for noncommercial purposes, without asking a fee. Permission to copy portions for classroom use is granted through the Copyright Clearance Center, Inc., Suite 910, 222 Rosewood Drive, Danvers, Massachusetts 01923, U.S.A. The complete backlist of publications from the World Bank is shown in the annual Index of Publications, which contains an alphabetical title list (with full ordering information) and indexes of subjects, authors, and countries and regions. The latest edition is available free of charge from the Distribution Unit, Office of the Publisher, The World Bank, 1818 H Street, N.W., Washington, D.C , U.S.A., or from Publications, The World Bank, 66, avenue d'iena, Paris, France. ISSN: Cover: Detail from William Gropper, "Construction of the Dam" (mural study, Department of the Interior, Washington, DC., 1937). Used by permission of the National Museum of American Art, Smithsonian Institution, transfer from the U.S. Department of the Interior, National Park Service. Robert W. Bacon is a professor of economics at Lincoln College, Oxford University, England, and a consultant to the Industry and Energy Department at the World Bank. John Besant-Jones is a principal economist in the Power Development, Efficiency and Household Fuels Division of the Industry and Energy Department at the World Bank. Jamshid Heidarian is a professor of economics at the University of the District of Columbia, Washington, D.C., and consultant to the Industry and Energy Department at the World Bank. Library of Congress Cataloging-in-Publication Data Bacon, Robert, Estimating construction costs and schedules: experience with power generation projects in developing countries / Robert W. Bacon, John E. Besant-Jones, and Jamshid Heidarian. p. cm. - (World Bank technical paper, ISSN ; no. 325) (Energy series) Includes bibliographical references. ISBN Electric power plants-design and construction-estimates- Developing countries. 2. Electric power plants-developing countries-costs-statistics. 1. Besant-Jones, John, Heidarian, Jamshid. III. Title. IV. Series: World Bank technical paper. Energy series. TK1193.D44B '21-dc2O CIP

5 ENERGY SERIES No. 240 Ahmed, Renewable Energy Technologies: A Review of the Status of Costs of Selected Technologies No. 242 Barnes, Openshaw, Smith, and van der Plas, What Makes People Cook with Improved Biomass Stoves? No. 243 Menke and Fazzari, Improving Electric Power Utility Efficiency: Issues and Recommendations No. 244 Liebenthal, Mathur, and Wade, Solar Energy: Lessonsfrom the Pacific Island Experience No. 271 Ahrned, Technological Development and Political Abatement: A Study of How Enterprises are Finding Alternatives to Chlorofluorocarbons No. 278 Wijetilleke and Karunaratne, Air Quality Management: Considerationsfor Developing Countries No. 279 Anderson and Ahmed, The Casefor Solar Energy Investments No. 286 Tavoulareas and Charpentier, Clean Coal Technologies in Developing Countries No. 296 Stassen, Small-Scale Biomass Gasifiersfor Heat and Power: A Global Review No. 304 Foley, Photovoltaic Applications in Rural Areas of the Developing World No. 308 Adamson, Bates, Laslett, and Pototschnig, Energy Use, Air Pollution, and Environmental Policy in Krakow: Can Economic Incentives Really Help?

6

7 Contents Foreword... Abstract... Acknowledgments... Abbreviations and Acronyms... Units of Measure... xi xiii xiv xv xv Executive Summary Importance of Cost and Schedule Estimation for Power Generation Projects Framework for Evaluating the Performance of Cost and Schedule Estimates Data Base of Power Generation Projects Statistical Approach to Analyzing the Performance of Cost and Schedule Estimates The Overall Performance of Power Project Cost and Schedule Estimates Group Performance of All Power Generation Projects Distinction between Thermal Power Projects and Hydropower Projects Prevalence of Bias and Uncertainty in Cost and Schedule Estimates Ex Post Analysis of Responsibility for Schedule Slip Reliability of the World Bank's Methodology for Computing Price Contingencies Significant Project Characteristics and External Variables for Cost and Schedule Estimates Thermal Power Project Costs Hydropower Project Costs Thermal Power Project Schedules Hydropower Project Schedules Grouping of Significant Variables v

8 7. Implications of the Analysis for Power System Planning Principal Findings Using Regressions to Improve Predictions for Project Costs and Schedules Risk and Planning Issues Measurement of Project Risk Distribution of Possible Project Outcomes Planning Issues Involving Choice between Sequences of Projects Basic Recommendations Annex 1: World Bank-Supported Power Generation Projects Used for the Analysis of Cost and Schedule Estimating Performance Annex 2: Regression Results with All Variables Included Annex 3: Comparisons of Actual Ratios and Predicted Ratios from Regressions for Costs and Schedules Annex 4: Statistics for Single-Variate Analysis of All Variables Annex 5: Ex Post Attribution of Factors Responsible for Schedule Slip in World Bank-Supported Power Generation Projects Annex 6: Methodology for Deriving Actual Project Costs in Constant Price Terms Annex 7: Analysis of Relationships for the Performance of Price Contingencies Relation between Actual and Estimated Current Costs Link between Actual Current Costs and Actual Constant Costs Relation between Actual and Estimated Cost Escalation Relation between Cost Overrun in Current Price Terms, in Constant Price Terms, and Errors in Predicting Inflation Rates and Project Schedules Appendix A7.1 Composition of the World Bank's Price Contingency Formula for Predicting Project Cost Escalation Annex 8: Computations of Probabilities of Exceeding Specific Project Costs Appendix A8.1 The Variance of a Predicted Value from a Regression Appendix A8.2 Probability that the Outcome of One Project Is Greater Than That of the Other vi

9 Annex 9: Assigning Probabilities to Scenarios for Risk Analysis Annex 10: The Calculation of the Mean and Variance of the Cost of Two Projects Annex 11: Applying the Option Approach to Construction Costs and Schedules A. Investment Valuation Under Uncertainty Using the Options Approach B. General Solution Methods C. A Simple Model for the Investment Option and Optimal Timing I D. Application of the Simple Options Model to Construction Costs and Schedules Dl. General Assumptions D2. General Formulation to Estimate Option Model Uncertainty D3. The Cases of Thermal and Hydropower Plants Annex 12: Performance of Power Demand Forecasts and of World Bank Oil Price Projections References Tables 3.1 Geographical Distribution of Power Generation Projects in the Data Base Distribution of Power Generation Projects by Year of Approval in the Data Base Distribution of Power Generation Projects by Installed Capacity in the Data Base Distribution of Thermal Projects by Production Technology, Primary Fuel, and Unit Size in the Data Base Variables and Characteristics Used in Regressions on Construction Cost Overrun and Schedule Slip Cases Omitted from Analysis Overall Statistics for Cost and Schedule Performance Comparison of Squared Correlations between Actual and Estimated Costs, and between Actual and Estimated Schedules for World Bank-Supported Power Generation Projects Comparison of Cost Overrun and Schedule Slip between World Bank-Supported Power Generation Projects and All Bank-Supported Projects Chances of Overruns Exceeding 20 Percent Sensitivity Level Squared Correlations between Cost Overruns and Schedule Slips for Thermal Power and Hydropower Projects vil

10 5.7 Ex Post Attribution of Responsibility for Project Schedule Slip Significant Variables for Thermal Power Project Costs (current values) Significant Variables for Thermal Power Project Costs (constant values) Significant Variables for Hydropower Project Costs (current values) Significant Variables for Hydropower Project Costs (constant values) Significant Variables for Thermal Power Project Schedules Significant Variables for Hydropower Project Schedules Grouping of Significant Variables for Project Cost and Schedule Estimates Sensitivity of the Levels of Predicted Values to Indicator Variables A1.1 World Bank-Supported Power Generation Projects Used for the Analysis of Cost and Schedule Estimating Performance.63 A2.1 Variables for Log of Thermal Power Project Costs (Current Values).69 A2.2 Variables for Log of Thermal Power Project Costs (Constant Values).70 A2.3 Variables for Log of Hydropower Project Costs (Current Values).71 A2.4 Variables for Log of Hydropower Project Costs (Constant Values).72 A2.5 Variables for Log of Thermal Power Project Schedules.73 A2.6 Variables for Log of Hydropower Project Schedules.74 A4.1 Single-Variate Regression Correlations.80 A4.2 Comparison of Significant Variables at 90 Percent Confidence Level between Multivariate Analysis and Single-Variate Analysis.81 A5.1 Ex Post Attribution of Factors Responsible for Schedule Slip in World Bank-Supported Power Generation Projects.83 A6.1 Standard Disbursement Profiles for Project Cost in Current Price Terms.86 A6.2 Example of Project Cost Derivation in Constant Price Terms: Algeria, Base Year A8.1 Probability that Project 1 Has Higher Cost (Schedule) than Project 2.97 A9.1 Pairs of Parametric Values that Fit the Required Regression Variance of for Scenarios where the External Variance Is a Function of the Size of the Middle Value.102 A9.2 Pairs of Parametric Values that Fit the Required Regression Variance of 0.01 for Scenarios where the External Variance Is Not a Function of the Size of the Middle Value.102 A9.3 Probabilities for Scenarios With Predetermined Variances for Costs and Demand viii

11 Figures 5.1 Relationship between Actual Costs and Estimated Costs for World Bank- Supported Thermal Power Projects and Hydropower Projects, Relationship between Actual Schedules and Estimated Schedules for World Bank-Supported Thermal Power Projects and Hydropower Projects, Distribution of Cost Performance for World Bank-Supported Thernal Power Projects and Hydropower Projects, (current prices) Distribution of Schedule Performance for World Bank-Supported Thermal Power Projects and Hydropower Projects, Errors in Cost Escalation Estimates for World Bank-Supported Power Generation Projects Approved between 1970 and A3.1 Plot of Actual and Predicted Ratios for Thermal Costs (current) A3.2 Plot of Actual and Predicted Ratios for Thermal Costs (constant) A3.3 Plot of Actual and Predicted Ratios for Thermal Schedules A3.4 Plot of Actual and Predicted Ratios for Hydro Costs (current) A3.5 Plot of Actual and Predicted Ratios for Hydro Costs (constant) A3.6 Plot of Actual and Predicted Ratios for Hydro Schedules A7.1 Comparison of MUV Index Actual Values with Values from Forecasts Made between 1974 and 1988 (Based on actual value in 1980 = 100) A12.1 Performance of Power Demand Forecasts for Developing Countries A12.2 World Bank Oil Price Projections in Constant 1987 US$ per Barrel ix

12

13 Foreword One of the main reasons for the ongoing reforms to power sectors around the world is the desire for better management of the economic and financial risks of investing in large power supply projects. These risks arise from uncertainty about future power demand, fuel prices, and-as shown in this paper-construction costs and schedules. Conventional planning approaches based on deterministic scenarios of the future under centralized decisionmaking have seldom given sufficient attention to, or reliable guidance on, these risks. Now, however, under the more decentralized planning process coming into use in reformed power sectors, both public and private decisionmakers will have to respond to the concerns of shareholders, consumers, financiers, and the general public about the risks of investing in large power projects. Although the risks from construction cost overruns and schedule delays are often serious for developing countries, they have been poorly understood to date. In response, the paper puts forward a number of straightforward techniques to improve the analysis of these construction cost and schedule risks. The paper also complements several other Industry and Energy Department and Energy Sector Management Assistance Programme (ESMAP) publications and seminars on structuring and financing power generation projects. Richard Stem Director Industry and Energy Department xi

14

15 Abstract This paper helps national planning and finance ministries, power utilities, and financing agencies to improve the reliability of their estimates for construction costs and schedules of power generation projects in developing countries and thereby to improve the selection and implementation of these projects. The paper examines estimates of construction costs and schedules that were made for a group of power generation projects approved for financing by the World Bank between 1965 and This group of some 64 thermal power plants and 71 hydroelectric plants is then subjected to a statistical analysis. From this analysis, the paper assesses the reliability of the estimates and identifies factors that were significantly associated with bias and uncertainty in them. The paper draws the following conclusions. First, the average estimating error among projects as a whole was too large to be ignored. Second, estimated values were significantly biased below actual values, and the accuracy of estimated values had a large variance. Third, the performance of cost estimates was much better for thermal power projects than for hydropower projects, but schedule estimates performed similarly for these two groups of projects. Fourth, the performance of estimated values can be related to a number of indicator variables through regression analysis, and these regressions can be used to derive expected values that carry less uncertainty than the corresponding estimated values. The paper then demonstrates how to improve the prediction of the actual construction cost or schedule for a power generation project by deriving an unbiased expected value from the estimated value and the appropriate regression equation for the project. There is a proviso, however, that the project has similar technology and implementation arrangements (i.e., public sector) to those that characterize the projects analyzed in the paper. The paper recommends that analyses of power generation projects include a case in which expected values are used for construction costs and schedules. This case would supplement the standard analysis that is based on appraised estimates of these values. Nevertheless, substantial uncertainty remains about the reliability of even the expected values. The regressions given in the paper can be used to provide a measure of project risk arising from this source. Consequently, the paper also recommends that the economic and financial risks associated with the selection of a particular power project or power development strategy should be explicitly considered during project appraisal. This analysis would elicit valuable insights about the riskiness of power generation projects under consideration, such as projects considered to be the least-cost option from the customary deterministic approach to power system planning. The paper presents straightforward techniques for evaluating frequently encountered questions of risk about power projects and development programs. xiii

16 Acknowledgments The authors gratefully acknowledge the valuable comments on drafts of the paper given by Dennis Anderson, William Buehring, Joseph Gilling, Vladimir Koritarov, Spiros Martzoukos, Lucio Monari, Lant Pritchett, Mark Segal, Charles Siebenthal, and Odd Ystgaard. Thanks also go to Vonica Burroughs and Carole-Sue Castronuovo for word processing assistance and to Paul Wolman for his editorial and production work. The authors take responsibility for any errors and omissions in the paper. xlv

17 Abbreviations and Acronyms CPI forex consumer price index foreign exchange G-5 France, Germany, Japan, United Kingdom, and United States G&T generation and transmission GDP ICB IDA MUV NPV O&M RPI SAR SD SEE SER UN gross domestic product international competitive bidding International Development Association UN Unit Value Index of manufactured goods exported from G-5 countries to developing countries Net Present Value operation and maintenance Retail Price Index Staff Appraisal Report standard deviation standard error of estimate standard error of regression United Nations Units of Measure dollars GW GWh MW US$ gigawatt gigawatt hour megawatt xv

18

19 Executive Summary This paper helps national planning and finance ministries, power utilities, and financing agencies to improve the reliability of their estimates for construction costs and schedules of power generation projects in developing countries and thereby to improve the selection and implementation of these projects. Project cost and schedule estimates can deviate from actual costs and schedules in two ways. First, estimates may be generally biased, in that the mean of the estimates for a group of projects differs significantly from the mean of the actual costs or schedules for the group. Second, even when such typical project bias is allowed for, estimates are still subject to uncertainty, in which the relationship between estimated and actual values shows a large variance around their mean values. By identifying and allowing for facts that lead to variations in the degree of bias in the estimates for particular types of projects, it is possible to reduce the overall uncertainty for the financing of power projects and the development of power systems. The paper proceeds in three main stages. First, it examines estimates of construction costs and schedules that were made for a group of power generation projects approved for financing by the World Bank between 1965 and This group of some 64 thermal power plants and 71 hydroelectric plants is then subjected to a statistical analysis. From this analysis, the paper assesses the reliability of past estimates and identifies factors that were significantly associated with bias and uncertainty in them. The paper concludes by reviewing the implications of these findings for the treatment of bias and uncertainty in estimates of construction costs and schedules for power system planning. The paper has four important findings: * Estimates of construction costs and schedules were fairly strongly correlated with the actual outcomes, but the average estimating error among projects as a whole was too large to be ignored. * These estimated values were significantly biased below actual values, and the accuracy of estimated values had a large variance. * The performance of estimated costs was much better for thermal power projects than for hydropower projects, but schedule estimates performed similarly for these two groups of projects. * The performance of estimated values can be related to a number of indicator variables through regression analysis, and these regressions can be used to derive expected values that carry less uncertainty than the corresponding estimated values. In covering only World Bank-supported projects, this analysis does not give a reliable impression of estimating performance for projects financed and implemented by private sector enterprises. 1

20 2 Estimating Construction Costs and Schedules For all power generation projects, appraisal estimates showed substantial optimistic bias and major uncertainty. For this group, the actual project cost (in current prices and excluding interest during construction) exceeded the estimated project costs on average by 21 percent of the estimated project cost, with a standard deviation of 36 percent. Likewise, the actual project implementation periods exceeded the estimated periods on average by 36 percent of the estimated periods, with a standard deviation of 42 percent. Cost overruns and schedule slips were weakly correlated, especially for hydropower projects. A simple sensitivity test reflected inadequately the inherent uncertainty in the estimates of project costs and schedules. Bias and uncertainty on such extensive scales have major impacts on investment selection and financing. In addition, the reliability of the World Bank's methodology for computing price contingencies for project cost estimates was examined. The results show that the Bank's methodology improved the prediction of cost escalation, but that substantial room for improvement remains. A multivariate regression analysis was undertaken to identify correlation between four performance ratios and 29 possible explanatory variables. Ten projects (five thermal and five hydro) were omitted from this analysis because of truly exceptional differences between their estimated and actual values that would give rise to misleading regressions. Thermal power projects were explicitly distinguished from hydropower projects because cost estimates for these groups behaved differently (average cost underestimation for thermal projects was 6 percent, for hydro projects, 27 percent). The six regressions that were analyzed thus covered thermal power costs (in current terms and constant terms), thermal power schedules, hydropower costs (current and constant), and hydropower schedules. The dependent variables were the ratios of actual to estimated values. The explanatory variables reflected project technology, project size, procurement method, host-country features, and World Bank appraisal guidelines. Of these variables, 18 were found to be correlated significantly with one or more of the performance ratios, as follows: 7 variables were significant in one of the regressions, 4 variables in two of the regressions, 5 variables in three of the regressions, 1 variable-estimated cost-in four of the regressions, and I variable-station extension dummy-was significant for five regressions. These variables were able to explain about half of the observed variations in the ratios of actual to estimated costs and schedules. The significance of many individual variables indicates that the use of an "average adjustment factor" (determined over all projects) would itself be biased. The analysis accepted fairly weak evidence of systematic correlation (10 percent significance test) to be sure of picking up significant factors. The findings show that further analysis should lead to better models of reducing risk in estimating project construction cost and schedules, in which risk assessment is able to rely on lower measures of risk. The paper then demonstrates how to improve the prediction of the actual construction cost or schedule for a power generation project by deriving an unbiased expected value

21 Executive Summary 3 from the estimated value and the appropriate regression equation for the project. There is a proviso, however, that the project has similar technology and implementation arrangements (i.e., public sector) to those that characterize the projects analyzed in the paper. The paper recommends that the analysis of power generation projects should include a case in which expected values are used for construction costs and schedules. This case would supplement the standard analysis that is based on appraised estimates of these values. Nevertheless, substantial uncertainty remains about the reliability of even the expected values. The regressions given in the paper can be used to provide a measure of project risk arising from this source. Consequently, the paper also recommends that the economic and financial risks associated with the selection of a particular power project or power development strategy are explicitly considered during project appraisal. This analysis would elicit valuable insights about the riskiness of power generation projects under consideration, such as projects considered to be the least-cost option from the customary deterministic approach to power system planning. The final section thus presents straightforward techniques for evaluating the following frequently encountered questions of risk about power projects and development programs. These techniques avoid undue analytical complexity yet overcome the deficiencies of simplistic sensitivity analysis, and hence they are intended to supplement standard least-cost analysis. Briefly stated, the techniques aim at * Ascertaining the cost level for a given project that will be exceeded with a specific probability and the probability that the project cost will exceed a specified value (similarly with project schedule). - Choosing between projects based on the probability of exceeding a cost limit and evaluating which project has the lower probability of exceeding a given cost limit. * Assigning probabilities to costs or schedule scenarios for risk analysis that are consistent with the variance of cost or schedule outcomes derived from the regressions. * Making the choice between a large project and a set of two or more smaller projects, where the reliability of estimates depends on the scale variables identified in the regression and risks also depend on project size. * Deciding whether to delay a project or not, where the crucial issue is the need for more time to improve estimates of key planning parameters-such as site geology, hydrology, or environmental impacts-for the project or its alternatives. An application of the financial options approach to the optimal timing of projects under uncertainty about construction costs and schedules is developed in the paper. Finally, the paper recommends that these techniques be tested operationally and developed in case studies, so that guideline/s can be formulated for using them in the appraisal of power generation projects.

22

23 1 Importance of Cost and Schedule Estimation for Power Generation Projects Developing countries are planning massive investments to meet their power needs (Moore and Smith 1990).1 The construction costs and schedules of power generation projects thus affect national economic development and the financial viability of project investors. Power projects are capital-intensive and require lengthy construction periods. In total, they account for a substantial proportion-averaging about 10 percent-of a developing country's total physical investment. Some hydroelectric and thermal power complexes in the largest developing countries rank among global megaprojects costing billions of dollars. Even modest-sized schemes-by global standards-in small developing countries can be huge in relation to the size of their economies. Reliable estimates of construction costs and schedules presented by power utilities and their consultants at the time of project approval are important for justifying a project on economic grounds and for planning the means of financing it. Faced with the huge economic and financial costs of expanding power supply, governments of developing countries are under pressure to ensure that power projects are selected with due consideration for the economic and commercial risks that arise from the uncertainty in these estimates. The economic impact of a construction cost overrun is the possible loss of the economic justification for the project. A cost overrun can also be critical to policies for pricing electricity on the basis of economic costs, because such overruns lead to underpricing. The financial impact of a cost overrun is the strain on the power utility and on national financing capacity in terms of foreign borrowings and domestic credit. The recourse in this situation is to reduce the scale of a project to a level commensurate with 1. It is estimated that developing countries are planning to install 384 GW of new generating capacity during the 1990s at a cost of about US$450 billion in 1989 price terms, which would increase their total installed generating capacity by about 80 percent over the amount installed in The principal production technologies for this new capacity are coal-fired steam plants (172 GW) and hydroelectric plants (137 GW). Gas turbines, fueled with natural gas, comprise 34 GW. Oil-fired steam plants comprise only 14 GW. 5

24 6 Estimating Construction Costs and Schedules the planned financial outlay, but this alternative is seldom rational for a power generation project because power generation units can produce benefits (i.e., power) only if they are fully installed. 2 The delay of output caused by slippage in a construction project schedule imposes economic costs when the power system is short of capacity or is supplying power from plants with high variable costs. The financial impacts on the power utility of schedule slippage are an increase in its financing charges and the possibility of incurring project loan repayments before the project generates revenues. This paper helps national planning and finance ministries, power utilities, and financing agencies to improve the reliability of their estimates for construction costs and schedules of power generation projects in developing countries and thereby to improve the selection and implementation of these projects. From an analysis of World Bank experience with estimating the costs and schedules for constructing power generation projects, the paper identifies factors that were significantly associated with the reliability of cost and schedule estimates. 3 In particular, it investigates whether project characteristics-size, technology and procurement conditions, and external variables (notably, country conditions and changes in World Bank guidelines for project appraisals)-were associated with differences in estimating reliability. From this case study, the paper draws implications for improving power system planning. 4 The paper is structured as follows. Chapter 2 lays out the framework for evaluating the performance of estimates of construction costs and schedules for power generation projects in developing countries. Chapter 3 summarizes the data base of power generation projects assembled for this evaluation. Chapter 4 describes the statistical approach used to analyze the performance of the cost and schedule estimates. Chapter 5 assesses the overall performance of estimates for the group of power generation projects as a whole, particularly the prevalence of bias and uncertainty in this performance. It also examines the reliability of the World Bank's methodology for computing price contingencies for project cost estimates. Chapter 6 identifies project characteristics that show a significant correlation with cost overrun and schedule slip. Chapter 7 concludes the paper by examining the implications of the analysis for the treatment of bias and uncertainty in power system planning. 2. The exception to this situation is to defer installation of one or more generation units in a multi-unit generation station, but this decision is usually made at the time of project approval rather than during implementation. 3. The paper does not compare estimates with actual values for fuel costs of thermal power plants or operation and maintenance (O&M) costs. 4. The results of this analysis could provide information for innovative planning methodologies. See, for example, Crousillat (1989).

25 2 Framework for Evaluating the Performance of Cost and Schedule Estimates Project cost and schedule estimates can deviate from actual costs and schedules in two ways. First, estimates might be biased generally, in that the mean of the estimates for a group of projects differs significantly from the mean of the actual costs or schedules for the group. If the estimates of all options for increasing power generation capacity were similarly biased, the net distortion to project selection would then be confined to erroneous timing rather than to incorrect choice of project, since the bias could be explicitly allowed for. Where the bias is not identical for all projects (as is actually the case), then a consistent direction in the bias for particular types of project can indicate the presence of a strong influence on estimates that, once identified, is often correctable in future estimates. Second, even when typical project biases are allowed for, estimates are still subject to uncertainty, in which the relationship between estimates and actual values shows a large variance around their mean values. By identifying and allowing for factors that lead to variations in the degree of bias in the estimates for particular types of projects, it is possible to reduce the overall uncertainty for the financing of power projects and development of power systems. 5 Most causes of deviation between estimates and actual values of costs and schedules fall into three categories: (a) poor development of estimates and supervision of projects by the project sponsor (normally a power utility) and its engineers; (b) poor project implementation by suppliers and contractors; and (c) changes to the external conditions (economic and regulatory) for a project. The long time span covered by the projects in this analysis encompasses a variety of economic and market conditions, as indicated by periods of high and low inflation and by several boom-and-bust cycles in the international market for power plant construction. Poor assessment of the prevailing pressures on the 5. Significant bias and uncertainty appear to exist in forecasts and estimates of most of the major planning parameters for power system development. For example, power demand forecasts in developing countries are shown to have this feature in Sanghvi and Vernstrom (1989). Oil price forecasts also have been highly inaccurate. The historic forecasting performance for these two parameters is shown in Annex 12. 7

26 8 Estimating Construction Costs and Schedules suppliers and contractors at the time of bidding for projects can lead to inaccurate estimates. Furthermore, projects that are marginally economic and politically motivated may be more liable to have optimistic cost estimates relative to actual construction costs than projects that show good economic returns because the sponsors of politically motivated and marginal projects are seeking to obtain approval for financing and to avoid criticism about high costs. Changes in project scope during implementation can have a significant impact on the project costs and schedules. Such changes can arise, for example, from the inability of design-stage investigation to eliminate risks from unknown geological conditions for construction of underground works, particularly for many hydropower projects. In addition, for first-of-a-kind projects in developing countriesand many of the power projects in the data base fall into this category-project estimators do not have a track record of similar projects as a basis for carefully analyzing major construction risks and deriving reliable contingencies for them. Instead, they often rely on unreliable rules of thumb for such contingencies. Some account must also be taken of such unpredictable events as natural disasters and civil disturbances that severely disrupt project implementation. In practice, however, it is virtually impossible to obtain a direct and reliable quantification of the allocation of responsibilities for cost and schedule deviations between these categories from the available information on project appraisal 6 and implementation. The closest approximation to this analysis that could be attempted was a broad allocation of responsibilities for slippage in project implementation schedules (see, in chapter 5, the section on ex post analysis of responsibility for schedule slip). Because a direct analysis of project factors that lead to cost and schedule overruns could not be undertaken, the paper evaluates the estimating performance of cost and schedules for a group of completed World Bank-supported power generation projects by means of a statistical analysis of the following relationships: * The prevalence of bias and uncertainty within Bank-supported power generation projects and in relation to all Bank-supported projects. * On the basis that significant bias and uncertainty is found in the first step, the relation between the estimated performance and various project characteristics, such as production technology and project size. * The relation between external factors associated with project implementation and estimating performance, particularly procurement method, country economic conditions, and pressure to complete the project because of demand for its output. The analysis is based on a comparison of the estimated cost at the time of project approval with the actual cost of implementing the project (as determined after project 6. This problem may explain the lack of published studies of cost and schedule estimating performance for capital projects (as opposed to the published studies that derive econometric models for predicting construction costs and schedules that are based on the actual costs and schedules for completed projects).

27 Framework for Evaluating the Performance of Cost and Schedule Estimates 9 completion) and a similar comparison for the project implementation schedule. The technique of analysis used is multiple regression (using the statistical package MicroTSP), which allows for the simultaneous correlation of the overruns with several indicators that may themselves be correlated. 7 The analysis of cost performance is done in both current price terms and constant price terms, since there is no prior basis for assuming that cost performance in the two cases is affected identically by factors that cause deviations between actual and estimated values. In other words, price effects on costs would have to be purely random to justify such an assumption, and the analysis reported in this paper does not support this assumption, even though the factors influencing cost performance were found to be similar for the two cases. The actual current costs are directly observable, whereas the actual constant costs have to be derived from the former. Estimates of project costs in both current and constant price terms are routinely given in World Bank staff appraisal reports. Both forms of costs exclude interest during construction for this analysis. The analysis of costs in current price terms allows for the impact of failure to allow correctly for price inflation. In the World Bank's appraisal of a project, the cost estimate in current prices is derived by adding to the constant price estimate a contingency for price inflation during the project construction period. 8 This price contingency allows particularly for the expected effect on the project cost of contractual price adjustment clauses relating to materials, labor, and equipment. 9 In view of the importance of this price contingency in estimates of project costs, this paper also includes in chapter 5 an analysis of the reliability of the Bank's methodology for computing price contingencies. The constant price cost estimate is the estimated cost of the project at the time of negotiating the project loan provided that there is no major alteration in project scope, 7. Where none of the indicators are correlated among themselves, a series of single-variable correlations between the overruns and the indicators would give identical results to a multiple correlation. Where such variables are intercorrelated, as in this study, multiple correlation is able to reveal which variables are significantly related to the overruns, allowing for the fact that other variables are also included in the explanation. Thus, some variables that appear significant in a single-variable context are not significant when other more important variables are included. Other variables that may not appear significant in a single-variable context can be revealed as significant in the multiple-variable context. This feature is illustrated in Annex 4 for the projects studied in this paper. 8. The World Bank's practice on contingencies for the projects under review is given in nowsuperseded World Bank internal documents (Central Projects Note [CPN] 3.11 of February ; "Project Cost Estimates and Contingency Allowances." and its Operational Manual Statement (OMS] 2.21 of May 1980, "Economic Analysis of Projects"). The World Bank has used explicit price contingencies for project cost estimates from 1970 onward. The present methodology for computing price contingencies was introduced around Another vintage effect on estimating performance is the request by the World Bank's Board of Executive Directors in the mid-1970s that all projects presented for its consideration have completed designs. Before then. most projects presented to the Board had cost estimates based on feasibility level work.

28 10 Estimating Construction Costs and Schedules quantities, or contract prices during project implementation. The World Bank includes a contingency in this estimate for cost (not price) increases attributable to minor changes in project scope and quantities that are expected to occur between the time of project appraisal and project completion. This physical contingency forms part of the estimated value of the project cost and is not intended to compensate for the possibility of bias toward underestimation. The paper also assesses the reliability of the World Bank's standard sensitivity test for uncertainty in estimating project costs and schedules. It is not the World Bank's practice to use contingency allowances as safety margins for bias and uncertainty. However, the World Bank does test the sensitivity of its economic analysis for power development programs to deviations from its estimates for project costs and schedules, typically for a 20 percent overrun from the expected value.'i In covering only World Bank-supported projects, the analysis applies to projects that were generally implemented by state-owned power enterprises with government financial support. This analysis therefore does not give a reliable impression of estimating performance for projects that were financed and implemented by private sector enterprises. 10. The World Bank also performs sensitivity analysis on the robustness of the economic justification by comparing the maximum discount rate at which the project forms part of the least-cost means of meeting power demand, with an estimate of the opportunity cost of capital to the host country.

29 3 Data Base of Power Generation Projects The data base assembled for this paper consists of 135 power generation projects in developing countries financed with World Bank loans and International Development Association (IDA) credits, of which 64 were thermal power plants and 71 were hydroelectric plants. Information on these projects is taken from World Bank documents (principally staff appraisal reports and project completion reports). These projects constitute virtually all the power generation projects approved for financing by the World Bank between 1965 and The analysis thus captures the World Bank's experience with estimation of project costs and schedules over the long term. Issues related to sampling were avoided in this case by including all, but only, World Bank-supported projects of this type. The projects are listed in Annex 1. World Bank-supported power generation projects are a suitable class for this type of analysis because they are * Based on classifiable technologies for providing the same product * Planned, designed, and procured according to well-established and identifiable practices e Not prone to significant changes in scope during implementation, so that estimated and actual outcomes are comparable * Fully implemented because they have to be completed to provide any output and, thus, project benefits * Well represented throughout all types of developing countries and over the three decades under study * Well documented in Bank staff project appraisal reports and project completion reports, in which the data on appraisal estimates and actual implementation costs and schedules have been objectively checked by World Bank staff. The actual project total costs cover a wide range-between $3.2 million and $1,782 million in current-price terms. The actual project implementation schedules also cover a wide range-between 1.2 and 14.4 years. The projects were implemented in 52 developing countries and were distributed between regions in the manner shown in Table 11

30 12 Estimating Construction Costs and Schedules 3.1. The majority of the thermal power projects were located in Asia and in Europe, the Middle East and North Africa, whereas the majority of the hydroelectric projects were located in Latin America and the Caribbean and in Sub-Saharan Africa. Table 3.1 Geographical Distribution of Power Generation Projects in the Data Base All Thermal projects Hydroelectric projects projects Regiona Number % Number % % Africa (Sub-Saharan) Asia Europe, Middle East and North Africa Latin American and Caribbean TOTAL aregions in this table correspond to the prevailing World Bank organizational categories. In terms of project vintage the data base is fairly well distributed over the 20-year range as shown in Table 3.2. Table 3.2 Distribution of Power Generation Projects by Year of Approval in the Data Base All Thermal projects Hydroelectric projects projects Period Number % Number % % TOTAL In terms of project size, the data base is well represented in all the capacity ranges for these projects, and the two groups have similar features, as shown in Table 3.3.

31 Data Base of Power Generation Projects 1 3 Table 3.3 Distribution of Power Generation Projects by Installed Capacity in the Data Base All Capacity range Thermal projects Hydroelectric projects projects (MW) Number % Number % % ,000-2, TOTAL The distribution of the thermal projects by type of production technology, primary fuel, and unit size is summarized in Table 3.4. The size of generating plant in the group of projects varies from about 2 MW diesel units to 400 MW and 500 MW units in recent coal-fired steam plants (India, Indonesia, and Korea) and a 550 MW dual-fired (fuel oil and natural gas) steam plant in Thailand. Table 3.4 Distribution of Thermal Projects by Production Technology, Primary Fuel, and Unit Size in the Data Base Number of Range of unit size Technology projects % (MW) Steam turbine Coal-fired Fuel-oil-fired Lignite-fired Gas-fired Multi-fueleda Subtotal steam turbine Gas turbine Combined-cycle Diesel TOTAL acomprising two projects with coal and fuel oil; two projects with natural gas and fuel oil; and two projects with coal, fuel oil, and natural gas as alternative fuels.

32 14 Estimating Construction Costs and Schedules The hydropower projects encompass a wide range of hydraulic heads from 12 to 1,035 meters, dams from 11 to 230 meters in height, tunnels from zero to 34 kilometers in length, and reservoirs from virtually zero to about 840,000 hectares in surface area. Extensions to existing thermal power stations accounted for 34 (53 percent) of all the thermal power projects, whereas for hydropower projects 19 (27 percent) were extensions.

33 4 Statistical Approach to Analyzing the Performance of Cost and Schedule Estimates The performance of estimates is measured by the following ratios: Cost performance. This is the ratio of the actual project cost to the estimated project cost. It therefore measures construction cost overrun. The analysis is done for two specifications of cost, one in current prices and the other in constant prices. Interest during construction is excluded in both cases. The estimate is usually based on substantial preparation work before the time of approval to lend by the main project financiers. These costs cover the capital works for the power generators and associated transmission facilities, engineering services, and local duties on imports. I' * Schedule performance. This is the ratio of the actual project implementation period to the estimated project implementation period. It therefore measures slippage of the construction program. The start of the implementation period is taken to be the date of project approval by the main financiers, and the end is the date of entry into service of the completed generation plant (formal acceptance by the power utility). These ratios standardize the measurement of performance for differences in project cost and schedule. This feature enables the performance of estimates to be analyzed in terms of statistical distributions. A value of unity for a performance ratio implies that the impact of associated factors on actual implementation has been correctly anticipated in the estimate. It does not necessarily imply that actual implementation was the best feasible and that the estimate was made correctly on this basis. Any deviation from unity represents the net effect of two deviations: that of actual implementation from the best feasible implementation performance and that of the estimate from the best estimate (equal to best feasible implementation, by definition). I1. Local import duties cannot be excluded because the actual payments for these duties are not given in the World Bank's project completion reports. 15

34 16 Estimating Construction Costs and Schedules The crucial point is that estimates should allow for all foreseeable events and experience with similar projects, so that deviations between estimates and actual outcomes should be unforeseen and random. This is the "null" hypothesis. The analysis reported in this paper tests this hypothesis by finding patterns through correlations. The analysis does not specifically examine the effects of unforeseen and unforeseeable conditions during project construction. These effects can only be assessed from an ex post analysis such as the assessment (reported in chapter 5 of this paper) of force majeure events on project schedules. The analysis proceeds in two main stages. In the first stage (reported in chapter 5), the analysis assesses the prevalence among the group of generation power projects as a whole of bias and uncertainty in estimating performance for costs and schedules in the following steps: * Among power generation projects themselves as a group. * Against all audited World Bank projects since 1974 (and thus approved from 1968), in all economic sectors for which reliable data are available on actual and estimated performance, numbering 2,032 projects. 2 * Against a reference distribution for the performance ratios based on the World Bank's standard sensitivity test for whether a project is still justified under a 20 percent overrun from the estimate (see, in chapter 5, the section on prevalence of bias and uncertainty). i An ex post assessment of the relative impact on schedule slip of the actions of clients/engineers, suppliers/contractors, and uncontrollable events. The reliability of the World Bank's methodology for computing price contingencies. Once the overall performance of estimating costs and schedules for power generation projects has been described, the second stage of the analysis focuses on identifying external variables and project characteristics that show a significant correlation with cost overrun and with schedule slip (as reported in chapter 6). The approach is to look for variables and characteristics that are known at the inception of the project, and that are plausibly correlated with actual costs or schedules, so that if these links are misestimated, these variables would also turn out to be correlated with the degree of cost overrun or schedule slip. If sufficiently strong correlations can be established, then experience of this effect can lead to better estimates of the likely costs and schedules for future projects. In such cases there would be less measured overrun or slip. The statistical analysis is thus searching for factors that have not been fully taken into account in constructing cost and schedule estimates and whose presence affect costs and schedules and will thus be correlated with the degree of cost overrun and schedule slip. 12. The analysis of cost and schedule performance for audited World Bank projects is reported in a World Bank Operations Evaluation Department Report (World Bank 1992).

35 Statistical Approach to Analyzing the Performance of Estimates 17 Because it is not possible to identify these factors reliably in advance of the analysis for this paper, it was decided to include a wide-ranging group of variables and characteristics in the analysis to maximize the possibility of identifying factors that are significantly associated with inaccurate cost and schedule estimates. Information was extracted from World Bank project reports on 29 project variables and external characteristics that might be expected to have some correlation with the degree of costs and schedules for projects as a whole. These characteristics and variables are listed in Table 4.1. They are organized to bring out some of the underlying factors that influence the performance of project cost and schedule estimates. Project-specific variables are categorized under technology, size, and procurement. External characteristics are assigned either to country variables or to World Bank guidelines for project appraisal. Technology variables reflect complexity of project construction, and it might be expected that the uncertainty of cost and schedule estimates increases with this complexity. The basic distinction in technology is between thermal power and hydropower, particularly in terms of the amount of plant and equipment that is constructed under the suppliers' control on their own premises, which is greater for thermal power projects. On the other hand, civil works at the project sites are more prominent in hydropower projects and face the uncertainties of local conditions. On these grounds alone, hydropower project estimates are expected to be subject to greater bias and uncertainty. An extension to an existing station is expected to produce better estimates than for a new station because estimators should be familiar with the specific station design and do not have to face the uncertainties associated with opening up a new site. Greater project size also is expected to increase project complexity and, thus, bias and uncertainty for estimates. Size is not only reflected in the obvious variables-generator unit capacity, station capacity, total cost, and construction schedule. Rather, certain project parameters carry their own estimating risks, such as the well-known uncertainty associated with underground works. Length of tunnel is thus tested, although the uncertainty often arises from difficult ground conditions (e.g., karstic limestone; see World Bank 1984) or from inadequate site investigation. Dam height and hydraulic head are features that reflect overall project size, not necessarily simultaneously (e.g., a project with a high dam can have a relatively low hydraulic head, and vice-versa). Reservoir area reflects another aspect of size-project impact on the immediate vicinity, especially displacement of resident population. It is expected that hydropower project schedules are affected by the time required to relocate people from the reservoir area and other projectrelated areas, since this component has carried a high degree of uncertainty for many past projects.

36 18 Estimating Construction Costs and Schedules Table 4.1 Variables and Characteristics Used in Regressions on Construction Cost Overrun and Schedule Slip Project-specific variables Technology Hydropower project or thermal power project (dummy variable) New power station or station extension (dummy variable) Civil works-estimated (percentage) Thermal power fuel and technology: cost as a proportion of estimated total project cost Diesel-fueled combustion turbine (dummy variable) Coal- or lignite-fueled steam turbine (dummy variable) Fuel-oil-fired steam turbine (dummy variable). Size Generator unit capacity (MW) Total project generation capacity (MW) Estimated project cost in current price terms (US$ million) Estimated construction schedule (years) Hydropower project features: Dam height for new hydropower station (meters) Hydraulic head for new hydropower station (meters) Reservoir area created by project (hectares) Length of tunnels (kilometers). Procurement Year of World Bank loan agreement Anticipated sources of suppliers and contractors, by estimated project foreign exchange costs as a proportion of estimated total project costs (percentage) Competitiveness of procurement process, by anticipated amount contracted under international competitive bidding as a proportion of estimated total project costs (percentage) Number of financing agencies in the project Main contractor is from the host country (dummy variable). Country variables Per capita income of host country in year of loan approval (constant US$)

37 Statistical Approach to Analyzing the Performance of Estimates 19 Average actual cost growth rate for project components procured from host country between year of loan approval and year of project completion-the GDP deflator (percentage) Actual growth in national (or state) power sales (GWh) between year of loan approval and year of project completion (percentage) Indian thermal power projects (dummy variable) Brazilian hydropower projects (dummy variable) Colombian hydropower projects (dummy variable) Index of actual average cost growth rate for imported project components between year of loan approval and year of project completion (UN Unit Value Index of manufactured goods exported from G-5 countries to developing countries-in constant US$ terms). World Bank appraisal guidelines Basis of project cost estimate-recent similar projects or tenders for major components of the project itself (dummy variable) Pre-1970 loan agreement (dummy variable) Post-1976 loan agreement (dummy variable). Procurement methods are particularly important for estimating project costs and schedules because they reflect the degree of competition in the tendering and contract award process. Since international suppliers and contractors dominate the market for constructing power stations in nearly all developing countries, the prevailing state of the order book for these firms is an important indicator of competition for a project. In the absence of a detailed analysis of this feature from the mid-1960s to the mid-1980s, the best available proxy indicator is project vintage, taken to be the year of the World Bank loan agreement, which usually corresponds closely to the time of award of major contracts for a project. This variable also captures any long-term secular change in estimating performance over the period of the project loan approvals. The specific degree of competition for a project can be indicated by the amount of procurement that took place under international competitive bidding (ICB), as opposed to other procurement approaches for the project, and by the proportion of foreign procurement measured by the proportion of total project costs that are incurred in foreign exchange. The origin of the main project contractor (from the host country rather than another country) is included as a dummy variable to provide an additional test for procurement. Finally, since official financing agencies tend to follow their own procurement guidelines (ranging from ICB by multilaterals to own-country preferences by bilaterals), procurement complexity, and

38 20 Estimating Construction Costs and Schedules hence estimating uncertainty, tends to increase with the involvement of more financing agencies in a project. Three country-specific characteristics and one international characteristic are tested for their impact on estimating performance. The first is the general level of economic support that a country can provide for the construction of complex facilities such as a power station, which relieves and mitigates the uncertainty from reliance on imports. The best available proxy for most countries is country-per-capita income (although this is not a perfect indicator because, for example, India and China have low per capita incomes but substantial industrial capacity, whereas a small middle-income country may have to rely virtually entirely on imported goods and services). The inflation rate in the host country during project construction is an indicator of economic (and, in some cases, political) stability and hence of an important element of uncertainty for constructing complex projects. The inflation rate for imported goods and services used in a project also influences the reliability of cost project estimates. The growth rate in power sales in the country is an indicator of how quickly the new power capacity is required and therefore of the keenness of the host power company to complete the project within the estimated time and cost. The international characteristic is a measure of international inflation that reflects cost growth for imported components of these construction projects. 13 In three countries, however-namely India, Brazil, and Colombia-the World Bank has supported a sufficiently large number of power generation projects to enable a country variable to be tested specifically for each of these countries. Country dummy is a co-variate-it picks up an average effect of variables whose coefficients are not allowed to be country specific. Where a country dummy is found to be significant, a formal regression of projects in that country would be the ideal approach to identifying the country-specific significant variables. In such a situation, the covariance of risk among projects in a country could then be explicitly considered. Tests for country-specific groupings for economic variables were not generally carried out because of lack of degrees of freedom with the number of countries (52) covered by the group of projects. Nevertheless, remarkably strong results are obtained given the huge variation in economic conditions encountered among the large number countries covered in the study. Finally, the World Bank's guidelines for project appraisal specify some key practices for estimating project costs and schedules. From the mid-1970s onward, such estimates were based, wherever possible, on bids for major project components. Otherwise, these estimates were based on completed designs and actual costs for recent similar projects. Likewise, a formal price contingency in the Bank's cost estimates has been required since 1970, and the current methodology was introduced in The analysis tests the impacts of these requirements. (A detailed assessment of the reliability of this 13. In cases where the change in price index (foreign or local) or power sales was negative (3 cases), the data on the variables are omitted from the data base because they cannot be used in log form for the regression equations.

39 Statistical Approach to Analvzing the Performance of Estimates 21 methodology is described in chapter 5, in the section entitled Reliability of the Bank's Methodology for Computing Price Contingencies.) Using multiple regression analysis, a sequential analysis is undertaken in which the cost and schedule overruns are each correlated with all the above variables and characteristics; then, variables and characteristics that are insignificant (using a t test) are dropped one at a time; and, finally, the regression equation is re-estimated. The procedure is iterated until all the remaining variables are significant. The final regression equations are taken as the "best" available predictors of cost and schedule overruns, based on knowledge of the set of variables and characteristics used for the analysis.

40

41 5 The Overall Performance of Power Project Cost and Schedule Estimates This chapter is organized in the following stages. First, the performance of cost and schedule estimates is given for World Bank-supported power generation projects as a whole. Second, estimating performance is analyzed separately for the group of thermal power projects and the group of hydropower projects. Third, the prevalence of bias and uncertainty in cost and schedule estimates is examined. The fourth section examines an ex post attribution of responsibility for schedule slip. The final section assesses the reliability of the World Bank's methodology for computing price contingencies in its construction cost estimates. Group Performance of All Power Generation Projects The group of power generation projects as a whole has a correlation (squared) of 0.76 between actual costs and estimated costs (in current prices and excluding interest during construction) and a correlation of 0.55 between actual schedules and estimated schedules (see Figures 5.1 and 5.2). It is clear that the estimates are fairly strongly related to the actual outcomes but that there is considerable inaccuracy among projects as a whole in the estimation of costs and schedules. Preliminary screening of the data on the ratios of actual to estimated values for both costs and schedules reveals a few cases with truly exceptional differences between estimated and actual values that should be treated as such and omitted from any statistical analysis that is looking for regularities. The nine omnitted cases are shown in Table 5.1. In addition, a hydropower project in Portugal (Seventh Power Project) consists of a number of dams, so that it is impossible to relate overall cost and schedule slip to the characteristics of a single construction, as is the case for all the other projects. Omitting these exceptional cases leaves 59 thermal power projects and 66 hydropower projects for detailed analysis. 23

42 24 Estimating Construction Costs and Schedules Figure 5.1 Relationship between Actual Costs and Estimated Costs for World Bank-Supported Thermal Power Projects and Hydropower Projects, (in current prices, US$ million) Estimated current cost log scale 3 Thermal 3150 A Hydro N x/ u/ E Sh A,E~~~~N NE A ig ~ ~ A NEA A N NA YA~~~~~~ /tx~~~ta Xh x Xx A 100N / ~~~~~~~~~~~~~~~~~~~~log scale Actual current cost The data on the variables for these projects are almost complete, with the exception of information on reservoir area. Experiments with this variable on the subset of available observations suggest that it does not have a significant correlation with the accuracy of cost or schedule estimates for hydropower projects.

43 The Overall Performance of Estimates 25 Figure 5.2 Relationship between Actual Schedules and Estimated Schedules for World Bank-Supported Thermal Power Projects and Hydropower Projects, Estimated schedule (months) log scale *K Thermal Hydro _ _ A 25 A 20 - * log scale Actual schedule (months)

44 26 Estimating Construction Costs and Schedules Country Brazil Colombia Panama Romania Sierra Leone Turkey Table 5.1 Cases Omitted from Analysis Project Hydro: Volta Grande Hydro: Las Mesitas Thermal: San Francisco Thermal: Second Turceni Thernal: Third Power Project (King Tom station) Thermal: Elbistan Uruguay Thermal: Battle Unit 6 Yugoslavia Zambia Hydro: Middle Neretva project: Grabovica and Salakovac dams Hydro: Kariba North Note: These projects were more prone to force majeure events than other projects and thus faced genuinely unpredictable major risks. (The impact of force majeure events and schedules are discussed in the section below on ex post analysis of responsibility for schedule slip.) The criterion for omitting observations (ratios of actuals to estimates) is that they were more than 4 standard deviations from the mean of the remaining points. Including such observations can give rise to seriously misleading regressions, since they force the regression into explaining such large outliers rather than the bulk of the more central observations. All the remaining observations lie within 2.5 standard deviations from the mean. The overall statistics-mean and standard deviation-for the group of projects before and after removing the exceptional cases are given in Table 5.2. Table 5.2 Overall Statistics for Cost and Schedule Performance (%) Project group Mean SD With exceptional cases Cost overrun Current costs Constant costs Schedule slip Without exceptional cases Cost overrun Current costs Constant costs Schedule slip Note: SD = standar deviation.

45 The Overall Performance of Estimates 27 Distinction between Thermal Power Projects and Hydropower Projects The first regression carried out examined cost overrun for the whole group of projects in the data base. Twenty-one of the independent variables are relevant to all classes of projects. The correlation coefficient for this regression was 0.56 without attempting to strip out the insignificant variables. With so many variables, this is a moderate result. It was therefore decided to examine the scope for obtaining a stronger correlation by carrying out separate regressions for thermal power projects and hydropower projects. Discriminating between these two project types allows variables that are significant to only one or the other type to be identified and, thus, improves the regressions. This initial separation can also be understood in physical terms because these groups of projects involve qualitatively different construction techniques, and also because preliminary indications showed that as groups they behave differently, especially as regards the typical cost overrun (see Figures 5.3 and 5.4). 14 Figure 5.3 Distribution of Cost Performance for World Bank-Supported Thermal Power Projects and Hydropower Projects, (current prices) Percent of projects 40 Thermal Hydro < < < < < 2.00 > 2.00 Ratio of actual cost to estimated cost 14. One indication of this difference is the much higher proportion of costs that are accounted by civil works in hydropower projects (53 percent average) than in thermal power projects (18 percent average).

46 28 Estimating Construction Costs and Schedules Figure 5.4 Distribution of Schedule Performance for World Bank-Supported Thermal Power Projects and Hydropower Projects, Percent of projects 60 - Thermal Hydro < < < < < 2.00 > 2.00 Ratio of actual cost to estimated cost One concern about splitting the group of projects into two subgroups is whether there are sufficient degrees of freedom (number of observations less number of independent variables). Significance tests take account of both the number of observations and the number of variables used in any of the regressions. In this case, it was possible to identify several strongly significant variables in each regression. The first step in this analysis was to find the correlations between actual and estimated costs and schedules for all World Bank-supported power generation projects, thermal power projects, and hydropower projects-excluding the 10 outliers. These correlations are compared in Table 5.3. Table 5.3 Comparison of Squared Correlations between Actual and Estimated Costs, and between Actual and Estimated Schedules for World Bank-Supported Power Generation Projects Cost Project group Current Constant Schedule All power projects Thermal power projects Hydropower projects Note: Sample excludes the 10 outlier projects.

47 The Overall Performance of Estimates 29 These results indicate (particularly for schedules) only a moderate correlation between actual and estimated values. Thermal power projects had a high correlation for both costs and schedules. Prevalence of Bias and Uncertainty in Cost and Schedule Estimates Analysis of the basic data on cost overrun and schedule slip for the 59 thermal power projects and 66 hydropower projects-excluding the 10 outliers-gives the following results. For thermal power projects as a whole there was a relatively small bias for costs, with an average underestimation of 6 percent, but for schedule there was a large average underestimation of 30 percent. Both the cost estimates and the schedule estimates showed substantial variation around these mean values, with a standard deviation of 23 percent (around a mean of 106 percent for actual to estimated values), for costs and a standard deviation of 30 percent (around a mean of 130 percent) for schedules. For hydropower projects both the average cost overrun of 27 percent and the average schedule slip of 28 percent were very substantial. The cost ratios showed an extremely high standard deviation of 38 percent (around a mean of 127 percent), whereas that for the schedule ratios was 28 percent (around a mean of 128 percent). Thus, even after removing the quite exceptional cases, it is clear that forecasts of schedules generally seriously underestimate the actual schedule for both thermal power and hydropower projects and that cost estimates were seriously underestimated for hydropower projects. Costs for thermal power projects were generally only slightly underestimated. There was also a very large variation in the reliability of estimates for costs and schedules for both thermal power and hydropower projects. Even with attempts to correct estimates by adding on a "typical" slip factor of (say) 27 percent for hydropower project cost estimates, deviations would still have been large. Costs and schedules for power generation projects have been estimated more accurately than for World Bank-supported projects in general, both with respect to the mean error and the standard deviation of these errors. The single exception is that cost overruns for hydropower projects have shown a substantially larger average error than the average cost overrun for the totality of World Bank-supported projects. The comparison with World Bank-supported projects as a whole is shown in Table 5.4.

48 30 Estimating Construction Costs and Schedules Table 5.4 Comparison of Cost Overrun and Schedule Slip between World Bank- Supported Power Generation Projects and All Bank-Supported Projects (%) Cost overrun Schedule slip Project group Mean SD Mean SD All Bank projects Thermal power projects Hydropower projects Note: SD = standard deviation. The effective cost overrun for all World Bank-supported projects may have been much higher than indicated here because the scope of many of these projects was substantially reduced during implementation to keep total project expenditure within the available funding (this did not happen with power generation projects). Source. World Bank (1992). A 20 percent sensitivity test for economic analysis is inadequate to allow for the inherent bias and variation in the estimates of power project costs and schedules. This sensitivity test can be expressed as a null hypothesis that the average forecast error is zero and the standard deviation of the forecast error is 20 percent. Assuming errors to be normally distributed, it follows from this hypothesis that 16 percent of all projects would be expected to have an actual-to-estimated ratio exceeding 120 percent (one standard deviation), and only 2 percent of projects would have this ratio exceeding 140 percent (two standard deviations).15 That is, one in six of projects might be expected to have a larger cost or schedule overrun than the one used for sensitivity analysis. In the light of the larger actual means and standard deviations found above for overruns, the chances of exceeding the sensitivity criterion are as given in Table Table 5.5 Chances of Overruns Exceeding 20 Percent Sensitivity Level (%) Cost Project grouip Current Constant Schedule Thermal power projects Hydropower projects Sensitivity tests are usually conducted only for overruns, so that a right-skewed distribution would be more accurate than a normal distribution. Finding an appropriate distribution would require an analysis of residuals, and hence a normal distribution was chosen just to illustrate the limitation of this type of sensitivity test. 16. For example, for the thermal power project cost overrun that has a normal distribution with mean 1.06 and standard deviation of 0.23, this is equivalent to asking what the chance is that a random drawing exceeds 1.2. The answer is equal to the probability of drawing from a standard normal distribution a value exceeding ( )/0.23 = which is 27 percent.

49 The Overall Performance of Estimates 31 Even allowing for 20 percent underestimation, the majority of hydropower projects could be expected to have a larger cost overrun and schedule slip, and the majority of thermal power projects could be expected to have a larger schedule slip, than the sensitivity analysis value. In order to construct a sensitivity test at a level where only a few projects could be expected to show larger cost overruns or schedule slip, a 40 percent level would be more appropriate. With such a large test deviation, however, standard deterministic approaches to project justification become virtually unworkable. This finding thus underscores the need for proper risk analysis rather than simplistic sensitivity testing. Finally, the standard procedure of testing sensitivity to cost overruns and schedule slips separately fails to capture the compounding effect on the benefit/cost ratio of a project when both errors occur. In this case, the increase in cost arising from schedule slip through the effect of time on the value of money would lead to higher chances of overruns exceeding the 20 percent level than those given in Table 5.5. The correlations between costs overruns and schedule slips for thermal power and hydropower projects were found to be weak, as shown in Table 5.6. Table 5.6 Squared Correlations between Cost Overruns and Schedule Slips for Thermal Power and Hydropower Projects Thermal power cost overrun 0.24 Thermal power schedule slip I Hydropower cost overrun 0.01 Hydropower schedule slip 0 The difference in magnitude of these correlations suggests that the unanticipated factors may have some common factors for costs and schedules of the thermal power plants but that for hydropower plants the unanticipated factors are completely different as regards cost overruns and schedule slip. Ex Post Analysis of Responsibility for Schedule Slip An ex post analysis of the causes of slippage in project implementation schedules was undertaken to attribute responsibility between the two main parties for project and implementation-namely client/engineer and contractor/supplier-as well as to uncontrollable events. This analysis covered 103 of the power generation projects approved for financing by the World Bank between 1965 and The relative importance of the causes was identified by counting the number of times each cause was cited in project completion reports. A wide range of causes were cited (they are listed in Annex 5; 25 for thermal power projects, 41 for hydropower projects). Many of the causes (transportation difficulties,

50 32 Estimating Construction Costs and Schedules delays in contract award, equipment failure during testing, replacement of substandard work, and shortages of materials) were cited in both classes of projects. Other frequently cited causes for hydroelectric projects were geological problems, design changes, bad weather, and poor project management. Thermal power projects were also subject to delays caused by labor disputes and shortages of skilled labor. A noticeable feature of the list of causes is the emphasis on implementation difficulties, with the implication that they would not have been anticipated in schedule estimates based on the assumption of good project implementation performance. However, it can be argued that project clients and their engineers should have been able to anticipate problems under their control (and have had the incentives to do so) in preparing their estimates of project schedules, even if they had to assume that contractors and suppliers would perform soundly and that uncontrollable events (such as natural disasters and civil disturbance) were too unpredictable to factor into their estimates. The citations were then attributed to categories-client/engineer, contractor/supplier, and uncontrollable events (relative frequencies of the citations are shown in Table 5.7). Table 5.7 Ex Post Attribution of Responsibility for Project Schedule Slip (%) Party orfactor Type of project responsible for slip Hydropower Thermal power Client/engineer Contractor/supplier Uncontrollable events 9 16 In order to test whether these factors are captured by the variables used in the regression analysis (see chapter 6), a simple test is used. Three "dummy" variables are created for the three causes of slip, and for any project the dummy is given a value of unity if that factor was identified as having been a problem during implementation (and zero if not). The best regressions for schedule slip, as given below, are then rerun with the inclusion of the three dummy variables. For neither thermal project schedule nor hydro project schedule are any of the dummy variables significant, suggesting that the included variables, which are known in advance of project implementation, are as successful as a crude ex post analysis in identifying sources of schedule slip. If the factors used to construct the dummy variables could be given weightings, rather than simply dichotomized as present or absent, then ex post analysis might be able to identify some of the variation that is unexplained by the regression.

51 The Overall Performance of Estimates 33 Reliability of the World Bank's Methodology for Computing Price Contingencies To allow for escalation in its project cost estimates during project implementation, the methodology introduced in 1976 by the World Bank computes a price contingency from the following three basic parameters: * Estimated project cost in constant price terms of year project approved ("base year") * Forecast escalation factor for the costs of imported project components * Forecast escalation factor for the costs of locally procured project components. This apparently simple formulation in fact involves a complex set of relationships between numerous variables, as shown by the fully expanded formula given in the appendix to Annex 7. The World Bank's formula thus has the disadvantage of compounding the uncertainties involved in forecasting many variables. The extent of this uncertainty is shown by the wide range of errors in cost escalation estimates for World Bank-supported power generation projects that is shown in Figure 5.5. It is noticeable that the greatest errors occurred during the periods of highest international inflation between 1970 and The factors that determine the reliability of this cost escalation formula are thus the accuracy of the projections for these variables together with the forecast pattern of commitments to project expenditures over the estimated schedule of project implementation. If full ex post information on assumptions and actual values for all these variables were available for the projects, the reliability of the formula could be gauged by a simple disaggregation of the formula to derive the contributions to cost escalation. Some of this information is not available, however, particularly on the proportions of actual costs that are incurred in each year of implementation. For purposes of assessing the reliability of this formula from the available project data, the analysis therefore proceeds in the following manner. The estimate of project cost in constant price terms is noted as CO(E). To this value, a price contingency factor is added to give an estimate in current price terms, CU(E). At the termination of project construction, the actual cost in current price terms, CU(A), is revealed. From this cost, an implicit actual value in constant prices, CO(A), is calculated using the disbursement formula described in Annex 6. The analysis proceeds by examining the cost overrun in current price terms, O(CU) = CU(A)/CU(E), and finding factors that are correlated with the extent to which actual costs exceed estimated costs (both measured in current prices). The analysis is carried out on the record of the group of power generation projects covered by this paper.

52 34 Estimating Construction Costs and Schedules Figure 5.5 Errors in Cost Escalation Estimates for World Bank-Supported Power Generation Projects Approved between 1970 and 1986 Year of approval 1986 $ _ l_l_l Ratio Note: Ratio of (Actual Cost Growth less Estimated Cost Growth) to Estimated Cost Growth, where cost growth is the difference between project cost in current prices and project cost in constant (base year) prices.

53 The Overall Perfonnance of Estimates 35 Since the estimate of current costs is based on an estimate of constant costs together with estimates of the price factors, errors in either or both of the latter terms will lead to errors in cost estimates expressed in current prices. The basis of this analysis is to examine a series of price relationships to find interrelationships between these different measurements. Four separate links are explored: a. The link between actual costs in current prices and estimated costs in current prices b. The link between current actual costs and constant actual costs c. The link between actual cost escalation (current actual costs relative to constant actual costs) to estimated cost escalation (current estimated costs relative to constant estimated costs) d. The link between the cost overrun (actual versus estimated) in current terms to cost overrun in constant price terms and the errors in predicting inflation rates and project schedule. These four links are interconnected and follow a natural sequence. The first reveals the degree of accuracy of the central value-the cost in current prices-whereas the second indicates the importance of inflation in the actual cost that emerges. Given that there is inflation (cost escalation), the third reveals the extent to which the estimated cost escalation is an adequate predictor of actual cost escalation. The fourth examines the relationship between the failure to produce unbiased estimates of actual costs (in current terms) and the failures to predict costs in constant price terms (the physical dimension) and to predict inflation (a function of inflation rates and construction schedule) correctly. Once it is established that any failure to forecast costs well is due to failures to predict costs in constant terms as well as to failure to predict inflation, then it is important to search for factors that are known at the time of project preparation that tend to be correlated with errors in cost estimation, so that an appropriate adjustment can be made to the estimates or a warning given on possible bias. Annex 7 presents the statistical analysis of these four links. The principal findings are as follows: a. The actual and estimated values, both measured in current costs, are strongly but far from perfectly correlated. Moreover, estimated values are significantly below actual values. Such a difference can be due to differences in predicting the physical costs (constant prices) or the inflation rate. b. In order to check the accuracy of the working of the price contingency formula, the actual current cost was correlated against the actual constant cost value, as well as the actual domestic inflation, the actual inflation of imported manufactures, and the actual schedule length. The strong relation between these variables (especially for thermal power projects) confirms that the "physical" aspect of the project can be separated from the inflation component.

54 36 Estimating Construction Costs and Schedules c. The actual cost escalation is correlated with the estimated cost escalation, and this revealed a strong but not perfect relationship, leaving room for further improvements in constructing this aspect of a project cost forecast. As expected, no difference in forecasting performance was found between thermal and hydropower projects. d. A regression was made of the cost overrun in current price terms on the cost overrun in constant price terms, errors in forecasting domestic inflation and imported inflation, and errors in estimating the schedule period. This showed that both the physical and inflation aspects of forecasting errors could be identified, because errors in estimating three of these variables-constant project costs, domestic inflation, and schedule length-were all significantly related to the overall error in estimating project cost in current terms. The relationship was weaker for hydropower than for thermal power projects, confirming an earlier finding that forecasting for the physical component for hydropower projects has been less successful than for thermal power projects.

55 6 Significant Project Characteristics and External Variables for Cost and Schedule Estimates In view of the size and distribution of the errors in estimating project costs and schedules, it is desirable to look for a method that identifies classes of projects that tend to have high or low cost overruns or schedule slips. The approach is to recognize that a number of project-specific factors will affect the actual cost and actual construction schedule of any project. Many of these factors are quantifiable (such as project size, measured either in costs or some physical unit), although some are only classifiable by their presence or absence (whether a project is new or merely an extension of an earlier completed project). Chapter 4 described in detail some 29 variables that can plausibly be considered to be correlated with actual costs and schedule. For these links, the sign of the correlation might be predictable (e.g., that in higher income countries the actual costs of a given project would be less than they would be if the same project were constructed in a lowerincome country). If such features are known and fully recognized, they would be taken into account in constructing the estimates of costs and schedules. Any errors in estimation are then associated with unforeseeable events or with incorrect assessments of the importance of the project-specific factors. In other words, the results of the regression analysis do not specifically identify the main risk factors but rather how well they are treated. Thus, some risk factors may appear significant because estimators did not consider them with due care, even though these factors would not generally be considered as among the most significant factors. Conversely, factors known to be risky may be so well treated that they do not appear among the significant factors in this analysis. Regarding the group of projects as a whole, purely random factors would make estimates diverge from actual, with some above and some below. Moreover, there would be no pattern in the errors taken in their entirety: Not only would the average error for a large enough group of projects be zero (or very near to zero) but the error and any observable variable known at the time of constructing the estimate would not be correlated. Where a factor has been allowed for but its impact is systematically under- or overemphasized, the regression residual would then be correlated with that factor. Even 37

56 38 Estimating Construction Costs and Schedules when a factor is positively (say) correlated with actual costs, the error can be negatively or positively correlated with that factor depending on whether the implicit link is over- or underestimated. ' 7 When all the relevant systematic factors have been allowed for in constructing estimated values, and when the strength of these links has been correctly identified, then the ratio of the actual to estimated values should fluctuate randomly around a mean of unity. This is then the null hypothesis-that no variables should have a significant correlation with the ratio of actual to estimated values. If a significant correlation can be identified, then the analysis has picked up tendencies to over- or undercompensate for risk factors by the project analyst. In this situation, the estimates were capable of systematic improvement taken for the group of projects as a whole. The use of regression techniques allows both the identification of any such significant factors and the quantification of their relative importance in terms of the amount of variance explained by them. The factors with the largest t values are significant at the highest confidence levels. For example. a t value of 1.67 gives a 90 percent confidence limit, and a t value of 3 gives a 99 percent confidence limit, for the number of observations that correspond to the number of power projects in the thermal group or the hydro group. The unexplained variance after allowing for the effect of significant factors has to be treated as the unpredictable risk for these estimates. As given by the formula in Annex 4, the t ratio on each variable in a regression can be used to calculate its partial correlation with the dependent variable. The squared partial correlation is the percentage of total variation in a model not explained by the set of all other variables included in the equation that is explained by the variable in question. The higher the t ratio, the higher the partial correlation attached to that variable. Although even preliminary inspection can reveal that certain variables are strongly correlated with differences in the accuracy of estimating actual values, with so many possible variables to check it is necessary to adopt a systematic search procedure, particularly since there are no prior indications of which factors have been taken into account imperfectly in arriving at the estimated values. Accordingly, multiple regressions of cost overrun and schedule slip were carried out on the factors described in Table 4.1. Trials with such regressions for quantitative variables, including the performance ratio 17. These ideas can be formalized: Let actual values (A) of costs or schedules be related to an indicator X and a random term u by the loglinear form: A = axb * eu; while the estimated value (E) is calculated on the basis of the level of factor X by the form: E = cxd. Hence the log of the ratio of actual to expected values is given by log (A/E) = log (a/c) + (b - d) log X + u. which is a simple form of the model used in the multiple regressions. The sign of (b - d) depends both on the sign of the actual relation and the degree to which the schedule is misestimated.

57 Significant Project Characteristics and External Variables 39 itself. indicated that it was most satisfactory to run a "double log" form of regression so that individual parameters can be interpreted as elasticities.' 8 Once the most "general" equation that included all variables had been estimated, variables that were "insignificant" (based on a 90 percent confidence level) were removed one at a time, and the equation re-estimated omitting that variable.' 9 The systematic elimination of insignificant variables not only reduces the degree of bias in the remaining regression but also reduces the overall level of uncertainty (variance) in it because it removes the variances associated with these variables. Given that it is not possible to predict which variables will be correlated with failures to estimate costs and schedules accurately before the analysis, it was considered reasonable both to use a large size of significance test in order to pick up even weak regularities in the data, and to use a two-sided test since the impact of a variable on arriving at estimated values could be either under- or overvalued. The final selected equation thus has only "significant" variables. The multiple correlation (R-squared) coefficient measures the percentage of the total variation of the performance ratio that is explained by the inclusion of the variables. A correlation of zero would indicate that the explanation was no better than that achieved by just using the sample mean of the dependent variable, whereas a correlation of 100 percent would indicate that a complete explanation for the variation had been achieved. A linearized correlation is obtained by correlating the anti-logs of the fitted values of the equation with the actual ratios. From this latter equation, the average residual (linearized standard error of regression) is calculated that indicates the unexplained residual variation in the ratio by the regression and that can be compared to the standard deviation of the basic data. The same approach is used for the analysis of both the cost overrun ratio and the schedule slip ratio. The log of each ratio is correlated with all variables, and then insignificant variables are sequentially removed. No attempt was made to impose the same factors in the analysis of schedule slip that were found to be important for cost overrun or vice-versa. The analysis for significant variables was carried out separately for the following six cases: thermal power project costs, hydropower project costs (costs in both current and 18. It should also be noted that in absolute terms the ratio of actual to estimate must fall between zero and infinity (a skewed distribution), whereas the log form of the ratio will be more symmetric and fall between plus and minus infinity. Taking logs thus supports the presumption that regression residuals will be normally distributed. 19. A confidence level of 90 percent (test size, 10 percent) indicates that were the null hypothesis of no correlation to be true, such a regression value would be expected to occur in random sampling in less than 10 percent of the time. It is possible to see from the test that some of the variables would have passed a higher significance test of 5 percent or even I percent (two-tailed significance results).

58 40 Estimating Construction Costs and Schedules constant values), thermal power project schedules, and hydropower project schedules. Annex 2 shows the regression results with all the tested variables included in the equation. Annex 3 shows plots of the actual ratio (actual value to estimated value) and predicted ratio (predicted value from one of the regression equations to estimated value) for the cost and schedule of each power generation project. These plots show how well the regressions are generally able to predict the value for each project over the wide ranges of the actual ratios among these projects. Thermal Power Project Costs The performance of project cost estimates is analyzed for two cases: first for costs expressed in current values (including the effect of price inflation on costs) and second for costs expressed in constant values. The results of the analysis for thermal power project costs (current values) are given in Table Table 6.1 Significant Variables for Thermal Power Project Costs (current values) Regression 2-tailed Variable coefficient Standard error t-stat. significance Intercept Log estimated cost Log estimated schedule Extension dummy India dummy Log civils costs ratio R-squared Mean of dependent SER SD of dependent Linearized R-squared Linearized SER Note: Dependent variable is log of the thermal cost ratio, based on 45 observations and using 10 percent retention rule. civils = civil works; SD = standard deviation; SER = standard error of regression. In the case of thermal power project cost overruns, the multiple correlation based on all explanatory variables was 80 percent (Annex 2). Once all the insignificant variables had been removed, a strong (linearized) correlation of 57 percent was obtained with five

59 Significant Project Characteristics and External Variables 41 significant variables. Since any regression that includes an intercept has the property that its mean prediction is equal to the mean of the actual data, this regression has an average prediction error (residual) of zero (compared to the bias of 6 percent in the raw data). By allowing for the variations in the measured indicator variables between projects, this regression is also able to reduce the variation around this unbiased value by about onequarter, from the standard deviation of 0.23 for the cost ratios of thermal power projects as a whole (reported in chapter 5 at the beginning of the section on prevalence of bias and uncertainty) to the 0.17 of the linearized standard error of regression. Both estimated costs and station extension dummy had a negative association with cost overruns. Thermal power projects in India, projects with long estimated schedules, and projects with a large estimated civil works component all had a positive association with cost overruns. The two variables related to project size (cost and schedule) thus influenced estimating performance in opposite directions. Where the variables are entered into the regressions in log form, the coefficients are elasticities. Thus, Table 6.1 shows that a 1 percent increase in the level of estimated costs is associated with a percent decrease in cost overruns, whereas a 1 percent increase in the share of civil works costs is associated with a 0.08 percent increase in cost overrun. The use of the regression equation to predict the actual fitted values is illustrated for one of the observations on thermal costs where the estimate was substantially too low. The India 911 project (loan date, 1978) had an estimated cost of $405.9 million, but the actual cost turned out to be $491.1 million. The ratio of actual to estimated values is therefore 1.21, rather than the unit value it should have had if the estimate had correctly predicted the actual outcome. Given that the group of thermal projects as a whole had a 6 percent cost overrun, it can be seen that even applying this average correction to the estimated costs would have still produced a substantial error. The regression equation in Table 6.1 is a project-specific way of making a prediction of the ratio of actual to estimated costs. For the India 911 thermal power project the estimated cost was (natural log value 6.006); the estimated schedule was 5.33 years (log value 1.673); the project was not an extension (dummy value 0.0); the project was in India (dummy value 1.0) and the civil cost ratio was (log value ). Multiplying these values by the regression coefficients shown in Table 6.1 and then summing these values gives a fitted value for the log of the cost ratio of 0.192, and the antilog of this ratio gives a predicted value of the cost ratio of In this case, the regression model is extremely accurate and clearly outperforms the basic estimate (ratio unity) or the average bias correction (ratio 1.06) as a predictor of the ratio of actual to estimated costs. Not all projects are predicted as well as this, but the linearized correlation of 57 percent shows that more than half the variation that would be left unexplained by the average bias correction approach (equivalent to using a regression model with just an intercept as independent variable) is explainable in terms of a few simple variables whose values were known at the time of loan approval.

60 42 Estimating Construction Costs and Schedules The results of the analysis for thermal power project costs in constant values are given in Table 6.2. Table 6.2 Significant Variables for Thermal Power Project Costs (constant values) Regression Standard 2-tailed Variable coefficient error t-stat significance Intercept Log of loan approval date Log of estimated cost Log of estimated schedule Log of MUV index Log of unit size Extension dummy Dummy for Indiadummy R-squared Mean of dependent SER SD of dependent Linearized R-squared Linearized SER Note: Dependent variable is log of the thermal constant cost ratio, based on 57 observations and using 10 percent retention rule. SD = standardeviation; SER = standard error of regression. Lower correlations are obtained in the case of constant costs than in that of current costs, although the standard errors of regression are virtually equal. The multiple correlation based on all explanatory variables for constant cost overruns is 58 percent (Annex 2), and on only the significant variables, 46.3 percent. Nevertheless, there are more significant variables for the constant costs case (8) than for the current costs case (5). Four variables-estimated cost, estimated schedule, extension dummy, and India dummy-are significant in both cases. Project size seems to be highly significant for the constant cost case, since estimated cost and estimated schedule, as well as unit size, are significant variables for this case. The significance of the MUV index and the 1976 dummy could indicate that the World Bank's methodology for computing price contingencies (see chapter 5) has a significant bearing on the performance of project cost estimates in constant values.

61 Significant Project Characteristics and External Variables 43 Hydropower Project Costs The results of the analysis for hydropower project costs (current values) are given in Table 6.3. Table 6.3 Significant Variables for Hydropower Project Costs (current values) Regression 2-tailed Variable coefficient Standard error t-stat significance Intercept Log estimated cost Log forex Log station size Log GDP deflator Extension dummy Log hydraulic head Colombia dummy Log financing agencies R-squared Mean of dependent SER SD of dependent Linearized R-squared Linearized SER Note: Dependent variable is log of the hydropower cost ratio, based on 66 observations and using io percent retention rule. SD = standar deviation; SER = standard error of regression. For hydropower projects, where the mean and standard deviations of the cost overrun are much larger than for thermal power projects, and where the regression on all variables has a correlation of 62 percent (Annex 2), the final regression is fairly successful, since the squared correlation is 51 percent. The linearized standard error of regression of 0.27 shows that the regression reduces the uncertainty around the mean overrun by one-third from the standard deviation of 0.38 for the cost ratios of hydropower projects as a whole. Again, the log of the estimated cost variable has a negative sign, indicating that a given percentage cost overrun was less likely with a larger financing plan. The log of the percentage of foreign exchange also had a strong negative relation. The station size had a positive relation, as did the local inflation rate. The hydraulic head (for projects that are not extensions) had an important positive relation with cost overrun, since this is a variable that can vary greatly in magnitude among projects. A particularly interesting

62 44 Estimating Construction Costs and Schedules finding is that the group of Colombian hydropower projects as a whole tended to have a lower percentage cost overrun than would be expected for their size, hydraulic head, and number of financing agencies (about six). A project-by-project analysis of this group confirms that their mean cost overrun (measured in current prices) was only 1.5 percent, compared with 27 percent for hydropower projects as a whole. 2 0 No such countryspecific effect was found for the group of Brazilian hydropower projects. The results of the analysis for hydropower project cost in constant values are given in Table 6.4. Table 6.4 Significant Variables for Hydropower Project Costs (constant values) Regression Standard 2-tailed Variable coefficient error t-stat significance Intercept Log estimated cost Log forex Log station size Extension dummy Log hydraulic head Log financing agencies Colombiadummy R-squared Mean of dependent SER SD of dependent Linearized R-squared Linearized SER Note: Dependent variable is the log of the hydropower constant cost ratio, based on 51 observations and using 10 percent retention rule. SD = standard deviation; SER = standard error of regression. Slightly lower correlations are obtained in the constant costs case than in the current costs case. The multiple correlation for constant cost overruns based on all explanatory variables is 56 percent (Annex 2), and on only the significant variables, 47.2 percent. The two cases share seven significant variables, and the exception is that GDP deflator is significant for only the current costs case. 20. The Las Mesitas Hydropower project is omitted from the data set because of its schedule slip of 160 percent. Its cost overrun was high at 60 percent, but even including this one raises the average for the Colombian projects to only 8 percent.

63 Significant Project Characteristics and External Variables 45 Thermal Power Project Schedules The results of the analysis for thermal power project schedules are given in Table 6.5. Table 6.5 Significant Variables for Thermal Power Project Schedules Regression 2-tailed Variable coefficient Standard error t-stat significance Intercept Log loan approval date Log forex Log station size Post-1976 dummy Post-1970dummy India dummy R-squared Mean of dependent SER SD of dependent Linearized R-squared Linearized SER Note: Dependent variable is log of the thermal power schedule ratio, based on 57 observations and using 10 percent retention rule. forex = foreign exchange; SD = standardeviation; SER = standard error of regression. For the schedule slip of thermal power projects, which have a substantial mean and standard deviation, the regression on all variables has a correlation of 58 percent (Annex 2). After dropping insignificant variables, the regression is again fairly successful, accounting for 40 percent of the total variance. The linearized standard error of regression of 0.22 is again about two-thirds of the standard deviation of 0.30 for the schedule ratios of thermal power projects as a whole. The positive sign for the log of the loan approval date indicates that for a given set of values of the other variables, thermal power projects approved later tended to have larger schedule slips. The interactions between the date of the loan approval and the post-1970 and post-1976 dummies indicate a general deterioration in the trend for estimating performance for thermal power construction schedules, both before 1970 and after 1976 but also show, that around those dates, steep improvements in performance, so that the trend decline was from a smaller level of underestimation from 1977 onward than it had been before The amount of project costs incurred in foreign exchange, the station

64 46 Estimating Construction Costs and Schedules size, and thermal power projects in India were all negatively correlated with schedule slip for thermal power projects. Hydropower Project Schedules The results of the analysis for hydropower projects schedules are given in Table 6.6. Table 6.6 Significant Variables for Hydropower Project Schedules Regression Standard 2-tailed Variable coefficient error t-stat significance Intercept Log estimated schedule Log per capita income Log % ICB Extension dummy Log hydraulic head R-squared Mean of dependent SER SD of dependent Linearized R-squared Linearized SER Note: Dependent variable is the log of the hydropower schedule ratio, based on 63 observations, and using 10 percent retention rule. SE = standard error; SD = standard deviation; SER = standard error of regression. The regression for hydropower project schedule slip is moderately successful, accounting for 44 percent of the total variance (compared to the regression on all variables that had a correlation of 70 percent, Annex 2). Again, the linearized standard error of regression of 0.20 indicates that the uncertainty is reduced by about one-third from the standard deviation of 0.28 for the schedule ratios of hydropower projects as a whole. The mean prediction of the ratio is equal to the mean of the ratio of actual to estimated schedules. The regression indicates that the schedule estimates were relatively more reliable at high levels of estimated schedule but less reliable for extension projects and for larger hydraulic heads on new projects. Schedule slip for hydropower projects was also positively correlated with per capita income and negatively correlated with the proportion of the project that was subject to international competitive bidding.

65 Significant Project Characteristics and External Variables 47 Grouping of Significant Variables The multivariate analysis of the performance of estimates for project construction costs (current values) and schedules reported in the previous sections identified a total of 18 significant variables at the 90 percent confidence level for the six regression equations that cover thermal power and hydropower projects separately. Of these variables, 7 variables are significant in one of the four regressions, 4 variables in two of the regressions, 5 variables in three of the regressions, I variable-estimated cost-in four of the regressions, and 1-extension dummy-was significant for five regressions. Thus, many factors are significant, but each has a relatively small impact on overall estimating performance. The 18 significant variables are distributed widely among the 29 tested variables listed in Table In terms of the five categories used for these variables, 2 of the significant variables (station extension and civil works costs) are technological; 5 (estimated cost, estimated schedule, station size, unit size, and hydraulic head) relate to project size; 4 (loan approval date, foreign costs, number of financing agencies, and proportion of ICB) reflect procurement; 5 (local inflation, per capita national income, MUV index, Colombia dummy, and India dummy) are country-specific; and 2 (post-1976 dummy and post-1970 dummy) fall under the category of World Bank appraisal guidelines. Variables from the same category, however, seldom reinforce each others' impacts because they tend to be scattered among the regression equations and in some cases are correlated in opposing directions. These results are summarized in Table 6.7. The presence of many variables in the regressions confirms the anticipated relationships postulated in the selection of variables for this analysis (chapter 4). To begin with, the basic technological distinction between thermal and hydropower projects is strongly confirmed. But the significance of civil works only for thermal costs is surprising, which indicates that the impact of this factor in estimates of schedules has been generally well handled. On the other hand, the finding of a greater underestimation of costs and schedules for extensions to hydropower projects, but a lesser underestimation of costs for extensions to thermal power projects. indicates that extension projects often are not as straightforward as expected. 2 In the case of thermal power projects, fuel type and production technology were not found to be significant variables in multivariate regression, although the diesel dummy was significant in the single variate analysis for both costs and schedules, and the coal dummy was significant for costs. 21. Since plant extensions constitute about 40 percent of the projects in the data base, it could be argued that this subcategory of projects should be analyzed separately to test whether its regressions have significantly different coefficients than those for the whole database. However, in this case (and also in the cases of the other dummy variables), the intercept is likely to capture the mean impact of any change of slope in the regression line, so it should give a good approximation of the impact of this variable.

66 48 Estimating Construction Costs and Schedules Table 6.7 Grouping of Significant Variables for Project Cost and Schedule Estimates Thermal power projects Hydropower projects Cost overrun Schedule Cost overrun Schedule Significant variable Current Constant slip Current Constant slip Extension dummy Log civil costs ratio + Log estimated cost Log estimated schedule + + * Logstation size * Log unit size *+ * Log hydraulic head n.r. n.r. n.r Log loan approval date * ++ + * Log forex Log financing agencies Log % ICB... Log MUV index - Log GDPdeflator + Log per capita income.. + Colombia dummy n.r. n.r. n.r. - - India dummy n.r. n.r. n.r. Post-1976 dummy - - * * Post-1970 dummy * * - Key: + positive correlation at 90 percent confidence level - negative correlation at 90 percent confidence level + + positive correlation at 99 percent confidence level - negative correlation at 99 percent confidence level * correlation is not significant at 90 percent confidence level n.r. not relevant The relationship between estimating performance and variables related to project size was mixed. For power generation projects as a whole, there appears to be a tendency for the percentage overrun to decline with the size of the estimated cost in current values. The fact that this variable appears for both hydropower and thermal power projects indicates that whatever leads to a failure to take into account fully the level of estimated

67 Significant Project Characteristics and External Variables 49 costs is common to both types of projects. On the other hand, estimated schedule was positively correlated with thermal cost estimation performance and negatively correlated with hydro schedule estimating performance. Similarly, station size was positively correlated with the estimating performance for hydro costs but negatively correlated with the estimating performance of thermal schedules. Hydraulic head had a strong positive correlation with the performance of both hydro costs and hydro schedules. 2 2 One particularly striking feature of the analysis is that many of the variables listed in Table 4.1 are not correlated significantly with the cost or schedule ratios, either singly or in combination with other variables in the multiple regression context. In the case of hydropower projects, for example, the dam height and tunnel length for new projects have single squared correlations of under 2 percent with the cost ratio and schedule ratio (a value of 5 percent would be needed to indicate significance for a single variable), and they are never significant in the multiple regression context. This suggests that the factors associated with those variables that affect costs and schedules have been correctly allowed for in constructing the estimates, but that there is some aspect caught by the size of the hydraulic head that was not fully allowed for in either the cost or the schedule estimates. 23 Procurement and country variables had mixed significance. Estimates for hydropower projects seem to be more sensitive than those for thermal power projects to local income levels (for schedules) and to foreign inflation rates (for costs). The Colombian hydropower projects tended to have lower costs overruns than generally after allowing for the impacts of other significant variables on estimating performance; likewise, Indian thermal power projects tended to have higher cost overruns but lower schedule slips. World Bank appraisal guidelines for computing price contingencies do not appear to have been a significant factor for cost-estimating performance in current terms. The 1976 guideline was found to be significant in the case of thermal costs in constant terms, but this could reflect a technicality because it was used to derive the actual costs in constant terms for the analysis (Annex 6). The significance of both guidelines (1970 and 1976) on thermal project schedule estimating performance is discussed in the section on thermal project schedules above. 22. The use of explanatory variables that reflect project cost (schedule) with a dependent variable that is the ratio of actual to estimated costs (schedules) can introduce a problem of spurious correlation, such as that residuals are heteroskedastic. In fact the use of actual costs (schedules) as a dependent variable with estimated costs (schedules) as an explanatory variable exhibits very strong heteroskedasticity (see Annex 7, section 1). Using the ratio, however, produces homoskedastic residuals. 23. This observation might be related to the finding that the Colombia dummy is significant for hydropower cost estimations performance, but the Brazil dummy is not significant. since the hydraulic heads of the Colombian projects are among the highest, and those of Brazilian projects are among the lowest, in the group of hydropower projects.

68 50 Estimating Construction Costs and Schedules Finally, the difference in conclusions about significant variables between the multivariate analysis and a single variate analysis are considerable, as shown in Annex 4. In the four regressions, single-variate analysis predicted only 11 of the 24 variables that were significant in the multivariate analysis at the 90 percent confidence level. None of the 6 significant variables that occur in two or three regressions in multivariate analysis were found significant in two or more regressions in single-variate analysis. The latter also falsely indicated 19 variables as significant.

69 7 Implications of the Analysis for Power System Planning The paper has focused so far on analyzing the performance of past estimates of construction costs and schedules for World Bank-financed power generation projects in developing countries. This concluding section examines the implications of the analysis for dealing with future projects. Estimates of construction costs and schedules are, of course, two of many parametric estimates and forecasts that characterize risk and uncertainty in choosing power generation projects (see Sanghvi and Vernstrom 1989 and the discussion in chapter 2 above, footnote 5). Most published analyses on this subject tend to focus on uncertainty in predicting future power demand, fuel prices, and cost of capital. This chapter thus fills a significant gap in the treatment of risk and uncertainty in this class of projects. The analysis identified two problems with historical estimates that need to be recognized in future project evaluation. First, for the group of projects as a whole (which covered virtually all Bank lending for completed generation projects since the mid- 1960s), there has been a very large downward bias in the estimation of costs and in the estimation of schedules. Clearly it is necessary to recognize that standard methods of estimating costs and schedules tend to be overoptimistic. If all projects had experienced equal or nearly equal bias in the estimation of costs and schedules, then the remedy would be simple: multiply the standard estimates by an "adjustment factor." Provided projects in the future were similar to those financed during the period of analysis, and provided that the methods of estimating costs and schedules were not changed substantially, then this simple adjustment would, on average, produce accurate predictions for the actual outcomes. In fact, such a simple adjustment rule is unlikely to be optimal because of the second feature of estimated costs and schedules identified in this paper in that, even allowing for the average underestimation, there is very large variation around this value. The average shortfall in cost estimates for all projects was 21 percent, but some projects came in on budget, whereas others had cost overruns of 40 percent or more. This variation in the accuracy of estimation presents a more substantial problem for project evaluation. 51

70 52 Estimating Construction Costs and Schedules This paper has shown how the apparent variability of cost and schedule estimation can be reduced by categorizing the projects by type, through regression analysis. It has also revealed that despite an intensive search for indicator factors that are known ex ante and that correlate strongly with actual cost and schedule overruns, there appears to be a substantial unpredictable element in these estimates. Once all predictable aspects of estimating performance for costs and schedules have been taken into account, there will be no overall bias in the predictions for any identifiable type of project. The residual inaccuracy can then be seen as the risk associated with costs and schedules rather than a systematic element in expected costs or schedules for a particular group of projects. The first section below recapitulates the principal findings of the paper. The next section then indicates how the regressions developed in the paper can be utilized to obtain more accurate predictions of project construction costs and schedules. The last section addresses the question of risk in predicting these costs and schedules. Various approaches to incorporating risks, as estimated from the regressions, into project decisionmaking are outlined. The chapter concludes by dealing with special issues that arise in choosing between power development programs that involve more than one project. Principal Findings A number of important implications for the planning of power generation projects in developing countries emerge from the analysis of the actual performance of construction cost and schedule estimates for such projects: a. Estimates were fairly strong correlated with the actual outcomes, but the average error among projects as a whole was too large to be ignored. The group of power projects as a whole has a (squared) correlation of 0.76 between actual and estimated costs (in current prices) and 0.55 between actual and estimated schedules. The modest values of these correlations indicates that the estimation process is rather inexact and that large errors have often been made. This is the first justification for seeking a method of improving the predictions of actual costs and schedules. b. The estimated values were significantly biased below actual values. The second crucial finding is that the estimated costs and schedules were, on average, severely biased downward. Estimated construction costs for projects as a whole were on average 21 percent below actual costs, whereas estimated schedules were on average 36 percent below actual values. Even when cases with exceptionally large overruns are omitted from the group of projects, estimated costs still averaged 17 percent below actual costs, whereas estimated schedules were an average 29 percent below actual values. Both of these findings indicate that, on average, an overoptimistic view of project costs and schedule performance was taken.

71 Implications of the Analysis for Power System Planning 53 C. The performance of estimated values differed sharply between different types of generating plants. The most striking difference found was between thermal and hydropower projects. The average cost underestimation for thermal projects was only 6 percent, whereas for hydro projects the average was 27 percent. Schedule estimates were rather similar, with thermal projects showing a 30 percent average underestimation and hydro projects on a 28 percent average underestimation. These results indicate that in attempting to make any adjustment to the estimates of costs and schedules, it would certainly be necessary to treat the costs of thermal projects differently from those of hydro projects. d. The performance of estimated values can be related to a number of indicator variables through regression analysis. Regression analysis, based on a large set of possible variables, indicated that 18 project variables had a significant relationship (at the 90 percent confidence level) in one or more of the six regression equations analyzed (for thermal and hydro project costs-current and constant-and schedules). Of the variables, 2 (station extension, and civil works cost) are technological; 5 (estimated cost, estimated schedule, station size, unit size, and hydraulic head) relate to project size; 5 (local inflation, per capita national income, MUV index, Colombia dummy, and India dummy) are country specific; and 2 (post-1976 dummy, post 1970-dummy) relate to World Bank guidelines. The residual variances from the regressions can be used to provide a measure of risk. These variables were used in the analysis of the ratio of actual to estimated costs and the ratio of actual to estimated schedules and were able to explain between 48 percent and 57 percent of the variation in these ratios. In other words, about half the variation in these percentage errors that would be left unexplained by using an average correction factor for all projects is explicable in terms of a few simple variables whose values were known at the time the project loan was approved. In addition, the significance of individual variables indicates that for a class of projects that took a particular value of the variable, the use of an "average adjustment factor" (determined over all projects) would itself be biased. Using Regressions to Improve Predictions for Project Costs and Schedules The size of the bias in estimating project costs and schedules indicates that the simplest way to make some improvement in predicting actual values would be to attach average correction factors to such estimates. If, for the set of 135 projects analyzed, cost estimates had been adjusted upward by a factor of 1.21 and schedule estimates by a factor of 1.36, then overall the adjusted estimated value would have been unbiased (i.e., the average ratio of predicted to actual values would have been unity). However, for individual subgroups of projects there would still have been a bias in the predicted values. Regression analysis also reveals that about half the variation in the differences among the ratios of actual to adjusted values is capable of being predicted and hence should not be treated as if it were a residual "risk."

72 54 Estimating Construction Costs and Schedules Using the regressions and the known values of the relevant associated variables, predictions can be made of the ratio of actual to estimated costs, and hence an improved prediction of actual costs can be obtained. The approach is illustrated for the India 911 thermal project. For this project, the actual costs turned out to be $ million, whereas the estimate of costs used for appraisal was $405.9 million. The project cost was thus underestimated by 21 percent. If it had been agreed to use the overall adjustment factor for thermal power project costs, then the estimated value would have been increased by 6 percent, still leaving a 15 percent underestimate. On the other hand, using the regression, as described in the first section of chapter 6, to produce a predicted cost would have given a predicted ratio of actual costs to estimated costs, for this class of project, of Applying that factor to the estimated cost of $405.9 million yields a prediction of just under $490 million, very close to the true value. It is therefore recommended that the analysis of power generation projects includes a case in which expected values are used for construction costs and schedules. These expected values can be derived from estimated values by applying ratios obtained from the regression equations given in chapter 6 in the manner illustrated above for the India 911 project. This case supplements the standard analysis that is based on appraised estimates of costs and schedules. The regression approach is designed to give the most accurate statistical prediction of actual costs and schedules based on historic experience. Two very important limitations must be noted. The very best regressions explain less than 60 percent of the variance of the ratio of actual to estimated values, with the remaining variation attributable to unidentified factors. If no substantial improvement in this goodness of fit can be found, then indeed a substantial element of genuine risk is present in appraising the construction costs and schedules of power generation projects. This implies that it is important that a coherent view is taken of the treatment of risk in project appraisal. Although 60 percent correlation is a low value relative to those obtained in many time series econometric studies, it is important to remember that the present study is of a cross-section type (where correlations are typically lower than in time series) and that the dependent variable is the ratio of the actual value to the estimated value rather than simply the actual value. Since the estimated value itself already encapsulates much specialist knowledge of the particular project, the regression is in effect seeking for influences on costs or schedules that have been systematically over- or undervalued by project appraisers. Remembering this interpretation, it is not likely that substantially higher correlations can be found (i.e., what is unexplained is indeed largely risk). Regression analysis cannot be expected to provide a perfect solution to the problem of forecasting actual values. The second aspect of the use of regressions for prediction is that a relation estimated on one set of data to predict outcomes for new data will give unbiased predictions only if the same relationship continues to hold between the indicator variables and the ratio of actual to estimated values as in the historic sample. If the relationship changes-because a new type of plant (e.g., combined cycle) is being considered, where the indicator factors

73 Implications of the Analysis for Power System Planning 55 affect cost and schedule overruns to a different degree than in the sample (e.g., diesels); or because of new ownership and contractual arrangements introduced, such as projects being undertaken by the private sector rather than by the past practice of the public sector; or because the accuracy of estimation itself changes-then bias can result from using the regressions. However, in such cases it is still likely to be more accurate to use the regression than to make no adjustments to estimated values, unless huge differences between the past and present relationships are expected. The analysis of the impact of the dummy variables for changes in World Bank estimating procedures on the accuracy of the estimates tends to confirm this view. For four of the six equations the dummy variables were insignificant, suggesting that changes in World Bank procedures in 1970 and 1976 were not associated with a significant improvement in estimation. Only for thermal schedules was there a measurable improvement in the accuracy of estimation. It thus seems reasonable to assume that it is unlikely that dramatic improvements in the accuracy of estimating costs and schedules will occur. However, where new technologies are involved there can be less confidence that the estimation errors will be similar in nature to those identified in the sample. The regressions also throw some light on the nature of the systematic underestimation of costs and schedules. Each significant variable in a regression points to a factor whose impact on the estimate was undervalued (positive sign) or overvalued (negative sign), and the size of the coefficients indicates the magnitude of such effects. For example, in the case of thermal project cost estimates (Table 6.1), the larger the estimated cost of the project itself, and for projects that were extensions of existing projects, the lower was the tendency to underestimate costs. Alternatively, the longer the estimated construction schedule, or the higher the ratio of civil to total costs, and if the project was in India, the greater was the tendency to underestimate costs. Although the construction of the cost estimates took into account the scale of the project, they tended to underplay economies of scale to costs themselves but not to make enough allowance for the fact that projects with a lengthy construction period would tend to have higher costs than projects of similar physical size with shorter estimated construction periods. Not enough allowance was made for the tendency for costs on thermal power projects in India to be higher than for similar projects elsewhere. The factors summarized in Table 6.7 highlight the significant variables. For any project where a variable is known to have a large value (e.g., a large hydraulic head for a hydropower project), it is sensible to be alert to the tendency to misestimate such projects. Where the project is only marginally viable, this can indicate the need for extensive sensitivity analysis to alternative cost scenarios or even for design modifications that reduce the influence of the problematic variable. The magnitude of the coefficients allows a quantification of the likely sensitivity of the estimates to the presence of the significant factor. Since the equations are estimated in double log form, the coefficients on the logs of the indicator variables measure elasticities-for a I percentage point increase in the size of the variable, the coefficient

74 56 Estimating Construction Costs and Schedules measures the percentage increase in the predicted actual cost levels for given values of the other indicator variables. Where the indicators are dummy (zero/one) variables, for which logs cannot be used, the exponent of the coefficient measures the percentage impact on the level of predicted costs of the presence of the variable. Table 7.1 shows the magnitudes of the elasticities for the factors found significant in the six regressions. Table 7.1 Sensitivity of the Levels of Predicted Values to Indicator Variables Thermnal costs Thermal Hydro costs Hydro Variable Current Constant Schedules Current Constant schedules Extension dummy* Civil cost ratio Estimated costa Estimated schedule Station size Unit size Hydraulic head Loan approval date Forex Financing agencies MUV index % ICB GDP deflator National income Colombia dummy* India dummy* Post-1976 dummy* Post dummy Note: Percentage changes of predicted values given a I percent change in a continuous variable and the presence* versus absence of a dummy variable. Consider the stylized fitted regression log (A*/E) = + M D + ylogx (X) where A* is the predicted level of costs (or schedules) from regressions, E is the estimate of costs, D is a dummy variable (value I when the certain factor is present and zero when absent), and X is the level of some indicator variable. Greek letters denote estimated regression coefficients. Taking antilogs and expressing the equation in terms of the desired variable (A*) yields: A* = E exp(a) exp(od) X. (2) The elasticity of A* with respect to X is given by XoA/A6X and is equal to y. The impact on A*, for a given value of X and E, of the presence (versus absence) of the dummy variable factor is measured by the ratio exp(p)/ exp(0) = exp(1). A value of less than unity indicates that the presence of the factor lowers the predicted outcome. The elasticity of the predicted value (A*) with respect to the estimated value (E) is unity, except in the case where the variable X is itself the estimated value. In the latter case the elasticity is (I + y), which measures the "scale" effect between E and A*. 'Where the estimated value is the explanatory variable, the sensitivity coefficient (elasticity) is 1+ regression coefficient (see the note above).

75 Implications of the Analysis for Power System Planning 57 For the dummy variables, the impact of their presence is proportional solely to the coefficients given above. It can be seen that the most important factor leading to an upward adjustment in the prediction is the impact on hydro schedules of the project being an extension. Here the estimated value, adjusted by the other factors, needs to be multiplied by a factor of exp(0.502), that is, by The largest downward adjustment is to estimated costs for hvdro projects in Colombia, after allowing for effects of other variables, which need to be multiplied by exp(-0.423), that is, by For variables where elasticities are available, the importance of a factor depends not only on the coefficient but on the scale. The equation expressed in form (2) in the note to Table 7.1 shows clearly the nonlinear nature of the relationship. To determine which factor is the most important, it is necessary to consider how much such a variable might change between projects. Where a factor exhibits high percentage changes between projects, such a factor will be more important (for a given value of the elasticity) in explaining variations in the predicted values between projects. Hence, both the coefficient and the ratio of the indicator values are important. Risk and Planning Issues Besides showing the presence of systematic errors in estimating performance, the regression analysis has revealed that it is not possible to make perfect predictions of the outcomes of project costs and construction schedule from the factors used in the analysis and that the degree of uncertainty is large. The standard deviation of the ratio of actual to estimated costs was 36 percent, and for the ratio of actual to estimated schedules it was 42 percent. In predicting the ratio of actual to estimated values for projects categorized through the regression analysis, the estimated standard deviations of the errors varied between 16.9 percent for thermal costs (current) and 26.7 percent for thermal schedules. Insofar as the regression models have captured all systematic variation, these magnitudes can be regarded as measures of the risk associated with costs and schedules. They also allow expressions to be constructed for the probability distribution of outcomes for a particular project. Such expressions can be adapted for use in proprietary power system planning models that specifically analyze financial and economic risk for a sequence of investments in long-term expansion programs for power systems. 25 Where risk is to be explicitly 24. Comparing two projects, alike in all respects (including the estimated values) apart from indicator value X, the ratio of predicted values will be from equation 2 in the note to Table 7. 1: Al/A 2 = (X 1 /X 2 )'- 25. Some planning models treat key operating variables-such as power demand, fuel prices, and hydrology-stochastically to derive a distribution of possible outcomes for the net present value of a selected power development program. employing such techniques as Monte Carlo simulations. Stochastic representation of construction costs and schedules could conceivably be added to such models. The stochastic outcomes of alternative programs could then be compared by a straightforward statistical technique (see the subsection below on planning issues involving choices between sequences of projects).

76 58 Estimating Construction Costs and Schedules factored into project analysis, it is possible either to use a discount factor appropriate to the degree of risk involved or to use a riskless discount factor and add in adjustments for the risks involved to the benefit/cost stream such as those derived in this paper. The assumption that similar factors will affect the outcomes of costs and schedules for future projects as they affected the projects in the sample allows the variance of errors from the regression equation to provide a measure of risk for future projects (i.e., a quantifiable probability distribution of outcomes). There will of course be genuine uncertainty-eventualities for which no probability can be constructed from historic experience-but for planning purposes it is valuable to have measures of risk available. Traditional investment analysis, where the outcomes are risky and investors are risk averse, tends to focus on the explicit tradeoff between mean return (or cost) and the variance. Investors choose the highest available "equal value" contour between means and variances that reflects their underlying attitude toward risk. (In effect, this technique requires the introduction of an extra explicit evaluation criterion that allows the investor's attitude toward risk to be encapsulated in the analysis.) This technique is feasible where the composition of numerous investments in a portfolio can be continuously varied to achieve the desired risk profile. It is less directly applicable to the selection of power generation projects, where only a few alternatives are to be considered and where the decision is faced only periodically and concerns a relatively large investment. Nevertheless, the measures of mean cost and variance of costs (or schedules) derived in this paper can be used to examine important planning issues for power generation projects, as illustrated in the following subsections. Measurement of Project Risk If the project is such that risk is to be taken into account in the decision making process, then the regressions provide a measure of project risk arising from variability in construction cost and schedule estimates. This measure is the standard error of the regression. Standard analysis of publicly financed projects under uncertainty (Arrow and Lind 1994) shows that if returns from the investment are independent of other components of national income, the government should choose the project that maximizes the expected return when using a discount rate appropriate for investments with certain returns. That is, because project risk can be spread thinly over a large taxpaying population, the government should ignore uncertainty in evaluating public projects. Where the share of the risk borne by individual taxpayers is substantial relative to their income, this conclusion has to be modified. Similarly, if some of the risk accrues directly to individuals, this should be discounted for, just as if it were a private

77 Implications of the Analysis for Power System Planning investment:. Where the return of the project is likely to be correlated with national income, Little and Mirrlees (1974) provide an adjusted criterion for determining whether or not to undertake the project. This criterion also requires the evaluation of the risk of the project and the correlation between the project return and the level of national income. Distribution of Possible Project Outcomes The regressions provide estimates of the distribution of the possible outcomes due to uncertaina, in construction costs and schedules, which can be used in risk analysis. In deciding whether to go ahead with a given project (or not), or which of two mutually exclusive projects to select, the basic criterion focuses on the expected value of returns, and hence on the expected values of costs and schedules. Nevertheless, in cases where the financing of a major cost overrun or an overrun beyond a specific limit to available funds would impose an unacceptable budgetary strain, it is important to have a high level of confidence that the predicted value has been truly identified by the regression with the best performance (i.e., the one least likely to have omitted any systematic factor). However, it is recognized that the actual cost is a random variable as seen from the ex ante standpoint, whose mean is the predicted value from the regression. Moreover, the variance of this random variable can be estimated from the regression. If it is assumed that the variable used in the regression (the log of the ratio of actual to estimated values) is normally distributed, then the distribution of outcomes can be computed. With the aid of this distribution it is possible to calculate information that can be used as a supplement to the decision making process. Examples of such calculations are described in Annexes 8 and 9 for the following questions: a. Finding the cost level for a given project that will be exceeded with a specific probability and the probability of the project cost exceeding a specified value (similarly with project schedule). b. Choosing between projects on the basis of the probability of exceeding a cost limit, evaluating the probability of exceedance that has the same cutoff value for the projects. and evaluating which project has the lower probability of exceeding a given cost limit. c. Assigning probabilities for risk analysis to a low scenario, an expected scenario, and a high scenario, where values are assigned to these scenarios so that these values and probabilities are consistent with a prior view of the variance of cost or schedule outcome, such as obtained from one of the regression equations derived in this paper. 26. This distributional issue may also apply to the low-income groups that do not benefit directly from the project (i.e., lack access to the public electricity supply system) but that would be adversely affected by reduced government expenditures on social services caused by a cost overrun for a power project.

78 60 Estimating Construction Costs and Schedules Planning Issues Involving Choice between Sequences of Projects Analysis of a program for power system expansion can involve choices that require more selections than an alternative between single projects. Three cases that are of considerable practical importance are frequently encountered: a. The choice between one large project and a set of two or more smaller projects, where the reliability of estimates of costs and schedules depends on the scale variables identified in the regressions, and risks also depend on project size. b. The choice between two power expansion programs, each made up of a sequence of several projects, where the crucial issue is fuel diversity (e.g., between a hydropowerdominated program and a mixed hydro/thermal power program). c. The choice of whether or not to delay a project, where the crucial issue is often that of improving the estimates for construction costs and schedules for a project or its alternatives by providing more time for further investigations into such aspects as project site conditions (e.g., geology, topography, and environmental impacts). 27 The first two cases raise the technical issue of how to construct the prediction of the total program cost and its variance from values for the individual projects derived from the regression analysis. Annex 10 shows how the mean and variance of a sum of the cost of two projects are derived, as well as how to obtain the mean and variance for the predicted cost (or schedule) when the regression model provides a predicted value for the log of the ratio of actual to estimated values. The formulas in Annex 10 can be manipulated to explore the possibility that a series of smaller projects with approximately the same total cost and output as a single large project may have a rather lower variance of costs and hence be less risky. Where the risk of the project is an important factor, a strategy of risk reduction through the use of a series of smaller component projects may be an important planning option. The decision to delay a project to obtain better information on costs and schedules can be taken in the context of the types of risk analysis outlined in Annexes 8 and 9.28 If the 27. Delaying a project is sometimes proposed under concerns about uncertainty of future fuel costs and power market demand. An interesting case is the use of financial options analytical methods for a power generation project whose output is sold under competitive bidding, where the main concern is uncertainty about the power sales price. See EPRI (1995). Another interesting application of this approach is to the case of extending a transmission system, for which see Martzoukos and Teplitz-Sembitzky (1992). 28. In addition to the uncertainty about the performance of estimates at project appraisal that are based on feasibility studies, engineering design work, and contract bidding, as reported in this paper, this issue can also arise in the context of preappraisal decisions because it is generally presumed that the reliability of estimates of construction costs and schedules improves as a project progresses through successive preparation stages-identification, prefeasibility, feasibility, engineering design, and contract bidding. Intuitively this presumption appears realistic when just considering bias in the estimates. However, the substantial variance found for the performance of appraisal estimates should caution against overconfidence in obtaining better estimates by delaying projects to obtain more information about them.

79 Implications of the Analysis for Power System Planning 61 project appears to have too high a risk or an excessive cost or construction schedule, then it can be sensible to consider delaying the project while more evidence on estimated costs and schedule is collected about the project or alternatives. The delay option can then be evaluated in terms of the change in expected costs and schedules from present estimates by means of decision-tree analysis or by adapting the financial options approach. An application of the options approach to the optimal timing of projects under uncertainty about construction costs and schedules is developed in Annex 11. Unless there is specific evidence to the contrary, it is probably sensible to assume that there will be no change in the variance of estimates by delaying a project. Basic Recommendations Two basic recommendations for operational analysis emerge from the analysis of estimates for construction costs and schedules of power generation projects in developing countries. First, because methods of estimating costs and schedules have been overoptimistic, the robustness of the analysis should be tested by applying a correction to the estimates of costs and schedules. An "expected" value should be used for this test, and it can be calculated using the appropriate regression once the features of the project have been identified. Second, because the regression analysis shows that the uncertainty in predicting costs and schedules is also too large to ignore, even when expected values are used, it is also recommended that the economic and financial risks associated with the selection of a particular power project or power development strategy are explicitly considered during project appraisal. A measure of risk is provided by the product of the estimated value and the standard error of the regression equation, as shown in Annex 10. Risk should be considered in the context of specific questions rather than in an abstract context. There are some basic questions that should be examined to elicit valuable insights about the riskiness of power generation projects that are considered to be the least-cost option from the customary deterministic approach to power system planning. The paper proposes straightforward techniques for analyzing some of these questions (see the section above on risk and planning issues, as well as Annexes 8, 9, and 10). These techniques should be tested operationally and developed in case studies, so that guidelines can be formulated for using them in the appraisal of power generation projects.

80

81 Annex 1: World Bank-Supported Power Generation Projects Used for the Analysis of Cost and Schedule Estimating Performance Table A1.1 World Bank-Supported Power Generation Projects Used for the Analysis of Cost and Schedule Estimating Performance Installed Loan capacity approval Country Project name (MW) year Thermal power projects Afghanistan (Power I) Kabul Algeria Algiers Bangladesh Ashunganj Thermal Botswana Morupule Costa Rica (Power IV) San Antonia Costa Rica (Fifth Power Project) Moin Cyprus (Power III) Moni Station Unit No Cyprus (Power IV) Moni Station Unit No Ecuador (Third Power Project) Cumbaya Egypt (Power II & III) Shourbrah El Kheiam Guatemala Guacalate Guyana (G&T Project) Garden of Eden/Rotterdam Haiti (Power I) Varreux Haiti (Second Power Project) Varreux Haiti (Third Power Project) Carrefur Honduras Nispero Power Honduras (Fifth Power Project) La Ceiba India Second Kothagudem India Singrauli India Third Trombay India Korba India Ramagundam India Second Singrauli India Farakka India Second Ramagundam (continued on next page) 63

82 64 Estimating Construction Costs and Schedules (Table Al. I continued) Installed Loan capacity approval Country Project name (MW) year Thermal power projects (continued) India Second Korba India Fourth Trombay India Comb. Cycle: Kawas, Anta, & Auraiya Indonesia Power IV: Muara Karang No Indonesia Power VI: Muara Karang Indonesia Power VII: Semarang Harbor Indonesia Power VIII & IX: Suralaya Units I & Indonesia Power XII & XIV: Suralaya Units 3 & Ireland Power II: Tarbert Ireland Power III: Tarbert Jordan Power I: Zarqa Jordan Second Hussein Thermal Jordan Power V: Aqaba Power Station Korea Gojeong Power Malaysia Port Dickinson & Johore Bahru Thermal Malaysia Power IV: Prai & Port Dickinson Thermal Malaysia Power V: Port Dickinson Thermal Malaysia Power VII: Prai Thermal Extension Malaysia Power VIII: Pasi Gudang Pakistan Karachi 'C' Thermal Power Station Panama Power III: San Francisco Thermal Philippines Power IV: Bataan Thermal Plant Philippines Power V: Bataan Thermal Electric No Romania First Turceni Thermal Romania Second Turceni Thermal Sierra Leone Power II: King Tom Thermal Sierra Leone Third Power Project: King Tom Thermal Sri Lanka Power VIII: Sapugaskanda Sudan Sudan Power II: Burri & Juba Thermal Syria First Mehardeh Power Syria Second Mehardeh Power Thailand South Bangkok Thermal No Thailand Bang Pakong Thermal Power

83 Annex 1: World Bank-Supported Power Generation Projects 65 Installed Loan capacity approval Country Project name (MW) year Thermal power projects (continued) Thailand South Bangkok Thermal No Turkey Elbistan Uruguay Power IV: Battle Unit No Yemen Wadi Hadramout Power Project Yemen Power II: Wadi Hadramawt Thermal Zimbabwe Power I: Hwange II Hydropower projects Argentina El Chocon Bolivia Second Empresa Nacional de Electricidad Brazil Estreito Brazil Xavantes Brazil Volta Grande Brazil Porto Colombia Brazil Marimbondo Brazil Salto Osorio Brazil Sao Simao Brazil Paulo Afonso IV China Lubuge Colombia Third Medellin Colombia El Colegio & Conoas Colombia Chivor Colombia Guatape Second Colombia First San Carlos Colombia Las Mesitas Colombia Second San Carlos Colombia Guadalupe IV Colombia Playas Costa Rica Fifth Power: Rio Macho & Cachi Ecuador (Third Power Project) Nayon Ethiopia Finchaa Fiji Manasow-Wailou Ghana Second Volta River Authority Ghana Kpong Hydroelectric (continued on next page)

84 66 Estimating Construction Costs and Schedules (Table Al. I continued) Installed Loan capaciry approval Country Project name (MW) year Hydropower projects (continued) Guatemala Aguacapa Guatemala Chixoy Honduras Fifth & Sixth Power: Rio Lindo Honduras Nispero Power Honduras El Cajon Iceland Sigalda Indonesia Power X: Saguling Ireland Pumped Storage Kenya Kamburu Kenya Gitaru Kenya Kiambere Lao-PDR Nam Ngum Madagascar Andekaleka Malawi Tedzani Sate I Malawi Nkula Falls II Malaysia Power IX: Bersia & Kenering Morocco Sidi Cheho-AI Massira Myanmar Kinda (Nyaunggyat Multipurpose) Nepal Kulekhani Panama (Second Power) Bayano Panama Fortuna Papua New Guinea Upper Ramu Peru Matucana Power Portugal (Power Project VII) 8 Hydro Plants Romania Riui Mare-Retezat Sudan Roseires Sudan (Second Power) Roseires Extension Sudan Roseires Extension Swaziland Third Power: Lupohlo-Ezulwini Tanzania Kidatu Hydroelectric Stage I Tanzania Kidatu Hydroelectric Stage II Tanzania Power IV: Mtera Thailand Ban Choa Nen Srinagarind

85 Annex 1: World Bank-Supported Power Generation Projects 67 Installed Loan capacitv approval Countrv Project name (MW) year Hydropower projects (continued) Thailand Pattan I Thailand Khao Laem Thailand (Power Subsector Project) Lan Suan and Chiewlarn Turkey Third & Fourth Cukurova Power Turkey Karakoya Turkey Sir Yugoslavia Middle Neretva Hydro: Grabovica and Salakovac Dams Yugoslavia Middle Neretva Hydro: Mostar Dam Yugoslavia Visegrad Zaire Ruzizi II Zambia Kariba North Zambia Kafue Hydroelectric Stage II

86

87 Annex 2: Regression Results with All Variables Included Table A2.1 Variables for Log of Thermal Power Project Costs (Current Values) Based on 42 Observations Regression Standard 2-tailed Variable coefficient error t-stat. significance Intercept Log loan date dummy Log estimated cost Log estimated schedule Log forex Log per capita income Log station size Log % ICB Log MUV growth Log GDP deflator Log unit size Basis for costs dummy Diesel dummy Coaldummy Steam dummy Extension dummy Log sales growth Log agencies Contractor dummy dummy India dummy Log estimated civil R-squared Mean of dependent SER SD of dependent Linearized R-squared

88 70 Estimating Construction Costs and Schedules Table A2.2 Variables for Log of Thermal Power Project Costs (Constant Values) Based on 55 Observations Regression Standard 2-tailed Variable coefficient error t-stat. significance Intercept Log of loan date Dummy for Log of estimated cost Log of estimated schedule Log of forex Log per capita income Log station size Log % ICB Log of MUV growth Log GDP deflator Log of unit size Basis for cost dummy Diesel dummy Coal dummy Steam dummy Extension dummy Log sales growth Log agencies Dummy for India dummy R-squared Mean of dependent SER SD of dependent Linearized R-squared 0.601

89 Annex 2: Regression Results with All Variables Included 71 Table A2.3 Variables for Log of Hydropower Project Costs (Current Values) Based on 50 Observations Regression Standard 2-tailed Variable coefficient error t-stat. significance Intercept Log loan date dummy Log estimated cost Log estimated schedule Log forex Log per capita income Log station size Log % ICB Log MUV growth Log GDP deflator Log unit size Basis for costs dummy Extension dummy Log sales growth Log height ( new) Log head (new) Log agencies Colombia dummy Brazil dummy Tunnel length Contractor dummy Log civil ratio dummy R-squared Mean of dependent SER SD of dependent Linearized R-squared 0.631

90 72 Estimating Construction Costs and Schedules Table A2.4 Variables for Log of Hydropower Project Costs (Constant Values) Based on 45 Observations Regression Standard 2-tailed Variable coefficient error t-stat. significance Intercept Log of loan date Dummy for Log of estimated cost Log of estimated schedule Log of forex Log per capita income Log station size Log % ICB Log of MUV growth Log GDP deflator Log of unit size Extension dummy Log sales Log dam height Log head Log agencies Dummy for Brazil dummy Colombia dummy R-squared Mean of dependent SER SD of dependent Linearized R-squared 0.577

91 Annex 2: Regression Results with All Variables Included 73 Table A2.5 Variables for Log of Thermal Power Project Schedules Based on 42 Observations Regression Standard 2-tailed Variable coefficient error t-stat. significance Intercept Log loan date dummy Log estimated cost Log estimated schedule Log forex Log per capita income Log station size Log % ICB Log MUV growth Log GDP deflator Logunitsize Basis for costs dummy Diesel dummy Coal dummy Steamdummy Extension dummy Log sales growth Log agencies Contractor dummy dummy India dummy Log estimated civil R-squared Mean of dependent SER SD of dependent Linearized R-squared 0.659

92 74 Estimating Construction Costs and Schedules Table A2.6 Variables for Log of Hydropower Project Schedules Based on 50 Observations Regression Standard 2-tailed Variable coefficient error t-stat. significance Intercept Log loan date dummy Log estimated cost Log estimated schedule Log forex Log per capita income Log station size Log % ICB Log MUV growth Log GDP deflator Log unit size Basis for costs dummy Extension dummy Log sales growth Log height ( new) Log head ( new) Log agencies Colombia dummy Brazil dummy Tunnel length Contractor dummy Log civil ratio dummy R-squared Mean of dependent SER SD of dependent Linearized R-squared 0.719

93 Annex 3: Comparisons of Actual Ratios and Predicted Ratios from Regressions for Costs and Schedules Figures A3.1 to A3.6 are plots of actual ratio (actual value to estimated value) and predicted ratio (predicted value from one of the regression equations given in chapter 6 to estimated value) for each power generation project, for each of the following variables: * Thermal power project costs (current values) * Thermal power project cost (constant values) * Hydropower project costs (current values) * Hydropower project costs (constant values) * Thermal power project schedules * Hydropower project schedules The numbers along the horizontal axis are references to individual projects. The numbers along the vertical axis are values of the ratios. 75

94 76 Estimating Construction Costs and Schedules Figure A3.1 Plot of Actual and Predicted Ratios for Thermal Costs (current) Ratlo of actual to predicted value ' 1.60" 1.40' Actual ratio A Predicted ratio ) Project number (ordered by date of loan agreement) Figure A3.2 Plot of Actual and Predicted Ratios for Thermal Costs (constant) Ratio of actual to predicted value 1.80 _ a- - Actual ratio A-- Predicted ratio 0.40 _ Project number (ordered by date of loan agreement)

95 Annex 3: Comparisons of Actual and Predicted Ratios 77 Figure A3.3 Plot of Actual and Predicted Ratios for Thermal Schedules Ratio of actual to predicted value 2.40 _ NE Actual ratio 1.2 D Predicted ratio Project number (ordered by date of loan agreement) Figure A3.4 Plot of Actual and Predicted Ratios for Hydro Costs (current) Ratio of actual to predicted value Actual ratio -- Predicted ratio Project number (ordered by date of loan agreement)

96 78 Estimating Construction Costs and Schedules Figure A3.5 Plot of Actual and Predicted Ratios for Hydro Costs (constant) Ratio of actual to predicted value HT-Actual ratio - Predicted ratio 1.00; _ Project number (ordered by date of Ioan agreement) Figure A3.6 Plot of Actual and Predicted Ratios for Hydro Schedules Ratio of actual to predicted value O Actlal ratio 1.40 A Predicted ratio Project number (ordered by date of loan agreement)

97 Annex 4: Statistics for Single-Variate Analysis of All Variables This annex contains schedules of single-variate correlations for all the variables included in the regression equations. These correlations are presented to show how the variables that are found to be significant from single variate analysis differ from those found to be significant from multivariate analysis. There is an equation that links the (unsquared) correlation coefficient to a t statistic that would be obtained in the single-variable regression: 20.5 r = t (t2 +N-2) t = r{(n -2) (I - r 2 (N is the number of observations) Hence a critical value of the correlation can be obtained from a t table. For example, with 54 observations, the critical value of t (two-sided test) at 90 percent is 1.67, and at 95 percent it is The corresponding critical values for r are thus (plus or minus) and Larger values than 0.05 for r 2 thus indicate a significant correlation between the variables at the 90 percent confidence level. In other words, finding a small r 2 is equivalent to finding a nonsignificant t-stat. 79

98 80 Estimating Construction Costs and Schedules Table A4.1 Single-Variate Regression Correlations Thermal power projects Hydropower projects Cost overrun Cost overrun Variable (current) Schedule slip (current) Schedule slip Log loan date * * Post 1970 dummy Log estimated cost 0.508* * * Log estimated schedule * * * Log forex * Log per capita income * Log station size * Log % ICB Log MUV growth 0.247* 0.230* 0.372* Log GDP deflator * Log unit size * * Basis for cost dummy * Diesel dummy 0.285* 0.300* - - Coal dummy * Steam dummy Extension dummy * Log sales growth Log agencies * Post 1976 dummy * * India dummy * - - Local contractor dummy Log civil costs ratio" 0.366* * Log dam height (new) * Log head (new) * Colombia dummy Brazil dummy * Tunnel length Note: Number of observations for all the regressions: 54. Significant value of correlation r at 90 percent confidence level: ± * Variable is significantly correlated at 90 percent confidence level. 45 observations.

99 Annex 4: Statistics for Single-Variate Analysis of All Variables 81 Table A4.2 Comparison of Significant Variables at 90 Percent Confidence Level between Multivariate Analysis and Single-Variate Analysis Relationship Thermal power projects Hydropower projects between Cost overrun Cost overrun variables (current values) Schedule slip (current values) Schedule slip Common Log civil costs ratio (+) Post 1976 dummy (-) Log hydraulic head (+) Extension dummy (+) vagriable Log estimated cost (-) India dummy (-) Log forex (-) Log estimated schedule (-) with same Post 1976 dummy ( Log GDP deflator (+) Log national income (+) signs (I I) Common Log estimated schedule variables (+ for multi. - for single) but opposite signs (I) Significant Extension dummy (-) Log station size (-) Extension dummy (+) Log hydraulic head (+) for multivariate, not India dummy (+) Log loan approval date (+) Log estimated cost (-) Log % ICB (-) significant Log forex (-) Log station size (+) for single Post 1970 dummy (-) Log financing agencies (+) variate (13) Colombia dummy (-) Significant Log loan approval date (-) Log estimated cost (-) Log loan approval date (-) Log estimated cost (-) for single Log station size LLogestimatedschedule(-) Log MUV growth(+) Log dam height(-) variate, not significant Log MUV growth (+) Log MUV growth (+) Basis for cost dummy (+) for multivariate (19) Log unit size Log unit size (-) Brazil dummy (+) Diesel dummy (-) Diesel dummy (-) Log civil costs ratio (+) Coal dummy (-) Log financing agencies (-)

100

101 Annex 5: Ex Post Attribution of Factors Responsible for Schedule Slip in World Bank- Supported Power Generation Projects Table A5.1 Ex Post Attribution of Factors Responsible for Schedule Slip in World Bank-Supported Power Generation Projects Responsible party orfactor Client/engineer Contractor/supplier Uncontrollable events Specificfactor or event Thermal power projects Legal requirements/bureaucratic procedure for awarding contracts Initial schedule was too optimistic Bid evaluation difficulties Delays in procurement/placement of orders Change in project scope Modifications to major equipment required Disagreement between Bank and borrower over contract award Site change Labor disputes/strikes in manufacturer's country Labor disputes/strikes in project country Shipping delays due to oil crisis Substandard work had to be redone Equipment failure during testing Skilled labor shortage Manufacturing difficulties Shortage of materials Contractor inefficiency/lack of coordination Technical problems with equipment Contractor bankruptcy Transportation difficulties Damage/need to redesign civil works due to earthquake or other natural Disaster Unusually bad weather Accident-damage to equipment Political turmoil/coup/invasion Civil disturbance (continued on next page) 83

102 84 Estimating Construction Costs and Schedules Responsible party orfactor Client/engineer Supplier/contractor Uncontrollable events Specific factor or event Hydroelectric power projects Initial schedule too optimistic Geological problems Financial difficulties/tariff implementation problems Inefficient project management/institutional weakness Design changes Change in project scope Relocation problems Delay in award of contracts/bid evaluation difficulties Design faults Procurement delays/difficulties Site change Land acquisition/site access problems Communication problems No bids received due to working conditions Major currencv devaluation threatened project viability Project sponsors backed out Legal problems/delay in settling claims World-wide inflation Contractor bankruptcy Contractor inefficiency/inexperience/incompetence Delays in shipping/deliver of equipment, transportation difficulties Substandard work had to be redone Shortage of materials Manufacturing difficulties Dam or tunnel collapse Damage to equipment (other than dam/tunnel) Water infiltration/pressure damage Fire Equipment failure during testing Labor disputes/strikes in manufacturer's country Fuel shortage Labor disputes/strikes in project country Communication problems Skilled labor shortage Change in contractors Legal problems/delay in settling claims Landslides/mudsl ides/rockfalls Unusually bad weather Political turmoil/coup/invasion/war Flood damage Earthquake Note: Factors attributed in World Bank Project Completion Reports.

103 Annex 6: Methodology for Deriving Actual Project Costs in Constant PriceTerms Since World Bank project completion reports do not provide data on project costs in constant-price terms and the actual disbursement patterns over the project implementation period, the methodology described below is used to derive equivalent actual constantprice costs from the actual current-price costs given in these reports. This methodology is the reverse of the World Bank's methodology for computing the price contingency for project cost estimates. a. Total actual project costs are allocated among the project implementation years according to standard disbursement profiles related to the total project implementation periods. These profiles are shown on the next page. b. The actual annual disbursements are divided between foreign exchange and local currency costs by the proportion of each in the total actual project cost. c. The stream of annual foreign exchange costs in U.S. dollar (US$) terms is converted to constant-price terms in the project start year with the actual MUV indices for the implementation period. d. The stream of annual local currency costs in local currency terms is converted to constant price terms in the project start year with the actual country CPI or GDP deflator for the implementation period. The local currency cost in constant price terms is converted into equivalent US$ terms at the average exchange rate in the project start year. e. The total project cost in constant US$ terms for the project start year is the sum of steps (c) and (d). A worked example is given in this annex. 85

104 86 Estimating Construction Costs and Schedules Table A6.1 Standard Disbursement Profiles for Project Cost in Current Price Terms Annual disbursement (of total) in years Implementation periods (years) SUM Source: These profiles are based on the generic formula for expenditure flow patterns for large capital projects given below. The formula was used by Independent Project Analysis, Inc. of Reston, Va., in a report published by the Industry and Energy Department of the World Bank (Merrow and others 1990). Proportion of total expenditure that occurs in year i of a total project construction period of I years: i [L (Cos )4.08 ]

105 Annex 6: Methodology for Deriving Actual Project Costs 87 Table A6.2 Example of Project Cost Derivation in Constant Price Terms: Algeria, Base Year 1973 Local Forex cost percent percent Foreign Annual Annual MUVfor cost in Annual local Local cost Annual forex foreign Year I local Average cost Local In Year I cost cost costs price terms cost annual (million CPI price terms Schedule (USS (US$ (Year] (US$ (USS exchange rate of (Year I = (million of year million) million)a 1.00) million) million)t (Dinnar/US$) Dinnar) 1.00) Dinnar) TOTAL Local Year I Price and Exchange Rate: US$ million Forex percent b Local cost percent In Year I Price Terns: USS million Total foreign cost Total local cost Total cost Forex = foreign exchange.

106

107 Annex 7: Analysis of Relationships for the Performance of Price Contingencies 1 Relation between Actual and Estimated Current Costs The first relationship serves to highlight the nature and extent of the difference between actual and estimated current costs. Were they always approximately equal, project appraisal could rely confidently on the estimated costs. Where these series are different, it is important first to be aware of the possible magnitude of the difference and next to analyze it for predictable features to be incorporated into project appraisal. For the group of projects as a whole, the regression of the actual cost in current price terms (CU(A)) on the estimated cost in current price terms (CU(E)) gave the result shown in equation A7. 1: CU(A) = CU(E) (1.7) (20.8) (A7. 1) R2= 0.76, Standard error of estimate (SEE) = 296; t statistics are in parentheses. If the estimated cost is on average an unbiased estimate of the actual value, then equation A7.1 should satisfy the hypothesis that the intercept is zero and the slope is unity. An F test comparing equation A7. 1 with the equation: CU(A) = CU(E) gives an F value of 0.55, compared with the critical value of However, the standard error of estimate, which is the average difference between the actual current cost and the value predicted by this relationship (and is the same as the standard error of the regression), is extremely large at 296 when compared with the sample mean actual current cost of 216 (in million US$). Moreover, the residuals are strongly heteroskedastic (being larger in absolute values at larger values of estimated cost), so that valid inferences cannot be based on the standard F test. The mean value of the ratio of actual to estimated current costs for the whole sample is 1.21, and this is significantly greater than unity, as shown by equation A7.2: CU(A)/CU(E) = 1.21 SEE = 0.40 (35.5) (A7.2) Once the data is put into ratio terms, the absolute values of the residuals do not exhibit heteroskedasticity, and so standard significance tests on equation A7.2 are valid. From this analysis it can be concluded that the actual and estimated current values are strongly, but far from perfectly, correlated. It appears that the ratio of actual to estimated values is significantly greater than unity and that there is a significant percentage bias in 89

108 90 Estimating Construction Costs and Schedules the estimation of current prices. There is a very large difference in the ratio for thermal power projects (1.06) and hydro projects (1.27), which indicates that the bias in estimating thermal power project costs in current price terms has been relatively small. Both groups have substantial standard deviations, but again that for hydropower projects (38 percent) is higher than that for thermal projects (23 percent). This difference can be due to differences in predicting either the physical costs (constant prices) or the inflation rates, but is likely to be primarily due to the former, since it would be expected that errors in forecasting the latter would not depend on the type of project involved. 2 Link between Actual Current Costs and Actual Constant Costs In order to analyze the sources of error in current price forecasting, it is necessary to look at the error in forecasting the constant price cost, as well as the errors in forecasting inflation. In order to check this physical error, it is important to check that the value derived for the actual constant cost from the disbursement formula described in Annex 6 is strongly related to the actual current value. If these two series are only weakly linked, then doubt would be cast on the construction of the actual constant price series and, as a result, analysis of the success in predicting the "physical" aspect of the project, by comparing actual and estimated constant costs, would be of less value. The regression includes as explanatory variables not only the actual constant price (CO(A)) but also the actual (annual) rate of inflation (ID), the actual (annual) rate of inflation of imported manufactures (IF), and the actual schedule length (SA). All these variables are involved in the relation between actual costs in constant prices and actual costs in current prices. The regression on the set of observations for which there are data is given in equation A7.3: CU(A) = CO(A) ID (14.8) (0.63) (A7.3) IF SA (4.1) (2.0) R2 = 0.74, SEE = 153 This equation shows that there is certainly a strong relation between the constructed actual constant cost value and the actual current cost value, allowing for actual inflation rates and schedule length. Separate regressions for thermal and for hydropower plants show that the relation is extremely close for thermal power projects (R 2 = 0.98) and is weaker for hydropower projects (R2 = 0.75). Since none of these "explanatory" variables can be known in advance, the equation cannot be used in a direct, predictive fashion. However, it does justify the attempt to separate out the inaccuracy in estimating the "physical" or constant cost dimension of the project from errors in estimating inflation rates and schedule lengths.

109 Annex 7: Analysis of Relationships for the Performance of Price Contingencies 91 3 Relation between Actual and Estimated Cost Escalation In order to check the extent to which errors are made in predicting actual current prices because of failures to anticipate cost inflation correctly (as opposed to the physical aspects of the project), the actual cost escalation, RA = CU(A)/CO(A), is regressed on the estimated cost escalation, RE = CU(E)/CO(E), in equation A7.4. RA= RE (7.3) (22.1) (A7.4) R2 = 0.79, SEE = 0.20 This relation appeared stable over time, in that the introduction of extra variables such as the date of the loan agreement or a post-1976 dummy 3 0 make only very slight changes to the fitted equation. Separation of the thermal power and hydropower project subsamples showed little difference in the relationship for types of plant, which is to be expected, since the failures to predict the inflation component should not depend on the type of plant but rather on assumptions that are common to all plant types. Failure to predict cost escalation correctly suggests that there is an aspect of the current cost forecast that is always likely to be in error, unless general rules for predicting domestic and international inflation can be improved. The results show that the World Bank's procedures were fairly accurate in predicting cost escalation, but that with an average error of 0.20, compared with the sample mean actual cost escalation of 1.14, there was still room for improvement in the construction of this aspect of the project price forecast. 4 Relation between Cost Overrun in Current Price Terms, in Constant Price Terms, and Errors in Predicting Inflation Rates and Project Schedules The themes of the three previous sections are pulled together by regressing the cost overrun in current price terms O(CU) on the cost overrun in constant price terms (O(CO)) and the errors in predicting foreign inflation (O(F)), domestic inflation (0(D)), and schedule (O(S)) in equation A7.5 to give: O(CU)= (CO) O(D) (F) (1.10) (12.4) (2.0) (1.6) (A7.5) O(S) (2.2) R2= 0.73, SEE = The estimated cost in constant price terms (CO(E)) includes the physical contingency for cost increases that is mentioned in section See main text footnotes 8 and 9 about changes in the World Bank's guidelines.

110 92 Estimating Construction Costs and Schedules Equation A7.5 shows that errors in estimating the project cost in constant price terms, errors in estimating the domestic inflation rate, and errors in estimating the project schedule were all significantly related to the overall error in estimating project costs in current terms. There was no significant relation with the error in estimating the imported rate of inflation; this finding does not mean that if an error were made in estimating the imported inflation rate (Figure A7. 1), such an error would not be reflected in total project cost errors, but rather that such errors were on average sufficiently small to be dominated by the other sources of error. The overall goodness of fit is quite high and indicates that this decomposition is fairly successful in identifying the sources of the cost overruns by type. Disaggregation into thermal power and hydropower project subsamples again shows a stronger fit for thermal power projects (R 2 = 0.85) and a weaker fit for hydropower projects (R2 = 0.67), which suggests that the attempt to split the cost overrun into its components is rather less satisfactory for hydropower projects. This may well relate to the basic problem identified earlier in predicting the physical aspects of costs for hydropower projects correctly. Figure A7.1 Comparison of MUV Index Actual Values with Values from Forecasts Made between 1974 and 1988 (Based on actual value in 1980 = 100) Year Note: Forecasts used for World Bank Project Appraisals of the Unit Value Index (in US$ terms) of manufactured goods exported from G-5 countries (France, Germany, Japan, UK., and U.S.) to developing countries.

111 Annex 7: Analysis of Relationships for the Performance of Price Contingencies 93 Appendix A7.1 Composition of the World Bank's Price Contingency Formula for Predicting Project Cost Escalation The World Bank's formula for computing price contingencies is: EE= B *C*pf +Cd *pd *ed where: EE i B is the estimated price contingency for escalation in project costs. is a year in the project implementation schedule of n years. is the estimated total project cost expressed in base year prices (i=l). Bi is the proportion of B that is committed in year i. cf d Cd f is the estimated proportion of Bi that consists of imported components. is the estimated proportion of B, that consists of domestically procured components, and thus equals ( I - C-). is the projected price index in year i relative to the price index in the project base year for imported project components, which is taken to be the forecast change in UN Unit Value Index (in US$ terms) of manufactured goods exported from the G-5 countries to developing countries (the "MUV Index"). p ed is the projected price index in year i relative to the project base year for the domestic economy, which is usually taken to be the domestic Retail Price Index ("RPI"). is the forecast change in the exchange rate in year i relative to project base year for the domestic currency in US$ terms (for estimating a price contingency in US$ terms).

112

113 Annex 8: Computations of Probabilities of Exceeding Specific Project Costs Case 1: The cost level for a given project that will be exceeded with a specified probability. If a variable (X) follows a normal distribution N(m,s), where m is the mean and s the standard deviation, there is a 50 percent chance that the actual value will exceed the mean. As a control of unlikely outcomes it is possible to calculate the "cut-off' at which there is only a specified (usually small) probability of exceeding this value. For example, the "cut-off' (K) value at which there is only a 10 percent chance of a higher outcome is given by the solution to the equation 0.1 = fx(m,s)dx K which is equivalent to the equation expressed in standard normal terms: 0.1 = X(O,l)dx (K- m)/s The ordinates that solve such an equation are given in standard tables of the normal distribution. For example, the value of the standardized score that cuts off the top 25 percent of the distribution is Hence, for known values of m and s, the cut-off value for X itself is K = m *s This approach can be applied to the Indian power project discussed in chapter 6. The predicted log of the ratio of actual to estimated costs was 0.192, while the standard error of the regression was (Appendix A8.1 to this annex shows how to compute the variance of a forecast). Hence, the cut-off value of the log of the ratio, which has only a 25 percent chance of being exceeded, is: K= * 0.189=0.320 The cut-off value of the ratio itself is thus and, given that the estimated value of costs was $406 million, the cut-off predicted actual value is $559 million, as compared with the central predicted value of $492 million. The regression thus takes the estimated value ($406.29) and produces a predicted value of $492 million, which is the appropriate figure for standard econornic evaluation. It also indicates that the value at which there is only a 25 percent chance of a higher cost is $559 million, and this finding can be used in risk analysis. This approach can be inverted to ask for a given cost limit, what is the probability of the project cost exceeding such a value. The fornal equation is: 95

114 96 Estimating Construction Costs and Schedules X(m,s)dX f3= L where L is the specified limit, and 1 is the probability of exceedance. Again this is put into standard normal form: X(O,l)dX f= (L-m)/s and, for a given standardized limit, the value of 1 can be derived from standard normal tables. In the above example, if the cash limit were set to $575 million, this implies the ratio of actual to estimated values would be 1.416, with a log of The standardized value would then be ( )/0.189 = 0.825, which has a 20 percent chance of being exceeded. Hence with a cash limit of $575 million, on a project estimated to cost $406 million, there is still a 20 percent chance of some overrun beyond the cash limit. Case 2: Chzoosing between projects on the basis of the probability of exceeding a cost limit. The approach indicated above for augmenting the analysis of a single project can be extended to the choice between two (alternative) projects. The standard question is: which project has the lower cut-off value of the distribution of costs that will be exceeded with a 50 percent chance in each case; and the answer is simply the lower of the predicted outcomes. Alternatively, the question can be focused on the smaller probability level of exceeding a higher cost level (K) than the predicted outcomes. Consider projects with means ml and m2, and standard deviations s, and s 2. With a cut-off probability of 25 percent the standardized score for each is, as before, Hence the cut-off cost levels are: K, = ml * si K 2 = m * s, It is then desired to choose that project with the lower value of K. If the values of the standard deviations are different then it is possible that a project with the higher mean cost ratio might nevertheless have a lower K value, because of its smaller standard deviation. It also is possible to calculate the probability (P) of exceedance that has the same cutoff value (H) for each project (H = Ki). At this value: ml + H * si = m2 + H * S2 H=(ml -m2)/(s2-si) The standard table for a normal distribution then indicates the probability of exceeding the common cut-off value of H.

115 Annex 8: Computations of Probabilities of Exceeding Specific Project Costs 97 This approach also can be inverted to ask which project has the lower probability of exceeding a given cost limit R. Since the expected values will in general be different for the two projects (El and E 2 ), the logs of the ratios of the cash limit to expected values (r 1 and r 2 ) will also be different. Putting both ratios into standardized form then allows the probability of exceedance of the absolute cash limit for project I to be compared with that for project 2. If project I has the lower probability, then fx(0, I)dX < f X(O, I)dX (ri-mo)isi (r 2 -m2)5s 2 When there is indifference between projects (i.e., the same probability of exceeding R applies to both projects), the limits of integration are equal, so that (r, - ml) / s = (r 2 - Mi 2 ) / S2 This expression can then be used to determine the specific value of R for indifference between projects, since it yields [ln(r/e 1 ) - ml]/s, = [ln(r/e 2 ) - m 2 1/S2 or R = exp [( s 2 (0nEI + ml) - s 1 (lne 2 + M 2 ) 1/ts2 - si }] A final criterion of interest is to ask what is the probability that project I will be more expensive than project 2, even when the mean value for project 1 is lower. The answer depends on the evaluation of a double integral (details are shown in Appendix A8.2 to this annex). Table A8. 1 gives some selected values for different parameter combinations. Table A8.1 Probability that Project 1 Has Higher Cost (Schedule) than Project 2 V W Where: V = (m 2 - m)s, W = s2/sl. An important result is that whenever m) is larger than ml, the probability that project I is more expensive has as an upper limit 0.5 as the ratio of s2 to si increases. For low values of this ratio, the chance that project 1 is more expensive can be very small.

116 98 Estimating Construction Costs and Schedules Consider projects where the value of m 2 is 0.2 and ml is 0.16, while S2 is 0.05 and s, is The parameter V is 0.8, and W is 2.5. The chance that project 2 is more expensive is 39 percent, despite the very much greater standard deviation. The table also assumes that there is no correlation between the outcomes of the alternative projects. Calculations showed that allowing for substantial positive or negative correlations had little effect on the probabilities, so that this issue could be ignored. Appendix A8.1 The Variance of a Predicted Value from a Regression The predicted value YF is based on regression of a sample size T between J known variables (XI... XJ) including the constant (XI = I all t) and the dependent variable. The predicted outcome is given by YF ~F = EJ I1 jx;f where the f3j denote estimated parameter values, and XjF are the assumed known values of the indicator variables. The variance of the forecast around the actual value is given by the standard formula: VarYF = a 2 + a 2 {E i; (XjFXiF) COV (Pi 3 j)) where Cov (fipj) is the estimated covariance or variance (i=j) of the regression parameters. For large samples, such as are used in the regressions in this paper, the terms in braces tend to be much smaller than unity, so that the variance of the forecast will be approximately equal to the residual variance (square of the standard error of regression). In comparing projects with similar mean outcomes, small differences in the variance can become important, and in such a case it would be more important to estimate the variance correctly. This facility is provided on modem regression programs. For example, the case of the India project referred to in the text has a variance of the residuals of = (for the log of the ratio) from the regression, while the true variance of the predicted value is Appendix A8.2 Probability that the Outcome of One Project Is Greater Than That of the Other Let X and Y be the random outcomes of the two projects, which follow a bivariate normal (correlated) distribution f(x,y). The probability that a drawing X (project 1) is larger than the Y value (project 2) depends on the integral: f f f(x, Y) dx DY 0 Y

117 Annex 8: Computations of Probabilities of Exceeding Specific Project Costs 99 This can be transformed to standard bivariate form: J f g(x, Y) dx dy - wy+v where g denotes the standard bivariate normal distribution, and V = (M 2 - m1 )/si, W = S2/SI. For particular values of V and W, the bivariate normal distribution is used to evaluate the above double integral (by a numerical method), and the results are shown in Table A8.1 for an uncorrelated distribution. Allowing the correlation coefficient to take values of -0.5, 0.0, or +0.5 made very little difference to the results.

118

119 Annex 9: Assigning Probabilities to Scenarios for Risk Analysis In the analysis of risky projects, it is necessary to choose alternative scenarios and to attach probabilities to these scenarios. For example, a project will have an expected cost outcome, and it is desired to investigate high and low outcomes and assign probabilities to all three cases. This method can then be extended to other planning variables such as demand growth, to give a multivariate probability of joint outcomes (e.g., high cost, low demand). The issue is how to combine the values chosen for the three scenarios with probabilities to be attached to them. The combination of the range of values chosen and the probabilities imply a variance to the set of outcomes. Since there is a prior view on the variance of cost or schedule outcome from the regression equations in this paper (and there may also be a view for the variance of the demand growth rate), it would be sensible to ensure that the sets of values chosen are consistent with these variances. This annex shows how this assignment can be done. Consider an expected cost outcome of M (middle). This is the expected value derived from the project analysis, adjusted if necessary from an estimated value by the regression equation. The analysis suggests that the variance of cost levels is a 2 M 2, where a 2 is the variance from the regression errors. Alternative scenario values L (low) and H (high) are to be chosen, together with probabilities for all three cases. To simplify the analysis it is assumed that the low and high outcomes are equidistant from the mean (expected) outcome. This in turn implies that, for the mean of the distribution to equal M, the probabilities of high and low cases must be equal. Let the probability of the middle case be ir, so that the probabilities for the other two cases are each (1 - ir )/2. The key insight is that if the variance across these three outcomes is equal to the known value or 2 M 2, there is a relation between the relative distance of L from M and the probability that must be assigned to the middle value itself. Let the value of L be expressed as A M ( where A will be less than unity), then the relationship is given by the formula (see the technical note at the end of this annex for derivation): Ir= I-{r/(l-A)} 2 (A9.1) An illustration is given using the India 911 example. The central cost estimate using the regression approach is (M) with the variance of the regression (C5 2 ) equal to Table A9. 1 shows combinations of A and ir that would produce a set of L, M, and H values with variance (a 2 M 2 ) equal to 8460 (the variance of the cost level). 101

120 102 Estimating Construction Costs and Schedules Table A9.1 Pairs of Parametric Values that Fit the Required Regression Variance of for Scenarios where the External Variance Is a Function of the Size of the Middle Value The above table shows, for example, that with a value of A of 0.75, which corresponds to a low cost of 368 (491.5 x 0.75) and a high cost of 614 (491.5 x 1.25), the probability that must be assigned to the middle value outcome is 44 percent, while both low and high outcomes must each have a probability of 28 percent. It can also be seen that if the high and low values are taken too near the medium value, there exists no set of probabilities that would yield a variance over the three outcomes equal to 8460, as predicted by external analysis. In the case where the variance of outcomes is not proportional to the value of the mean outcome, then a different formula is required. In this case the relation between the probability of the central case and the range of cases (as expressed by A ) is: 7r=l-{C/[(-_A)Mj) 2 (A9.2) where M is the value of the middle estimate. Consider a central growth rate of 0.04 (4 percent per annum), where the variance of growth rates of demand has been established from other studies to be Table A9.2 gives the corresponding tradeoffs. Table A9.2 Pairs of Parametric Values that Fit the Required Regression Variance of 0.01 for Scenarios where the External Variance Is Not a Function of the Size of the Middle Value A Hence when A = 0.600, a low-growth-rate scenario of and a high-growth-rate scenario of 0.056, combined with a probability for the middle-scenario-growth rate of 60.9 percent, will provide a set of growth rates and probabilities with variance 0.01.

121 Annex 9: Assigning Probabilities to Scenarios for Risk Analysis 103 Probabilities can be simply multiplied to obtain joint events provided that the cost distribution is independent of the demand distribution. Combining the two examples gives the probability matrix in Table A9.3 that has a variance for costs of 8460 and a variance for demand of Table A9.3 Probabilities for Scenarios with Predetermined Variances for Costs and Demand Costs Demand a aderived from the corresponding values for it in Tables A9.1 and A9.2, namely, x The method could be extended to correlated distributions, but this is much more complicated and it would also require an estimate of the correlation between demand errors and cost errors. A different extension to nonsymmetric probabilities of high and low cases is also possible, but again the formulas will be much more complex. Technical Note: The Derivation of Equation A@9.1 For a symmetrical distribution of outcomes, the values are A M, M, and (2 - A )M, which yields an expected value of M whatever the probabilities. Denoting the probability of M by ir, it follows that the variance across these three outcomes is: (I -i (AmM-M) )[(2-A)M-M]2 = (I-7r)[(l-A)M] 2 Equating this to the externally identified variance of a 2M 2 yields equation A9. 1.

122

123 Annex 10: The Calculation of the Mean and Variance of the Cost of Two Projects Let mi be the expected (mean) cost predicted for project i (i=1,2), and v; (=si 2 ) be the predicted variance of project i. From standard statistical theory, the mean and the variance of the sum of the costs of the two projects is given by m=ml +m 2 v = vl + v,- + 2cov 12 where m is the mean of the total, v is the variance of the total, and cov 12 is the covariance of the distribution of costs for the two projects. Assuming that the distribution of costs are independent, the variance of the sum is equal to the sum of the variances. Hence, the standard deviation of the sum is given by s = 4 (vi + V 2 ) These formulas 3 ' can be applied to the expressions developed in the paper for predicting mean costs and the variances of costs. The regression model predicts the log of the ratio of actual costs to expected cost, where the latter are the values in the World Bank Staff Appraisal Reports (SARs). Denote the predicted values from the regression by P P = log (A/E) Now P is determined from the regression equation in terms of the characteristics of projects, while E is available from the SAR. Hence, to obtain the prediction of the actual value we have: A = E exp (P) This can be justified as the mean (or unbiased) value of A, where P is the mean prediction of the log of the ratio, by the general statistical results that if E(x) = 0 then E [g (x)] = g (e) 31. The formulas can be generalized for more than two projects. Where the projects are implemented sequentially over time under a long-term development program, the costs of the projects should be expressed in terms of their present values in a common base year (usually year 0 of the development program) according to when they would be constructed under the program. 105

124 106 Estimating Construction Costs and Schedules hence 0 = g-' {E [g (x)]) The variance of A also is required and here an approximation is used. Following the above general notation: Var [g (x)] = (dg/dx) 2 Var[x] where the derivative is evaluated at the mean of x (i.e., at 0). Since the variance of P is known from the regression equation, as shown in Annex 8, Appendix A8. 1, and is denoted W: W = Var [log (A/E)] = (1/A) 2 Var A Var A = A 2 W So the standard error of the predicted cost is approximately equal to the predicted cost multiplied by the standard error of regression. For example, the India 911 project had a variance for the log of the ratio of actual to estimated (W) of 0.035, while the predicted actual value (A) was Hence, the variance of the actual value is 8460, with a standard deviation of 92. For two projects with their predicted costs and variances of costs, the overall predicted total cost and variance of the total cost can then be calculated. It can be seen that if (a) project costs are independent; (b) the predicted value of the ratio of actual to expected costs is independent of project size; and (c) the variance of the ratio is independent of project size, then the variance of a sum of two smaller projects is less than that of a single larger project with the same predicted cost as the sum of the predicted costs of the two projects. For example, where the predicted costs of the small projects are each 50 percent of the predicted cost of the one larger project, their mean total cost is the same, while the variance of the large project would be twice that of the sum of the variances of the two smaller projects.

125 Annex 11: Applying the Option Approach to Construction Costs and Schedules This annex illustrates the application of the option approach to investment timing for capital projects that are subject to uncertainty about their construction costs and schedules. 32 It uses the results of the regression analysis in this paper for power generation projects in developing countries to provide the estimates of unbiased values and variances that are needed to apply the option approach to this class of projects. The first two sections of the annex give a general introduction to investment valuation under uncertainty using the option approach and to general solution methods for this approach. The third section develops a simple model for finding the optimal timing for investments, which is then applied in the final section to the general cases of thermal and hydropower projects. These cases provide generic measures of the critical benefit/cost (B/C) ratios and their variances for the optimal timing of investments in these two subclasses of projects. The results confirm the findinas of the main analysis in the paper-that investment risks from construction are greater for hydropower projects than for thermal power projects. Under some plausible assumptions about uncertainty, it is shown that the critical value of the B/C ratio has a premium over the value based on expected (nonbiased values) equal to about 17 percent in the case of thermal power projects, and likewise to 21 percent for hydropower projects. The variance for hydropower projects is also greater than that for thermal power projects. A. Investment Valuation Under Uncertainty Using the Options Approach The so-called real option approach deals with the theory of investments under uncertainty, where uncertainty is treated in a continuous time framework. It examines the impact of irreversibility induced by the existence of fixed capital costs under conditions of uncertainty. Capital costs are viewed as sunk; that is, little, if any, of these costs can be recovered after the investment. The main assumption is that investment can always be deferred. Future investment is viewed as a mutually exclusive alternative to investing today. Even without uncertainty the problem is one of optimal timing, because although the Net Present Value (NPV) of investing today may already be positive, deferring the investment would be preferable if it increased the NPV. Keeping the investment option alive is thus valuable. When the result from the option analysis is not to invest, the investment prospect is not abandoned completely but is deferred to the future. Option models are an improvement on simple NPV models because they capture the economic risk that new information might be revealed by waiting, possibly by investing 32. This annex is based on text and analysis provided by Spiros Martzoukos, to whom the authors are grateful for this contribution. 107

126 108 Estimating Construction Costs and Schedules in further studies. Option models do not consider all kinds of uncertainty, and in that sense they are a partial treatment of risk. They certainly cannot capture directly the risk of changes to the external economic and regulatory conditions, if such conditions affect the economics of the project only after initiation. Option models can capture such risk if relevant information is revealed during the study period. They indirectly capture the ex post uncertainty by requiring a premium before investing. Ideally, such uncertainty should be captured by appropriately discounting the expected project payoffs, regardless of the adoption of the option models. Two premiums are considered by the option model. One is due to deterministic growth trends that capture optimal timing in the deterministic sense. The other is a premium for learning something by waiting under uncertainty. This does not necessarily imply that uncertainty is reduced by waiting, although this could be also the case, for example, in the case of technological uncertainty. It implies mainly that the estimate for the mean changes. The timing (waiting) premium only has zero value when optimal timing has been reached. Investing earlier kills this premium. The optimal time to invest occurs when the value of the underlying asset (the benefits of the investment) exceeds its cost by some predetermined margin. This margin is a function of the discount rates and parameters of the stochastic processes of the underlying variables, such as growth trends (if any) for the underlying investment benefits and costs, and uncertainty. For the discount rates, a continuous time capital asset pricing model is assumed to hold. Option models capture the value of the flexibility to invest in the future if it is profitable to do so but not to invest if conditions worsen. This flexibility is captured in the asymmetry of the distribution of the expected payoff function under the flexibility to invest only if it is profitable to do so. This distribution is unaffected for positive future NPVs but is truncated for negative NPVs, which are replaced by the value of zero. The resulting asymmetry often values the option more highly than investing today, which implies that optimal timing has not been reached yet. An even higher NPV is required to justify a commitment to invest. Uncertainty enhances the effect of irreversibility, and hence the value of flexibility, by increasing the required margin before investment should take place. Thus the option theory revises the conclusion of the classical approach to investment valuation that investment should take place immediately when NPV becomes positive. Such an NPV valuation rule is applicable only when investment is not deferrable. Instead of looking at the NPV, the ratio of benefits over costs is often compared with a critical ratio derived from the model. Only when the actual ratio reaches the critical ratio has optimal timing been achieved. Using a B/C ratio allows option models to be used for two uncertain parameters, namely uncertain benefits and uncertain costs, even though they were originally designed to incorporate only one uncertain parameter. If more than one large investment alternative exist, then the one-alternative analysis underestimates the critical B/C ratio. Assuming two investments of similar scale, the

127 Annex 11: Applying the Option Approach to Construction Costs and Schedules 109 critical ratio for the best option depends on the ratio for the second-best one (Martzoukos 1995). The theory of financial option valuation has been adapted to the economic analysis of capital investments. 3 3 For power projects a least-cost approach is used to select capital investments. The preferred investment is the one that achieves the system goals at the least economic cost. To incorporate this criterion into the option formulation, a donothing or a minimal-investment scenario can be used as the alternative to the proposed large investment, where expansion of supply is achieved (for a while at least) through purchase or import of power or through investments in small and versatile thermal units. The large investment alternative, such as a large hydropower or thermal plant, or a transmission line to connect a load center to a central grid, which involves much higher capital costs but lower operating costs than the alternative, would incur fixed (sunk) costs. The do-nothing alternative has its own operating costs, which are avoided if the large investment is implemented. Hence, in the option model the sunk capital costs are considered to be the costs of exercising the investment option, and the avoided costs of the do-nothing scenario to be the benefits of investing. Both the capital costs on one side and price of power imports or fuel on the other side are effectively cost components but represent different types of uncertainty. Capital costs represent technological or institutional uncertainty, whereas the other costs represent import price or fuel price uncertainty. In a least-cost approach (that is now obviously equivalent to the option B/C analysis), the "benefits" should at least exceed the "costs." The question is, by how much? B. General Solution Methods There are two alternative methodologies to find a solution to a specific investment problem. The first is the contingent claims (option) approach, and the second is the stochastic dynamic optimization approach. Both methods are effectively equivalent to a stochastic optimization of the expected benefits minus the expected costs and differ only in the method of approximation to the same problem specification. The investment option valuation problem is expressed as a fundamental partial differential equation for the option value, which can be solved by numerical methods (see Brennan and Schwartz 1978), or, more rarely (except for simple problem formulations), analytically. The numerical methods can be adjusted to treat options with discrete changes in the benefits and costs if investment is deferred. They can also be adjusted to treat options when the relevant parameters are not constant but are continuous and 33. In the real options literature, important reviews are Pindyck (1991), Dixit (1992), and Dixit and Pindyck (1994). The last is the most updated comprehensive review and is a highly recommended technical reference. A comparison of alternative treatments for uncertainty is given in Crousillat (1989). Applications in the power sector have appeared in Paddock and others (1988), Crousillat and Martzoukos (1991), and Martzoukos and Teplitz-Sembitzky (1992). A recent World Bank application was in an energy options study of Pakistan in July The importance of the deterministic optimal timing is highlighted in Schramm (1989).

128 110 Estimating Construction Costs and Schedules deterministic functions of time (see Martzoukos 1995a). The prolific financial economics literature on option pricing can be used when it offers a model that has the same or similar assumptions to the problem at hand. 34 Another way to solve the same option problem, and of course derive the same result, is the so-called lattice approach. The underlying continuous stochastic process can be approximated by a discrete-time random walk in a decision-tree-like (lattice) fashion. Dynamic programming can then be used to value the initial investment option. The solution methodology starts at the terminal boundary conditions (option maturity, usually a very large number of possible outcomes after many intermediate steps at which options are faced over a long time horizon) by valuing each terminal option at the maximum of its exercise price or zero. Then the analysis proceeds backward in time to derive the initial option value. At every step, the option value is a probability weighted average of the subsequent option values. The process at each step allows for the possibility of exercising an option (at a node in the lattice) by calculating the maximum of the live investment option value or its exercise value (the NPV). 35 This method thus derives the valuation of the investment option and the estimation of the critical B/C ratio through properly discounting the asymmetric expected payoffs of the option. Although this method uses the concept of dynamic programming in a solution methodology that evolves backward, it is not explicitly formulated as a dynamic programming problem. It is rather a hybrid of the option approach to the problem formulation and a backward decision-tree solution methodology. The standard deviation used in the option model (and captured indirectly in the numerical solution for the partial differential equation or directly in the tree-type lattice) effectively replicates a scenario approach. This scenario predicts increasing forecast error in terms of the time frame and is given as a continuous (lognormal) distribution instead of only a few discrete points with some probabilities attached to them. For example, a standard deviation of 10 percent per year, which is equal to a variance of I percent per year, would imply that the variance of the percent change in the underlying variable would be 9 percent around the forecast in 9 years; this equals a standard deviation of 30 percent for that year. The method is an improvement over scenario methods since (a) a continuous distribution captures more information than just a few selected points, (b) it captures the asymmetry of the flexibility to invest in the future only if and when it is optimal to do so, and (c) it inherently discounts properly under uncertainty. Scenario or decision-tree methods cannot estimate the critical B/C ratio nor can they indicate how close the scenario is to the optimal decision. Option theory gives the correct discounting methodology for cash flows under uncertainty. A feature of option models is that they allow different discount rates for 34. The first seminal papers of the financial options literature include Samuelson (1967), Black and Scholes (1973), and Merton (1973). 35. The methodology was demonstrated by Cox and Ross (1976).

129 Annex I1l: Applying the Option Approach to Construction Costs and Schedules 111 benefits and the costs. Furthermore, this discount rate could be a function of time (in models that are solved with numerical techniques). Stochastic dynamic programming is a mathematical methodology that does not deal with the problem of correct discounting but can benefit in this respect from insights gained from the option methodology. Consequently, the two methods can be made strictly equivalent, and they should give the same results. 36 C. A Simple Model for the Investment Option and Optimal Timing In the deterministic sense, optimal timing implies an optimization of the function defined as benefits (B) minus costs (C). When investment is deferred, these components grow continuously at gb and gc. The discount rate (risk-free rate for a risk neutral option) is denoted as R. The terms 8B and c are defined as the difference between the discount rate and the actual expected growth rates for the investment benefits and costs. Both of them are effective discount rates. The first is the opportunity cost of not investing, and the second is the opportunity cost of investing. The deterministic optimization problem solves max[b*exponent(-5bt) - C*exponent(-6ct)] where maximization is in respect to time t. Equating the derivative with respect to time to zero gives the optimal timing: t = log[8cc/(8bb)]/(8b-6c) In the absence of uncertainty, 8C/8B equals the critical B/C ratio. 37 A case for the investment option under uncertainty with a known analytic solution is the McDonald and Siegel (1986) model, which extends Samuelson's (1967) work to the case of investment timing. It can be easily solved with a calculator or a spreadsheet. The critical B/C ratio is given by T/(, - 1) (A 11. 1) where the parameter X equals = (0.5 - (5B - C)/2) + [((6B - 5C)/G 2-0.5)2 + 28C/a 2 ]. The variance used in the option model equals ay 2 = (y 2 B + Cy 2 c - 2 cbycrbc (Al 1.2) 36. The relationship between the option approach and dynamic programming is discussed in Dixit and Pindyck (1994, chapter 4). 37. At optimal time, t* equals zero, so the critical benefit-cost ratio equals 8c/ sa

130 112 Estimating Construction Costs and Schedules and is in fact the variance of the actual B/C ratio given as a function of the individual variances and the correlation between them. The negative sign implies that if benefits and costs are very strongly and positively correlated, the effective variance that affects the option price decreases. The following simple example shows how this model works in the two parameter case with growth rates of the deferred investment benefits and costs both equal to zero, the continuous discount rate equal to 10 percent, and the cost uncertainty and the benefits uncertainty both equal to 10 percent. It is also assumed that the correlation between costs and benefits is 10 percent. If these assumptions about zero growth rates are not proper, then the model that assumes constant model parameters would not hold. In that case it would be necessary to test first whether the deterministic optimal timing has been reached, which can be done by assuming that the investment is deferred for one or two years. Then the economic analysis is repeated in which the benefits and costs are discounted to the present. If the NPV of the deferred investment exceeds the NPV of investing today, then clearly the investment should wait and it would not be necessary to proceed with the option analysis. Otherwise, the analysis follows with the use of the option model. The effective uncertainty cs equals \(.10*.10+.l10* 10-2.*.10*.10)=0.134 and the critical B/C ratio equals As expected the results are sensitive to the values chosen for uncertainty. If one uncertainty equals 10 percent and the other equals 5 percent, then the effective uncertainty equals 10.7 percent and the critical ratio equals If one uncertainty equals 10 percent and the other equals 15 percent, then the effective uncertainty equals 17.2 percent and the critical ratio equals In an example similar to the base case above for when two large investment alternatives of similar scale exist, Martzoukos (1 995b) shows that if the actual ratio of the second-best equals 1.0 (or 1.20 or 1.40), then the critical ratio for the best investment equals 1.30 (or 1.41 or 1.57). D. Application of the Simple Options Model to Construction Costs and Schedules In this section, the options valuation approach to investment timing is applied to the problem of uncertainty in construction costs and schedules, using the generic regression equations for thermal and hydropower generation projects in developing countries that are derived in this paper. The simple model given in section C of this annex is used for this example.

131 Annex 1 1: Applying the Option Approach to Construction Costs and Schedules 113 Dl. General Assumptions The following four assumptions apply to this example: a. Uncertainty is measured around the expected value, so that a correction has been applied for bias to the estimated value. b. The linearized errors of the regressions are used as proxy for the ex ante uncertainty used in the option model, although in fact they represent ex post tuncertainty. They are specific to the time horizon (corrected for bias) for project completion, so in the option models "option variance" = variance*t; this implies that the derived option model uncertainty for the relevant parameter (schedule or cost) is a forecast error that is specific to the time adjusted for bias, and is a forecast error increasing in respect to time. The detailed derivation of option model standard deviations is more complicated and follows in the next section for both the benefits (related to schedule slip) and costs. c. The relationship betwveen normal and lognormal distributions. When a variable is normally distributed as N(g,c), the notation implies a normal distribution with mean,u and standard deviation (. If the logarithm of a variable is distributed N(I.,o), then the variable is said to be lognormally distributed with mean equal to exp(ji+.5c 2 ) and variance equal to exp(2u+cy 2 ){exp(a 2 )-l 1. To derive a yearly option model a, a2 is replaced in the equation above with a 2 t, where t is the best estimate for the time to project completion and is adjusted for bias. d. Definition of linearized error. The logarithm of the ratio of actual cost over predicted (estimated) cost is the dependent regression variable: Ln(C/Cpred) = Cregr + error Thus, by taking the exponent of both the left and right sides of the regression equation: C = exp(lncpred + Cregr) + (linearized error)j(cpred) The first expression on the right side equals the mean of a lognormal distribution and has a standard deviation equal to the linearized error adjusted with 4(Cpred). A similar result holds for schedule slippage. D2. General Formulation to Estimate Option Model Uncertainty This section derives the estimate of the uncertainty for the benefits and costs that can be used in the option model. This estimate is case specific. It depends on the percent correction for bias that is given by the regression model for both schedule and costs, and the actual time predicted.

132 114 Estimating Construction Costs and Schedules a. Approximating schedule uncertainty. Demand uncertainty and fuel price uncertainty are usually considered to be the factors associated with benefit uncertainty. 38 Construction slip uncertainty can augment these two factors, and can even replace them on the assumption that: 1. Project timing given by the deterministic methodology is in general optimal in terms of excess capacity, so that demand uncertainty for a project of given capacity is small; and 2. Fuel price uncertainty is not reduced by waiting due to the long project construction period. By considering schedule slippage uncertainty as the relevant uncertainty for the benefits (or, in any case, as one of the relevant uncertainties for the benefits) in the option model, it is implicitly accepted that waiting will improve the estimate for the expected time to complete the project. Since time to completion can only be a positive number, this complies with the lognormal distribution assumption of the option model. The benefits from the investment option are denoted as B, and they equal exp(-tr)num, where t equals the project completion time, R the continuous discount rate, and Num a number that represents the expected benefits discounted to the end of the project completion period. First, the variance a 2 of B is needed for the option model if the variance of t is known. It is assumed that Num is not sensitive to the estimate of t. In the option model, the entity B is lognormally distributed, thus InB equals -tr + ln(num) and is normally distributed with variance a 2 t. So the variance of t equals a 2 t/r. Because t is the predicted time corrected for bias, the variance equals a 2 tpredexp(tregr)/r. Second, the variance of t is estimated from the regression model. It is known that ln(t/tpred) = tregr, thus "t/tpred" equals exp(tregr), with the standard deviation equal to the linearized error e multiplied by tp,d. So the standard deviation of t equals e'i(tpred). From the above: el(tpmd) from which: = a'l[(tpredexp(tregr)/r] a = e4[r/exp(tregr)] (Al 1.3) (b) Approximating cost uncertainty. Cost uncertainty relates directly to the cost C of exercising the investment option. From the regression model ln(c/cpred) = Cregr, 38. The considerable uncertainty that exists in forecasts for oil prices and power demand is confirmed by the Bank's experience, as shown in Annex 12.

133 Annex I 1: Applying the Option Approach to Construction Costs and Schedules 115 CCpred := exp(cregr) with standard deviation equal to the linearized error e. Equivalently it is assumed that the standard deviation of C equals eacpred- The option model gives the assumption that C is lognormally distributed, so InC has a standard deviation of C 2 t, where t equals the expected time and is adjusted for bias to tpredexp(tregr). The relation between the means of normal and lognormal distribution is used to derive p: C = Cpredexp(Cregr) = exp(q+.5e 2 Cpred) SO.L = InCpred + Cregr -.5e 2 Cpred. Finally, an approximate value of a is derived by solving iteratively the equation for the relationship between the variances of the log-normal and the normal distributions: e 2 Cpred = exp(2g+a 2 t){exp(a 2 t)-l } (Al 1.4) D3. The Cases of Thermal and Hydropower Plants The following linearized errors for the two main groups of power generation projects were derived in chapter 6 of the paper: Thermal Hydro Schedule Cost (constant) The predicted and adjusted schedules are given by ln(t/tpred) = tregr, where t/tpred is the average schedule slip derived from the regression equations (30 percent for thermal, 28 percent for hydro): Thermal Hydro tregr ln(1.30) = ln(1.28) = The predicted and adjusted costs are given by ln(c/cpred) = Cgr, where C/Cp,d is the average cost overrun derived from the regression equations (6 percent for thermal, 27 percent for hydro): Thermal Hydro Cregr ln(l.06) = In(1.27) = Assuming a continuous discount rate R of 10 percent, equation Al 1.3 is solved for the investment option benefits CB: Thermal Hydro ab

134 116 Estimating Construction Costs and Schedules Equation A 11.4 is solved by numerical methods for the investment option cost ac: Thermal Hydro ac Assuming a squared correlation between cost overrun and schedule slip for thermal power projects of.24 and for hydro power projects of.01 (Table 5.6 of the paper), which imply correlations of.4899 and.1, the relevant standard deviations are computed from equation A 11.2 and the critical B/C ratios are computed from equation AI 1.1 (section C), as follows: Thermal Hydro CSB/C (B/C) Thus the critical benefit-cost ratios that account just for uncertainty in construction costs and schedules about expected (unbiased) values (assuming that the deterministic optimal timing is already exceeded) are case specific. In general they are shown to be higher for hydro power plants, mainly because of the lower correlation between costs and schedules for hydropower projects. To recapitulate the main insight from this application of the option approach, if uncertainty changes in the way that is modeled, then the invest now option should be exercised only when the estimated benefit/cost ratio exceeds based on expected (unbiased) values in the general case of thermal power generation projects, and exceeds in a similar fashion for hydropower generation projects.

135 Annex 12: Performance of Power Demand Forecasts and of World Bank Oil Price Projections Figure A12.1 Performance of Power Demand Forecasts for Developing Countries Deviation (percent) 60I Mean-1 STD - ' +Mean deviation Mean +1 STD Year 1 Year 3 Year 7 Year 10 Note: Deviation percent is deviation of forecast demand from actual demand as a proportion of actual demand based on a sample of about 200 power demand forecasts. Source: Sanghvi and Vernstrom (1989). Figure A12.2 World Bank Oil Price Projections in Constant 1987 US$ per Barrel Dollars per barrel 1980 forecast 1982 forecast ' ' - ~~~~~~~ ' ~~1984 / forecast Actual / forecast 1988 forecast Source: Crousillat (1989). 117

136

137 References Arrow, K. J., and R. C. Lind "Uncertainty and the Evaluation of Public Investment Decisions." In Cost Benefit Analysis (2nd edition), ed. R. Layard and S. Glaister. Cambridge: Cambridge University Press. Black, Fischer, and Myron Scholes "The Pricing of Options and Corporate Liabilities." Journal of Political Economy 81: Brennan, Michael J., and Eduardo Schwartz "Finite Difference Methods and Jump Processes Arising in the Pricing of Contingent Claims: A Synthesis." Journal of Financial and Quantitative Analysis 20: "Evaluating Natural Resource Investments." Journal of Business 58: Cox, John, and Stephen A. Ross "The Valuation of Option for Alternative Stochastic Processes." Journal of Financial Economics 3: Crousillat, Enrique "Incorporating Risk and Uncertainty in Power System Planning." Energy Series Paper 17. World Bank, Industry and Energy Department, Washington, D.C. Crousillat, Enrique, and Spiros Martzoukos "Decision Making Under Uncertainty: An Option Valuation Approach to Power Planning." Energy Series Paper 39. World Bank, Industry and Energy Department, Washington, D.C. Dixit, Avinash "Investment and Hysteresis." Journal of Economic Perspectives 6 (Winter): Dixit, Avinash, and Robert Pindyck. Princeton University Press Investment under Uncertainty. Princeton, N.J.: EPRI (Electric Power Research Institute) Option Pricing for Project Evaluation: An Introduction. TR , Project , Final Report, January. Little, I. M. D., and J. A. Mirrlees Project Appraisal and Planning for Developing Countries. London: Heinemann. Martzoukos, Spiros H., and Witold Teplitz-Sembitzky "Optimal Timing of Transmission Line Investments in the Face of Uncertain Demand." Energy Economics 14 (January): Martzoukos, Spiros H. 1995a. "Issues on Irreversibility and Investment using the Contingent Claims (Real Options) Approach." Unpublished Dissertation, Finance Department, The George Washington University, Washington, D.C. 1995b. "Rational Indecision and a Numerical Investigation of the Two-Dimensional Free Boundary Problem." School of Business and Public Management Working Paper 95-16, The George Washington University, Washington, D.C. McDonald, Robert, and Daniel Siegel "The Value of Waiting to Invest," Quarterly Journal of Economics 101:

138 120 Estimating Construction Costs and Schedules Merrow, Edward W., and Ralph F. Shangraw, Jr., with Scott H. Kleinberg, Lisa A. Unterkofler, Richard Madaleno, Edward J. Ziomkoski, and Brett R. Schroeder "Understanding the Costs and Schedules of World Bank Supported Hydroelectric Projects." Energy Series Paper 31. World Bank, Industry and Energy Department, Washington, D.C. Merton, Robert "The Theory of Rational Option Pricing." Bell Journal of Economics and Management Science 4 (Spring): Moore, Edwin A., and George Smith "Capital Expenditures for Electric Power in the Developing Countries in the 1990s." Energy Series Paper 21. World Bank, Industry and Energy Department, Washington, D.C. Paddock, James L., Daniel R. Siegel, and James L. Smith "Option Valuation of Claims on Real Assets: The Case of Offshore Petroleum Leases." Quarterly Journal of Economics 103 (August): Pindyck, Robert "Irreversibility, Uncertainty and Investment." Journal of Economic Literature 29 (September): "Sunk Costs and Benefits in Environmental Policy: I. Basic Theory." MIT Energy Laboratory Working Paper MIT-CEEPR , Cambridge, Mass., March. Samuelson, Paul "Rational Theory of Warrant Pricing." Industrial Management Review 6 (Spring): Sanghvi, Arun, and Robert Vernstrom, with John Besant-Jones "Review and Evaluation of Historic Electricity Forecasting Experience ( )." Energy Series Paper 18. World Bank, Industry and Energy Department, Washington, D.C. Schramm, Gunter "Optimal Timing of Transmission Line Investment." Energy Economics 11 (July). Teplitz-Sembitzky, Witold "Option Pricing in Power Planning." Draft working paper, Industry and Energy Department, The World Bank, Washington, D.C. World Bank Geological Problems and Cost Overruns: A Survey of Bank-Financed Hydroelectric Projects, World Bank Energy Department Note 61, July. Operations Evaluation Department Annual Review of Evaluation Results Washington, D.C.

139 DiStributors Distributors of ~~~~~~CHINA GREECE ITALY of NORWAY China Financial & Econom,c Papasotinou S.A Lcosa Commissonana Sansoin SPA Narvesen Information Center W orld Bank Publishing House 35. Soumarna Sir Via Duca Di Calabna, 8. Da Fo SC Dong Jie Athens Casella Postale 552 PO Box 6125 EttersLad Asahgate Publications Beijing Tel (1) Firenze N-0602 Oslo 6 Pacific SINGAPORE, TAIWAN, Van Dienmen Edntons Techniq 1/1 Book Department MYANMAR, BRUNEI Ch. de Lacuez 41 Publishing Asia CH1807 Blonay Pe Ltd Tel (021) Prices and credit terns c1arti Tel (11) Fax (1) Te. (55) Tel (22) Kallang Pudding Road Fax. (021) front country to cozntry. Fan (1) Fax (55) Fax (22) Golden Wheel Building from country to country. HONG KONG, MACAO Singapore TANZANIA Consult yo;r local distributor COLOMBIA Asia 2000 Ltd JAMAICA PAKISTAN Tel (65) OxCord University Press before placing an order Intoenlace LUda. Sales & Circulation Department tan Randle Pubiishers Ltd. Mirza Book Agency Fax (65) Maktaba Street Apartado Aereo Seabird House unal Old Hope Road 65. Shahrah-e-Quaid-e-Azarn ashgate@asianconnen.com PO Box 5299 Bogota D E Wyndham Street, Central Kiopston 6 PO Box No. 729 Dar es Salaam ALBANIA Tel. (1) Hong Kong Tel Lahore SLOVAK REPUBLIC Tel (51)29200 Adnon Ltd Faxa (1) Tel Fax Tel (42) Slovan GT.G. Ltd. Fax(51)46622 Pedat Rexhep Str Fax Fax. (42) Krupinska 4 Pall 9, Shk. 1Ap 4 COTE D'IVOIRE URL: http /Aw.salesOasa2000.com hk JAPAN PO Box 152 THAILAND Tirana Centre dedinion el de Diftusion Eastem Book Servce Oxford Uivenrsiry Press Bratislava 5 Central Books Distribution Tel (42) 27419; Alricaines (CEDA) HUNGARY Hongo 3-Chome. 5 Bangalore Town Tel. (7) Silom Road Fax. (42) B.P 541 Foundation for Market Bunkyo-ku 113 Sharae Faisal Fax. (7) Bangkok Abidjan 04 Plaleau Economy Tokyo PO Box Tel. ( ARGENTINA Tel Dombovari Ut Tel. (03) Karachl SOUTH AFRICA, BOTSWANA Fax (2) Oficina del Libro lntremacionat Fax H-1117 Budapest Fax- (03) Tet. (21) Forsngle lt7es: Av. Cordoba 1877 Tel or URL httpijwww)bekkoame or.ip-svl-ebs Fax (21) Oxford University Press TRINIDAD & TOBAGO, JAM Buenos Anres CYPRUS oup@oup khi enum.com.pk Southem Alnca Systemalics Studies Unit Tel' (1) Center of Applied Research Fax: KENYA PO Box 1141 #9 Wafts Sreet Fax. (1) Cypnus College Africa Book Service (E.A( Ld. PERU Cape Town 60B0 Curepe T 6, Diogenes Street, Engomi INDIA Ouaran House, Mfangano Street Ednonal Desarrollo SA el. (21) Trinidad, West Indies AUSTRALIA, FIJI, PAPUA NEW GUINEA, PO Box 2006 Allied Publishers LId. PO Box Apartado 3824 Fax (21) Tel SOLOMON ISLANDS, VANUATU, AND Nicosa 751 Mount Road Nairobi Lima 1 Fax WESTERN SAMOA Tel Madras Tel. (2)23641 Tel (14) Forsubcnptironao@ers DA. Infontoabon Services Fax Tel (44) Fax (2) Fax (14) International Subscnption Service UGANDA 648 Whnehrxse Road Fax: (44) PO. Box Gusfro Ltd. Mncham 3132 CZECH REPUBLIC KOREA, REPUBLIC OF PHILIPPINES CraighaWI Madhvani Building Vrctona National Inlormation Center INDONESIA Da0qon Trading Co. Lid lntemational Booksounce Center Inc Johannesburg 2024 PO Box 9997 Tel. (61) prodelna. Konviktska 5 Pt. Indira Limited P.O. Box 34 Sure 720, Catyand 10 Tel' (11) Plot 16/4 Jinja Rd Fax. (61) CS Prague I Jalan Borobudur 20 Yeoeida Condominium Tower 2 Fax: (11) Kampala URL hntpij/ww.dadirect.com au Tel (2) PO. Box 181 Seoul H.V dela Costa, comer TeVFax: (41) Fax. (2) Jakarta Tet: (2) /4 Vatero St. SPAIN AUSTRIA URL' httpijwww.nis.ci Tel. (21() Fax. (2) Makati, Meto Manila Mundi-Prensa Libos, S.A UNITED KINGDOM Gerold and Co. Fax. (21) Tel: (2) Castello 37 Microafo Ltd Graben 31 DENMARK MALAYSIA Fax (2) Madnd PO. Box 3 A-1011 Wien SamfundsaLtteratur IRAN University of Malaya Cooperahve Tel- (1) Anon, Hampshire GU34 2PG Tel' (1) RosenoemsAlld 11 Kowkab Publishers Bookshop LiUmied POLAND Fax (1) England Fax (1) DK-1970 Fredenksberg C PO Box PO. Box 1127 International Publishing Service tsai.es/mprensa Tel (1420) 6848 Tel. (31) Tehran Jalan Panlai Baou Ul Piekna Fax: (1420)89889 BANGLADESH Fax: (31) Tel (21) Kuala Lumpur Warzawa T Mundb-Prensa Barcelona Mico Iidusines Developmenl Fax 98 (21) el: (3) Tel (2) Consell de Cent, 391 ZAMBIA Assislance Swcety (MIDAS) EGYPT, ARAB REPUBUC OF Fax (3) Fax: (2) Barcelona University Bookshop House 5, Road 16 At Ahrmm Ketab Sara Co. Pubhshers Tel: (3) Great East Road Campus Dhanmordi FlArea A/GaaalStreet Khaled Eslambok Ave., MEXICO PORTUGAL Fax- (3) PO. Box Dhaka 1209 Cairo 6th Shreet INFOTEC Livrana Ponugal Lusaka Tel. (2) Tel (2) Kusheh Delafmoz No 8 Aparlado Postal Rua Do Carmo SRI LANKA, THE MALDIVES Tel. (1 I EAt. 482 Fax: (2)81118B Fax. (2) Tehran Tialpan Lisbon Lake House Bookshop Tel or Mexico D.F Tel: (1) P.O Box 244 ZIMBABWE BELGIUM The Middle East Obserer Fax Tel (5) Fax: (1) , Sir Chittampalam A Lorngman Zimbabwe (Pte.)ULd Jean De Lannoy 41, Shen Stre ketab-aora@neda.set.ir Fax: (5) Gardiner Mawatha Toude Road, Ardbonnie Av. duro 202 Cairo ROMANIA Colombo 2 PO. BoxoST Bnrsels Tel (2) IRELAND NETHERLANDS Compans De Lbrani Bucuesti S.A. Tel: (1) Southenon Tel: (2) Fax. (2) Govemment Supplies Agency De LindeboonVtnOr-Publikaties Sir Lipsaoni no 26, ctor 3 Fax (1) Harare Fax: (2) Odig an tsolathair P.O. Box 202 Bucharest Tel' (4) FINLAND 4-S HabnourI Road 7480 AE Haakslbenoe Tel. (1) SWEDEN Fax. (4) BRAZIL Akateemiren Kuqakauppa DClin 2 Tel. (53) Fax. (1) Frites Customer Servce PublcacbesTmcncas lntemaoonais PO. Box 23 Tel. (1) Fax: (53( Regenngsgpon 12 Ltda. FIN Helsanbl Fax: (1) RUSSIAN FEDERATION S Stockhotm Rua Pa/xono Gomide, 209 Tel. (0(12141 NEW ZEALAND lsdalelsoto Ves Mir> Tel' (86( Sao Palo, SP Fax: (0) ISRAEL EBSCO NZ Ltd. 9a, Kelpachory Pereulok Fax: (8) Tel: (11) URL: IttIpJlboAoknetculbe.ttleka/ Yoznot Literalune Ud. Private Mai Bag Moscow Fa: (11) P.O. Box New Market Tel (95) Wernnergren-Williams AB FRANCE Tel Avrv Aucklarid Fax (95( P O. Box 1305 CANADA World Bank Pubicadions Tel. (3) Tel: (9) S Solna Renoul Pubittag Co. LUd 66, avenue d'lana Fax: (3( Fax: (9) SAUDI ARABIA, fatar Tel: (8) Algoma Road Paris Jean Book Store Fax: (8) 27-0W71 Ottawa, Ontw K1B 3Y8 Tel (1) vS7 ROY.O ntkrmabonal NIGERIA PO. Box 3196 Tel: =3 Fax: (1) POBox13056 UnversmityPressLnUmed Riyadh11471 SWTrZERLAND Fax: TelAovt Three Crnwnis Building Jicho Tel: (1( Lbaine Payno GERMANY Tel: (3) Prvate Mail Bag 5095 Fax: (1) Senrvce Institobonnel UNO-Verblag Faa: (3) Ibadan C6tes-d-Montbenon 30 Poppedorter A/lee 55 Tel: (22) Lausanrse Bon Pasestbme anaut1ontylmtidle East Fax (22) Tel: (021) Tel' (228) Index Informabon Semices Fax: ( Fax: ( PO.B Jensalem

140

141 RECENT WORLD BANK TECHNICAL PAPERS (continued) No. 288 Biggs, Shaw, and Srivastiva, Technological Capabilities and Learning in African Enterprises No. 289 Dinar, Seidl, Olem, Jorden, Duda, and Johnson, Restoring and Protecting the World's Lakes and Reservoirs No. 290 Weijenberg, Dagg, Kampen Kalunda, Mailu, Ketema, Navarro, and Abdi Noor, Strengthening National Agricultual Research Systems in Eastern and Central Africa: A Framework-for Action No. 291 Valdes and Schaeffer in collaboration with Errazuriz and Francisco, Survleillanice of Agricultural Price and Trade Policies: A Handbookfor Chile No. 292 Gorriz, Subramanian, and Simas, Irrigation Management Transfer in Mexico: Process and Progress No. 293 Preker and Feachem, Market Mechanisms and the Health Sector in Central and Eastern Europe No. 294 Valdes and Schaeffer in collaboration with Sturzenegger and Bebczuk, Surveillance of Agricultural Price and Trade Policies: A Handbookfor Argentina No. 295 Pohl, Jedrzejczak, and Anderson, Creating Capital Markets in Central and Eastern Europe No. 296 Stassen, Small-Scale Biomass Gasifiersfor Heat and Polwer: A Global Reviezw No. 297 Bulatao, Key Indicatorsfor Family Planning Projects No. 298 Odaga and Heneveld, Girls and Schools in Sub-Saharan Africa: From Analysis to Action No. 299 Tamale, Jones, and Pswarayi-Riddihough, Technologies Related to Participatory Forestry in Tropical and Subtropical Countries No. 300 Oram and de Haan, Teclhnologiesfor Rainifed Agriculture in Mediterranean Climates: A Review of World Bank Experiences No. 301 Edited by Mohan, Bibliography of Publications: Technical Department, Africa Region, JulyI 1987 to April 1995 No. 302 Baldry, Calamari, and Yameogo, Environmental Impact Assessment of Settlement and Development in the Upper Leraba Basin No. 303 Heneveld and Craig, Schools Count: World Bank Project Designs and the Quality of Primary Education in Sub-Saharan Africa No. 304 Foley, Phiotovoltaic Applications in Rural Areas of the Developing World No. 305 Johnson, Education and Training of Accountants in Sub-Saharan Anglophone Africa No. 306 Muir and Saba, Improving State Enterprise Performance: The Role of Internal and External Incentives No. 307 Narayan, Toward Participatory Research No. 308 Adamson and others, Energy Use, Air Pollution, and Environmental Policy in Krakow: Can Economic Incentives Really Help? No. 309 The World Bank/FOA/UNIDO/Industry Fertilizer Working Group, World and Regional Supply and Demand Balances for Nitrogen, Phosphate, and Potash, 1993/ /2000 No. 310 Edited by Elder and Cooley, Sustainable Settlement and Development of the Onchocerciasis Control Programme Area: Proceedings of a Ministerial Meeting No. 311 Webster, Riopelle and Chidzero, World Bank Lendinigfor Srnall Enterprises No. 312 Benoit, Project Finance at the World Bank: An Overview of Policies and Instruments No. 313 Kapur, Airport Infrastructure: The Emerging Role of the Private Sector No. 314 Valdes, Schaefferin collaboration with Ramos, Surveillance of Agricultural Price and Trade Policies: A Handbookfor Ecuador No. 316 Schware and Kimberley, Information Technology and National Trade Facilitation: Making thte Most of Global Trade No. 317 Schware and Kimberley, Information Technology and National Trade Facilitation: Guide to Best Practice No. 318 Taylor, Boukambou, Dahniya, Ouayogode, Ayling, Abdi Noor, and Toure, Strengthening National AgricuIltural Research Systems in the Humid and Sub-humid Zones of West and Central Africa: A Frameworkfor Action No. 320 Srivastava, Lambert and Vietmeyer, Medicinal Plants: AnI Expanding Role in Development No. 321 Srivastava, Smith, and Forno, Biodiversity and Agriculture: Implicationsfor Conservation and Development No. 322 Charles M. Peters, The Ecology and Management of Non-Timber Forest Resources No. 323 Edited by Dominique Pannier. Corporate Governance of Public Enterprises in Transitional Economies No. 324 Cabraal, Cosgrove-Davies, and Schaeffer. Best Practicesfor Photovoltaic Household Electrification Programs

142 ~~~~~~~~~~~~~~~~~~I J.i I~~~~~~~~~~~~~~~~~~~~~~~