Measuring ROI in E-Government

Transcription

1 Measuring ROI in E-Government

2 Table of Contents Introduction 3 The Need For Measurement ROI and the Public Sector 3 Why Some Agencies Fail to Measure 3 Why ROI Valuation is Critical to E-Government Success 4 How Valuation Studies Assist Agency Management 4 Valuation Methodologies for E-Government The Crucial Role of Benchmarking 5 Measurement Categories Costs and Benefits 5 Creating a Valuation Model 5 Valuation Rationales for E-Government Initiatives 6 Valuation Models for E-Government Initiatives 6 Methods of ROI Valuation 9 Summary Confirming ROI Critical to E-Government Growth 10 Notes 11 About the Authors 12 About the Research 12 2

3 Introduction In less than a decade, the overall value of the United States Internet economy has grown from virtually nothing to almost $200 billion. 1 The public sector is no exception to this explosive e-growth phenomenon. The Gartner Group predicts that government spending on information technology initiatives will increase by 4.4 percent annually to $101.1 billion by With information technology initiatives consuming a significant portion of government budgets, managers in the public sector are looking for the skills needed to develop a comprehensive return on investment (ROI) valuation framework, which will in turn, ensure the success of their initiatives. In March of 2001, Anexsys, LLC, a Bank One majority-owned limited liability company, partnered with Indiana University-Bloomington s Institute for Development Strategies and the e-government analysts, META Group Inc., to review the issue of evaluating return on investment in government technology projects. The objectives of this project were to: demonstrate a need to confirm return on investment in e-government initiatives, explore how specific methodologies are employed to determine e-government ROI and outline a recommended course of action for agencies implementing ROI that can be used by any agency. Most of the information cited here is the result of this team s research. The Need For Measurement ROI and the Public Sector Many government agencies use some form of ROI valuation, e.g., lower transaction cost, earlier deposits into treasury or facility reductions, to determine e-government project spending and approval. Yet, traditional ROI models fail to encompass the broader requirements of government. Unlike the private sector, government agencies must serve every constituent and cannot tailor their services to a specific market niche that is more profitable. Additionally, governmental agencies are often required to work within specific regulations and legislation that eliminate more cost effective or efficient approaches. For example, a department of natural resources might wish to implement a voice response telephone system to handle hunter registrations and accept payment using a credit or debit card. If existing legislation prohibits the use of credit cards, then legislative action will be needed before the service can be implemented. Finally, not every policy goal is built around a tangible financial benefit for an agency or its constituents. Quite often, agencies are charged with providing solutions that simply provide a convenience for constituents or reduce a specific constituent burden, such as waiting in line. Why Some Agencies Fail to Measure Most government organizations fail to measure the success of their information technology (IT) investments in general.ii In their book Powering Up, Katherine Barrett and Richard Greene describe several key issues government leaders say prevent them from conducting ROI studies: iii Limited resources Benefits over status quo appear obvious Politically powerful decision makers are able to generate funding without justifying economic/financial feasibility Performance measurement systems are not in place or data from the system is not used to make a systematic evaluation 3

4 Why ROI Valuation is Critical to E-Government Success The Cost of Failure is High E-government capital projects are not often built. When they are constructed, they are usually very complex endeavors that are required to go through a laborious series of capital planning and budgeting processes. These processes take considerable planning and involve substantial amounts of time, money and energy. Once construction has begun, physical infrastructure projects are difficult (and expensive) to halt or reverse. In addition, the results of e-government initiatives commonly have a shelf life of dozens of years. By evaluating ROI early in an e-government initiative, agencies have the opportunity to prevent costly and embarrassing errors. Agencies demonstrating a ROI track record for their initiatives will be in a superior position to win additional funding or funding for future projects. Any Measure is Better Than No Measure According to Gilbs Law iv, any kind of measurement is better than no measurement at all. Thus, creating even the most basic valuation when hard figures are unavailable is of value and greatly preferred to no data. How Valuation Studies Assist Agency Management An ROI valuation is critical to e-government success. A comprehensive valuation framework can assist agencies in the following ways: Provide close integration of information technology to business objectives. Because IT is such a critical component of most initiatives, it is imperative that IT efforts be aligned with the agency s policy objectives. An effective ROI study helps agency officials guarantee this integration of technology and mission. Quantify value of non-financial benefits of e-government. Using a variety of techniques, it is possible to arrive at specific values of the intangible benefits of e-government, such as convenience and time savings. For instance, one can calculate the actual value of the time savings to a constituent who is able to renew his or her driver s license via the Internet or telephone. Demonstrate both cost savings and cost avoidance. In the example above, we can measure tangible financial benefits in two ways. First, we can calculate savings from immediate efficiencies (reduction in paperwork, reduced payment float, etc.). In addition, we can measure cost avoidance, such as decreased storage and supply charges. Anticipate legislative requirements. It is likely that government agencies will be increasingly required to conduct ongoing ROI studies. Agencies that fail to measure ROI of e-government initiatives may face difficulties in receiving both initial as well as ongoing funding. Successfully compete for funding. Agencies that enter funding battles armed with ROI data to support their proposed initiatives will have a substantial advantage competing for scarce dollars. Manage risk more effectively. The stakes involved in any e-government initiative are high; failures are truly disasters than can irreparably damage public trust and endanger future funding. By thoroughly measuring ROI before undertaking an e-government project, an agency will have a clear understanding of both costs and timelines. In the event that an initiative requires revision, managers armed with cost and time 4

5 information will be equipped to make educated choices, minimizing risk while continuing to ensure the success of the initiative. Valuation Methodologies for E-Government The Crucial Role of Benchmarking It is impossible to accurately measure the effectiveness of an e-government initiative without first examining the existing business process. Agencies must take care to map out current workflows in great detail and then determine which steps could be omitted with an electronic system. From there, managers can determine how those eliminated steps could be quantified and translated into reduced costs and shortened time frames. Measurement Categories Costs and Benefits Implementation of an ROI framework requires identification and measurement of specific categories of costs and benefits. While the difficulty and accuracy involved in measurement varies considerably across the different types of costs and benefits, their inclusion is essential for a complete and accurate ROI assessment. v Of course there are the obvious costs that must be measured such as: fixed costs, hardware acquisition, software, database development, telecom equipment, hiring and training of new personnel, etc. Then there are the costs that go beyond the traditional IT cost model, such as: building, maintaining and managing relationships. Expected benefits, too, must be measured in advance of beginning the valuation model. Evaluate the proposed initiative and develop ways to evaluate benefits such as: improved ability to data share, improved customer service, better communication, more accessibility to quality information, better resource control, enhanced decision-making, continuous process improvement, streamlined networks, reduced paper and transaction costs and reduction of errors. Creating a Valuation Model Most e-government initiatives are established to reflect the policy direction of the agency. Other common initiative objectives are: increasing the competitive advantage, creating greater citizen satisfaction and reduction of cost by providing online services. Carol Kelly, of META Group, Inc., has suggested that the following steps be taken to create a valuation model : vi Identify a valuation model for each major stakeholder, including citizens, business-tobusiness government and internal employees. Determine components based on public policy makers articulated agenda, such as: decentralization, flexibility and personalization of services. Develop explicit analysis criteria for each scorecard component (see Why Some Agencies Fail to Measure). Determine measurement points for data collection, such as lower transaction costs and earlier deposits into the treasury. Map the existing regulations/approval processes. Determine if revisions to the financial dimension are needed. 5

6 Develop methods to quantify intangible benefits. Ensure that the overall valuation model measures stated policy requirements. Valuation Rationales for E-Government Initiatives As we have discussed, capital investment decisions need to be made with great care. A variety of sophisticated project evaluation techniques can be used to systematically evaluate the return on capital investments. One can view the ROI framework as a generic way of analyzing any capital investment. Within the ROI framework, there are a variety of tools. The most appropriate tool for a given situation depends on the nature and type of project being undertaken and the resources on hand. Valuation Models For E-Government Initiatives vii Net Present Value (NPV) Model If the cash flow returns and cost streams from a project are private, than a basic NPV analysis is appropriate. The NPV is the present value of the net cash flows minus the net investment. Each cash flow is estimated incrementally (on a monthly or yearly basis, for example) and is discounted to its present value. Rationales for Selecting a Specific Method Determine whether the benefits and costs are private or social An initiative that directly benefits constituents such as a time saving or ease of use is considered social, while one that directly benefits the jurisdiction, such as efficiency and/or lowered transactions costs is private. Determine whether the benefits and costs are tangible or intangible Automated data entry is a tangible benefit; a constituents convenience is an intangible benefit. Determine whether the intangible benefits can be reasonably and agreeably quantified Measurement of intangible, or soft, benefits is usually quite subjective. Depending on the nature of the benefit, the creation of a financial valuation may be challenging. FORMULA Net Present Value = Present Value of Net Cash Flows Net Investment The dollar value of the net investment is subtracted from the sum of the present value of the cash flows, and an NPV dollar values is derived. The NPV dollar figure indicates precisely the increase in the value created from successfully completing the investment. The basic decision rule for independent projects (projects that are not dependent on each other) evaluated for NPV is that decision makers should accept all projects with an NPV greater than $0. For mutually exclusive projects (projects where only one project will be selected), decision makers should accept the project with the largest NPV, assuming it s greater than $0. NPV is the best, more theoretically accurate investment evaluator when the cash flows are private and tangible. For e-government projects, NPV is especially appropriate for internal management system investments that are designed to result in specific, tangible, internal productivity improvements. The NPV framework seems particularly applicable for e-procurement projects, such as Internet exchanges, and provides a useful starting point to begin evaluating intangible benefits. 6

7 Benefit Cost Analysis (BCA)Model Many capital projects involve benefit and cost streams that are social and intangible. For such projects, BCA is the recommended model. FORMULA Benefit Cost Analysis = Net Total Benefit/Net Total Cost There are two critical aspects of understanding and performing social BCA on e-government projects. First, any such project evaluation should involve an understanding of the consumer surplus generated by the project. Consumer surplus occurs when constituents place a higher value on e-government benefits than it costs the jurisdiction to provide. The second critical factor to consider is the social benefits of the project. Social benefits are the benefits enjoyed by individuals who did not pay for the good or service but who did receive value from its provision. These are also known as third-party benefits. BCA valuation is of particular benefit in evaluating large-scale infrastructure projects that have high fixed costs, such as installing new fiber-optic lines or integrating new databases with legacy systems. Without a definite understanding of consumer surplus (which may be zero for new technologies prior to adoption), the benefits of large-scale, new technology projects will be undervalued by decision makers and not undertaken. Valuation Models Net Present Value (NPV) Benefit Cost Analysis (BCA) Cost Effectiveness Analysis (CEA) Portfolio Analysis (PA) Another valuable aspect of social BCA is that it allows you to place a monetary value on intangible concepts, such as the value of time, life and natural resources. One very important aspect of e-government involves the savings in time to citizens and businesses in conducting business with the government. Such a savings is difficult to calculate, but is nevertheless an important potential benefit of e-government. Another potential intangible benefit is simply making the general citizenry more comfortable with using computers and the Internet in managing their everyday affairs, thereby reducing the digital divide between people who are computer literate and people who are not. Social BCA also allows decision makers to include a variety of other potentially important factors, which makes it more flexible than the standard NPV analysis. For example, one might choose to create and use a social discount rate rather than a private discount rate to discount the cash flows. The discount rate, private or social, should be based on the opportunity cost of capital. The social discount rate can include more policy-related factors than a standard risk premium incorporates, such as reducing the digital divide in a rural area, which might lower the discount rate and make the project look more attractive. Social BCA also enables the decision-maker to explicitly include distributional considerations into the analysis, such as the project s differential impact on citizens of different income levels or different industries in the region. 7

8 Cost Effectiveness Analysis (CEA) Model For many e-government projects, a full-blown social BCA may be unnecessary, expensive and time-consuming, especially for small-scale pilot projects where intangible benefits and costs are impossible to agree upon, or where there is an overriding political, strategic imperative to do the project. FORMULA Total Cost Effectiveness = Total Cost/Total Output CEA provides a means of analyzing the benefit return in relation to the capital investment. CEA is especially useful in comparing the effectiveness of alternative levels of capital expenditures. For example, a comparison of expected constituent utilization rates from different application packages with different price tags, or the expected likelihood of scalability from different pilot projects. CEA is useful for analyzing the incremental benefits of specific goals or objectives in relation to marginal costs. Therefore, policy makers can clearly see how much they have to spend for a desired result, and they can choose the tradeoff point. Portfolio Analysis (PA): Managing Risk Among Multiple Initiatives Thus far, we have discussed evaluating individual initiatives. However, most e-government strategies involve the ongoing planning and implementation of multiple e-government projects. For such an orientation, a Portfolio Analysis (PA) approach to managing investments is useful. viii Portfolio analysis quantifies aggregate risk relative to expected returns of an entire portfolio of initiatives. FORMULA Total Aggregate Risk = Return of Investment/Risk of Investment The PA approach enables decision makers to look at their entire portfolio of ongoing projects and evaluate individual projects in terms of their impact on the entire portfolio of investment projects. PA is particularly useful for analyzing the aggregate financial risks associated with e-government investments. For individual projects using any present-value analysis method, such as NPV and BCA, the financial risk factors are already factored in using their respective interest rates risk premium, which reflects a higher degree of risk. Projects with greater risk have greater risk premiums and higher discount rates. These do not capture the aggregate risk of the entire e-government investment portfolio. This may be especially important for risk-averse decision makers. They simply may not want to take on many high-stakes, high-risk projects at the same time, if at all. The PA approach provides managers with a vehicle for understanding and conveying to decision makers the risk of their entire e-government investment portfolio. Moreover, the PA approach enables the measurement of the correlation between individual investments. This is a useful diversification tool for reducing the overall risk of the entire portfolio, without necessarily sacrificing the overall impact of the capital investment. In addition, the PA approach adds an important quantitative and strategic dimension to e-government ROI decision making. This provides an important bridge that links the rigorous evaluation of specific projects to an overall strategic plan that directly maps e-government projects and measures to the organization s strategic mission, goals and objectives. The overall impact across the organization from implementing an e-government strategy may be tracked through a balanced scorecard approach. 8

9 Methods of ROI Valuation NET PRESENT VALUE (NPV) BENEFIT COST ANALYSIS (BCA) COST EFFECTIVENESS ANALYSIS (CEA) PORTFOLIO ANALYSIS (PA) WHAT IT DOES Measures dollar return on projects Calculates project costs relative to benefits (tangible and intangible) affecting diverse groups. Quantifies tangible and intangible benefits for specific, standardized populations. Quantifies aggregate risk relative to expected returns of an entire portfolio of initiatives. FORMULA Present Value of Net CashFlow=Net Investment Net Total = Benefit/Net Total Cost Total Cost Effectiveness = Total Cost/Total Output Total Aggregate Risk = Return of Investment/Risk of Investment WHEN TO USE When cash flows are private and benefits tangible When cash flows are social and benefits are both tangible and intangible When cash flows are private and benefits are both tangible and intangible. When valuating an agency s total risk. Cash flows can be social or private and benefits can be tangible or intangible, but both must be consistent across the portfolio. BASIC DECISION RULE For independent projects with NPV greater than $0, the project should be accepted If ratio is greater than 1, there is a positive return on investment and the project should be accepted. If ratio is less than 1, there is a negative return. If ratio is greater than 1, there is a positive return on investment and the project should be accepted. If ratio is less than 1, there is a negative return. If ratio is greater than 1, there is more return relative to risk and the portfolio is within tolerable risk levels. If ratio is less than 1, it means risk is higher relative to return, and the portfolio exceeds tolerable risk. ADVANTAGES Straightforward Flexible. Allows additional intangible benefits to be included Useful for analyzing incremental benefits, in terms of achieving specific goals in relation to marginal costs. Good for risk-adverse decision makers who may not want to attempt multiple highstakes, high-risk projects simultaneously. DISADVANTAGES Limited to internal capital investments; does not allow for intangible benefits Can be expensive, time consuming. Difficult to agree on values of intangible costs. Technique not directly related to the outcome; measurement is limited to the costeffectiveness, not outcomes. Complex, but could possibly be the most accurate in terms of strategic management. 9

10 Summary Confirming ROI is Critical to E-Government Growth The advent of e-government has been a great benefit to constituents and agencies alike, fundamentally changing the way citizens interact with their government. Through these initiatives, the government empowers its citizens with quick and convenient access to a wide array of services, while greatly reducing costs. As e-government reaches maturity, the most successful projects will consistently be produced by agencies that have mastered ROI valuations. Confirming ROI in major initiatives will be a critical factor in future e-government growth. Using valuation studies, agencies will be able to: Better compete for funding of present and future projects within shifting political climates, policies and agendas Correlate intangible returns with tangible effects for lawmakers and constituents Minimize risks when undertaking e-government initiatives Ensure that policy goals are met by the project plan 10

11 Notes i. Dr. David Audretsch, Dr. Jon P. Gant and Dr. Craig L. Johnson. A Return on Investment Framework for Evaluating E-Government, p. 3 (Bloomington: Indiana University School of Public and Environmental Affairs, 2001). ii. Government Performance Project iii. Barrett, Katherine and Greene, Richard, Powering Up: How Public Managers Can Take Control of Information Technology (Washington, D.C.: CQ Press, 2001). iv. Federal CIO Council, ROI and the Value Puzzle, p.21 (Washington D.C. Capital Planning and IT Investment Committee, 1999). v. Dr. David Audretsch, Dr. Jon P. Gant and Dr. Craig L. Johnson. A Return on Investment Framework for Evaluating E-Government, p. 3 (Bloomington: Indiana University School of Public and Environmental Affairs, 2001). vi. Kelly, Carol, Measuring E-Government Effectiveness. File: 40 (Stamford, CT: META Group Inc., April 2001). vii. Dr. David Audretsch, Dr. Jon P. Gant and Dr. Craig L. Johnson. A Return on Investment Framework for Evaluating E-Government, p. 3 (Bloomington: Indiana University School of Public and Environmental Affairs, 2001). viii. Federal CIO Council, ROI and the Value Puzzle, p.21 (Washington D.C. Capital Planning and IT Investment Committee, 1999). 11

12 About the Authors Anexsys, LLC Anexsys is a leading provider of e-government solutions. Established in 1995 as a joint venture between Bank One and US Bancorp, the company was created to build and operate the nation s largest electronic tax payment system. Today, Anexsys is reshaping the public sector by helping federal and state government provide more effective and efficient service to their constituents. The company has a proven track record of designing, building and operating integrated Internet applications, voice-response systems, call centers and other large-scale information, payment and customer-care solutions. Clients using the company s solutions include the states of Ohio, Kentucky and Wisconsin, as well as federal agencies such as the Departments of Treasury, Health and Human Services, Education and Army and Air Force Exchange Service. Bank One, NA Headquartered in Chicago, Bank One Corporation is the nation's sixth-largest bank holding company, with assets of more than $275 billion. Bank One currently serves more than 50-million credit card customers, 6.9-million retail households, 489,000 small businesses and 20,000 middle market companies. It manages $162 billion of clients' investment assets. About the Research Indiana University Indiana University Bloomington-Institute for Development Strategies The Institute for Development Strategies, sponsored by the Office of Research and the University Graduate School and the School of Public and Environmental Affairs, is a university-wide research program linking faculty resources from various departments and campuses and integrating them with external projects and research being undertaken elsewhere in this country and in other countries. The Institute is inter-disciplinary and focuses on research to promote economic development at the city, regional, national and international levels. A particular focus of the Institute is on the links between globalization, entrepreneurship, and the strategic management of regions to promote economic development. META Group, Inc. META Group is a leading research and consulting firm, focusing on information technology and business transformation strategies. Delivering objective, consistent, and actionable guidance, META Group enables organizations to innovate more rapidly and effectively. Their unique collaborative models help clients succeed by building speed, agility and value into their IT and business systems and processes. 12