Measuring Impacts: Building Support for Economic Development. Kenneth E. Poole, Ph.D. President/CEO

Transcription

1 Measuring Impacts: Building Support for Economic Development Kenneth E. Poole, Ph.D. President/CEO 2010 Best Practices Conference MAEDC May 6, 2010

2 About the Council for Community and Economic Research National membership group, created 1961 Data products ACCRA Cost of Living Index Business Finance and Incentives Database Economic Development Organizations Survey Training resource Regional economic analysis research methods training LMI Training Institute Technical assistance Statewide survey of green jobs Local community & economic research support for projects Advocacy for local community and economic data 2

3 Overview Why Measure? Why Evaluate? Ways to Measure Performance and the Logic Model Case Examples Tools for Measuring Impact A Few Key Issues & Some Words of Advice 3

4 Evaluate, Benchmark, or Project: What s the Difference and Why Does It Matter? THREE KEY APPROACHES 4

5 Why Bother? Key Thesis: Performance builds credibility for and confidence in your economic development efforts Options for building credibility Money Friends Data Our focus today 5

6 What is Performance? How has the economy or the fiscal situation changed over time? How well did you achieve public goals with your economic development investments? How can you improve your economic development programs? What can you predict about future impacts resulting from your investment? How does your community compare with others? 6

7 Key Distinctions Evaluation Examining investments made to assess impact Benchmarking Monitoring socioeconomic measures to identify challenges or opportunities Projection Using known or speculated impacts to predicting the most likely future result 7

8 Key Differences Between Evaluation & Benchmarking Evaluation A process, not a single event Compares What is with What would have been or What should be Search for answers from the program itself Applied research related to ongoing investments and/or programs Practical solutions, not to be sit on the shelf Data based on program information Benchmarking A snapshot, often repeated periodically Compares across places Often, exploratory, seeking to identify problems or successes Focuses on identifying resource gaps to address problems Data based on independent statistics 8

9 Making Your Case: Key Issues in Selecting the Right Approach Projecting Investment Impacts Do the assumptions pass the smell test? Are you accounting for expected dislocation? Benchmarking Progress Have you selected the right measures to compare? Does your planned investments contribute significantly in affecting those metrics? Measuring Program Impacts How much did investment really influence (or cause ) business behaviors? How strong is the data available? 9

10 Challenges in Conducting Evaluations Concern about how evaluation research will be used Lack of credibility given to internal evaluations Lack of expertise among program managers in: Evaluation techniques Data collection process Lack of resources to conduct evaluation Concern that diverted resources take away from program success May 7,

11 Creating an Evaluation Logic Model Types of Measures Inputs / Resources Activities Program Outputs Socio economic Outcomes Linked to Strategic Objectives Guiding Detecting Formulating Funding Provided Staffing Facilities Available Partners Engaged Milestones and Deliverables Generated from tactics or tasks designed to achieve particular results Program measures tied to specific Program Objectives Combined report of program impacts (or predicted program impacts) Benchmark indicators of progress towards a Strategic Vision. Might not be directly tied to any program, but should be tied to Vision 11

12 What Do We Measure? Jobs created or retained Private investment leveraged Total public investment Tax revenues generated 12

13 Need for More Compelling : Lessons from Benchmarking Knowledge Jobs ( Quality Jobs ) Information tech jobs Mgt, prof, & tech jobs Workforce education Immigration of knowledge workers US migration of knowledge workers Mfg value-added High-wage traded services Globalization ( Quality Markets ) Export focus of mfg and services Foreign direct investment Economic Dynamism ( Vitality ) Gazelle jobs Job churning Fastest-growing firms Initial public offerings Entrepreneurial activity Inventor patents The Digital Economy ( New Infrastructure ) Online population Internet domain names Technology in schools E-government Online agriculture Broadband telecommunications Health information tech Innovation Capacity ( Emerging Opportunities ) High-tech jobs Scientists and engineers Patents Industry investment in R&D Non-industry investment in R&D Movement to a Green economy Venture Capital Source: 2008 State New Economy Index Indicators 13

14 Challenges with Measuring Sporadic and inconsistent data collection Rarely validated by a third party Data not always relevant or used Limited priority given to data collection (or analysis) Concern about negatively impacting business relationship 14

15 Case Studies: Benchmarking & Projections and Evaluation EXAMPLES TO HELP YOU FRAME YOUR THINKING May 7,

16 Indicators & Benchmarks Census Bureau Local Employment Dynamics (LED) Community Economic Development Hot Reports May 7,

17 Benchmarking Over Time Maine Economic Growth Council Measures of Growth May 7,

18 Projections / Impacts Example: 200 new distribution jobs in Milwaukee / Madison What is the likely job impacts 468 includes: 256 in the transportation and warehousing 32 in retail trade 31 in health care What is the likely regional economic impact -- $25.8 million Construction of facility is considered separately Dislocation is not often considered at all May 7,

19 Model for Evaluation: Portfolio Maine Comprehensive Economic Development Evaluation System 2 parts innovation and traditional econ dev programs 1. Innovation (i.e., technology programs) completed for past 9 years 2. Traditional business development -- first completed in 2009 Third-party survey of business and community clients Funded by an annual assessment on every program being evaluated Benchmark outcome measures tied to: o o Comparator states Annual Maine Economic Growth Council benchmark report 19

20 Impact Scenario: Sample Economic Impact of Local vs. Corporate Ownership of Wind Energy Production in Big Stone County, MN (operations phase) Ownership Scenario Community Wind (5% opportunity cost of capital) Community Wind (8% opportunity cost of capital) Annual Impact on Value Added Annual County-Wide Employment Impact $1,259, $639, Corporate Wind $249, Source: Arne Kildegaard and Josephine Myers-Kuykindall, University of Minnesota, Morris, Community vs. Corporate Wind: Does It Matter Who Develops the Wind in Big Stone County, MN?, Sept

21 Model for Evaluation: Program NIST Manufacturing Extension Partnership Program Objectives Reduce business costs Expand business revenues Also tracks Jobs created and retained Measures program efficiency (e.g., dollars invested per client assisted) Sample MAIM Scores Evaluation includes: 3 rd party business survey of assistance recipients Minimally Acceptable Impact Measures (MAIM) index Panel reviews colleagues assessing other centers Case studies of successful ventures 21

22 Tools and Techniques for Evaluating or Projecting Impacts THE NITTY GRITTY DETAILS 22

23 Using Economic and Fiscal Impact Analysis as an Evaluation Method Economic Impacts Jobs created/retained Payroll generated (average wages) Private investment leveraged Businesses or entrepreneurs assisted Jobs per dollar invested Fiscal Impacts Tax revenues generated Examples: Sales taxes, property taxes, income taxes, etc. Government costs occurred Direct or indirect one-time or on-going expenses 23

24 Understanding Importance of Economic Interdependencies Interindustry linkages (sellers) Inter-industry linkages (purchasers ) Business-to-business purchases (indirect impact) Examples: Autos buying plastic parts Medical buying instruments Direct Consumption Retail sales Sales to out-ofarea customers Purchases of Labor & Value Added Wages Interest Profits 24

25 Economic Multipliers Recognizes inter-industry linkages ( pebble in a pond ) Quantifies recycling of dollars within and leakage of dollars from the local economy Forces the analyst to consider BENEFITS and COSTS associated with indirect impacts Provides simple tool for estimating overall economic impacts (current or future) 25

26 Calculating a Multiplier Differs depending on unit of measure Employment Business Revenues Personal Income (e.g., wages, transfer payments) Multiplier = (Direct + Indirect + Induced) Direct Includes the direct effects 26

27 Rules of Thumbs For metros-employment multipliers of greater than 3.0 are VERY unusual Manufacturing: Typical range = 2 to 2.5 Non-manufacturing: Typical range = 1.3 to

28 Common Errors in Using Multipliers Made up or misunderstood. Regional preferences differ Defining the industry ( What s the multiplier for manufacturing? ) Defining the study area (The region matters.) Changes in technology 28

29 Common User Errors in Conducting Impact Analyses Emphasizing benefits without recognizing negative consequences Displacement issues (What were residents doing before there was a baseball stadium?) Multiplier effects (What are the public services required for immigrants to the community?) Emphasizing costs without recognizing positive consequences Displacement issues (What are the true costs if we are not adding a lot of new economic activity?) Multiplier effects (What are the revenues being generated from new business activity, especially increased retailing?) Timing of Impacts Investment Today, Impacts Tomorrow 29

30 Fiscal Impact Analysis Summary of Costs Additional public services required for project success Additional public services required to respond to added population Expanded infrastructure needed to handle new business activity Cost of incentives Direct, indirect, and tax benefits funded through public investment Allocation of incentive the public benefit vs. the private benefit from the investment 30

31 The Key Issues in Conducting Fiscal Impacts Generating good estimates of economic impacts Defining relevant costs Estimating costs (average vs. marginal costs) Balancing the presentation of benefits and costs Equity issues (who is impacted is important) Difficulty in generalizations across jurisdictions 31

32 Software: Economic Impact Models U.S. Bureau of Economic Analysis (RIMS II) Minnesota Implan Group, Inc. (IMPLAN) Regional Economic Models Inc. (REMI) Local university analysts proprietary models 32

33 Software: Fiscal Impact Models Federal Reserve Board Fiscal Impact Tool Local university analysts proprietary models Budget, planning, or economic development offices combine commercial models with their own tweaks e.g., Maryland Resource Allocation Model Other proprietary models 33

34 Modeling or Tools Used To Measure Impacts Frequently used economic modeling tools RIMS II U.S. Bureau of Economic Analysis Implan Minnesota Implan Group, Inc. Analyst Economic Modeling Specialists, Inc. (EMSI) Policy Insight Regional Economic Models, Inc. (REMI) 34

35 RIMS II U.S. Bureau of Economic Analysis Strengths Models are built using unsuppressed gov t data Lots of Industry detail (473) The only model with general public name recognition (BEA) Weaknesses Relatively simple technique used to develop multipliers No software; user interface for a series of spreadsheets No forecasting or historical data No access to underlying data 35

36 Implan: Minnesota Implan Group Strengths Lots of industry detail; 500+ industries Detailed agricultural sectors Flexible model configuration Multi-region capabilities Weaknesses Somewhat simple modeling techniques No time series or forecasting capabilities 36

37 Analyst : Economic Modeling Specialists, Inc. Strengths Web subscription More than 700 industries; 800 occupations Can adjust estimated exports (relevant for small areas) Zip level data allows easy-tobuild sub-county models Includes basic cluster analysis (using pre-defined clusters from other research) Part of a suite of products Forecaster and Career Pathways Weaknesses Static model Relatively new to the economic development market (released in 2005) Not well known and greatly tested for this purpose Initially created as a workforce training product Academic articles available, but not comparative 37

38 Policy Insight: Regional Economic Models, Inc. (REMI) Strengths Attempts to model a great deal of economic behavior Can model multiple regions Includes a 50-year forecast (for transportation planning) Most well known of dynamic modeling system Weaknesses Difficult to use correctly Very limited industry detail Most expensive 38

39 Which Tool Should You Use? For the best multipliers all provide relatively similar results RIMS II typically largest For simple impact studies all can be useful, cost may be a factor For lowest cost RIMS II, followed by IMPLAN For brand recognition RIMS II, REMI, and IMPLAN For forecasting REMI For geographic detail IMPLAN and EMSI (zip code level) For the most up-to-date data EMSI (quarterly) Web access EMSI, REMI For phased projects or policy analysis -- REMI For cluster analysis -- EMSI For fiscal impacts crude tools in several of the models 39

40 Fiscal Impact Tool Federal Reserve Fiscal Impact Tool (FIT) Free Useful to evaluate fiscal impacts of local economic development projects This tool was developed to enable local economic and community developers to learn more about the local tax impacts of economic development projects. The Excel workbook provides defaults and simple assumptions enabling the user to input specifications of a project and quickly ascertain what sales and property (and other) tax gains will accrue to an area. 40

41 Fiscal Impact Tool (cont.) In addition to performing an automated tax revenue assessment, this tool can be used for many data purposes. The workbook provides: Census of Retail Trade data on the number of retail establishments, amount of retail sales, and the retail sales per capita and dollar of income of counties and cities, Historical city and county population estimates, and County per capita income figures. The tool is available at Not being fully supported by the Fed in the future 41

42 Lessons Learned From Experience A FEW TIPS AND OTHER WORDS OF ADVICE 42

43 What s Next How to Tell the Story of Econ. Dev. Impacts Communication Strategies Written vs. oral Know Your Audience Policy makers, legislators, media, general public, program managers, clients K.I.S.S. (Keep It Short and Simple) Importance of an Independent Evaluation 43

44 Why We Often Can t Determine Economic Development Success 1. Lack of a clear vision or goals What s the logic behind the intervention? What s the best measure - jobs? 2. Policies not aligned to state s vision or goals 4. Policy and program design excludes evaluation component 5. Lack of resources for evaluation 3. Lack of program resources Can t do enough of anything to have an impact OR Trying to do everything 6. Success expected too quickly 44

45 Recommendations for Moving Forward Demand a vision ; toward which one can benchmark Reward actions rather than promises Require measures; but don t get hung up on jobs Measure public return on public investment (ROI) Examine the entire investment portfolio; don t get hung up on individual programs Invest in the evaluation system, including data management and third-party assessments 45

46 Some Words of Advice Spin-off impacts Real, but often overstated Question earnestly any multiplier impacts over 2 times the direct impact Measuring good jobs 120% average wage rule of thumb Certain occupations are more likely to offer benefits Legislators or councilors who play gotcha won t get cooperation Use qualitative as well as quantitative measures Case studies Capacity building Evaluate for continuous improvement 46

47 Thank You! Ken Poole , ext