Policy Design, Implementation and Evaluation -- Rationale, Efficiency and Systemic Concerns

Transcription

1 Policy Design, Implementation and Evaluation -- Rationale, Efficiency and Systemic Concerns Prepared by Thomas Andersson Deputy Director Directorate for Science, Technology and Industry Organisation for Economic Co-operation and Development (DSTI/OECD) For the SME Forum: Public Policies for SMEs in Europe Lisbon, April Introduction Today, the interest among policy makers in how to improve the performance and social contributions of small and medium-sized enterprises (SMEs) is greater than it has ever been. This interest is fuelled by the perception that the intertwined processes of globalisation and technical progress have given rise to a strong trend for large firms to internationalise and downsize their operations. SMEs, on the other hand, are seen as confronted with a number of new opportunities to do things that previously required large scale, but also as subjected to strong pressures for change which they have difficulty in managing effectively on their own. In addition, it has become clear that many policies and programmes directed at assisting the development of SMEs are not fully effective, and that more could be done. However, there is a lack of adequate information in this respect, partly because of the limited application of evaluation and critical assessment of policy making in this area. In the next section, I will briefly review some of the policy challenges confronting OECD countries with respect to SME development in the New Economy. Building on the experience and current trends in OECD countries 1, I will then turn to policy rationale, efficiency and evaluation of SME policies. I will end with a few remarks on policy implementation. Along the way, I will emphasise the importance of systemic concerns. 1 The OECD Working Party on Small and Medium Sized Enterprises provides a unique forum for policy makers in this area to meet, compare and assess experiences, and to co-operate on key issues.. On June 2000, back to back with the 14 th Session of this Working Party, the OECD will for the first time organise a Conference for Ministers in charge of SMEs and Industry; Enhancing the Competitiveness of SMEs in the Global Economy: Strategies and Policies.

2 2. A New Economy and SMEs A number of recent developments have led to intense debate about whether or not we are living in a new economy. The most important factor underlying the argument for a new economy is the growth record of the United States, which is continuing its prolonged boom and exceptional productivity performance with a striking absence of inflationary pressures. This unexpectedly strong performance is not confined to the United States; a number of other highly developed countries mostly relatively small ones such as the Netherlands, Australia and Ireland are also doing very well. On the other hand, not all countries are showing signs of this new dynamism. It appears that the forces pushing for convergence in growth rates have weakened in the 1990s, and there are now stronger signs of economic divergence. There are also changes in the factors that appear to drive economic growth. In particular, this applies to information and communications technology (ICT), which was long observed to show up "everywhere but in productivity figures". Today, the contribution of ICT to output and productivity growth is visible, significant and rising (Schreyer, 2000). 2 Rather than stemming from ICT per se, however, it would seem that growth is strongly dependent on the interplay of ICT with changes in innovation processes, organizational change and upgrading of human skills. Among the most conspicuous changes is the trend towards take-overs and acquisitions, and the formation of strategic alliances between the biggest and most powerful firms. Many questions have been raised about what all this means for small firms. SMEs play a key role in all OECD countries in many respects, and especially in terms of employment. Figure 1 shows the distribution of the labour force within the manufacturing sector in a selection of countries as of the late 1990s. As can be seen, using a definition for SMEs of firms employing up to 99 people, these firms account for a share of between 30% and 60% of the labour force in all the countries except the United States. Using the US definition of an SME as a firm with up to 500 employees, the category accounts for more than 50% in all countries except the United States. Today, however, the bulk of total employment is found not in manufacturing but in services, and there SMEs have an even stronger position, particularly in wholesale and retail trade and in hotels and restaurants. In construction, SMEs tend to account for 80 to 90 percent of all employment. Again, the distribution varies across countries (OECD 1997ab and 2000). 3 Moreover, the importance of SMEs in employment tends to be increasing. From the mid 1980s to the early 90s, SMEs displayed more rapid net employment growth than larger firms in all OECD countries. This development is partly because of rationalisation in large firms following pressure to increase productivity and, lately, focus on core business. Some of the new jobs in SMEs thus emerge as the result of outsourcing from large firms, and today there is a very dynamic 2 In Canada, the United Kingdom and the United States, the growth contribution of ICT equipment amounts to about half of the entire growth contribution of fixed capital. In France, Germany and Japan, the contribution of ICTs to output growth has been somewhat smaller, but is still significant. 3 For example, SMEs share of total private sector employment has been estimated to reach about 80 percent in Ireland and Switzerland, about 70 percent in Japan, Belgium and the United Kingdom, and some 50 percent in Italy and Finland as of the mid 1990s (OECD 1997b).

3 development of SMEs in business services, many of which co-operate closely with manufacturing firms. However, jobs in SMEs are less stable; there is much more turmoil caused by births and deaths among these firms compared to bigger ones. Also, SMEs display much lower productivity growth than bigger firms, their rate of internationalisation is much lower on average as is their R&D intensity and their use of new technologies. Are SMEs destined to fall behind? There is no one answer to this question that applies to all SMEs, but many categories of SMEs enjoy an advantage in terms of speed and flexibility and these are major advantages, given the nature of the new economy. They are less likely than big firms to suffer lock-in with respect to existing plants, technologies and organisational structures. They tend to be strong on incremental innovation, which is close to the adoption and use of technologies for new market needs. ICTs are also opening up new opportunities, allowing the advantages of small scale to be combined with economies of scale and scope through networking among SMEs (or with larger firms or other actors such as universities). There is evidence that networking is becoming key for enhanced specialisation among SMEs. Our understanding of the new economy is certainly far from perfect. The OECD has just embarked on a major project to examine the magnitude and nature of the changes that are taking place in growth performance; an interim report will be published in June this year and the final report is due in Some policy issues are nevertheless clear-cut: reduced costs for accessing and using information infrastructure are crucial for SMEs. Well functioning venture capital markets, economic and regulatory conditions that allow for stable and secure transactions, mobility and the diffusion of technologies, knowledge and skills, are all important. Furthermore, the prevailing economic and social conditions must be conducive to entrepreneurship and should not discourage risk-taking. On a more fundamental level, however, the policy challenges are complex: individuals and countries are affected by a range of inter-related policies. They fare increasingly differently, raising concern about the existence of a digital divide, characterised by exploding income differences as those who engage in cumulative learning reap the gains of expanding information flows, while those who lack the means and knowledge of how to gain access to the new technologies are left behind. Social safety nets obviously have a role to play in counteracting such developments. One should not, however, underestimate the importance of the policy frameworks in place with respect to SMEs, start-ups and entrepreneurship. 3. Rationale for Policy SMEs face special challenges but also special opportunities. SMEs are important for the economy, particularly job creation. So, is that not enough justification for implementing SME policies? The answer is no it does not warrant providing support simply to help SMEs. One cannot sufficiently emphasise the importance for governments of taking the concept of rationale seriously. Special SME polices or programmes should be adopted only to the extent that there is a particular rationale for doing so. And, in many cases, there is, in the form of market, government or systemic failures. Let me give a few examples: i. Due to their limited capacity to carry fixed costs, SMEs often suffer from asymmetric information and have difficulty evaluating the benefits of, e.g., new technologies or entering distant markets.

4 ii. The social returns of the evolutionary process of SME development are generally higher than the expected private returns, resulting in under-investment by the market. iii. The development of networks is confronted with various information and regulatory barriers, which may prevent networks from taking shape spontaneously in the first place or, at least, significantly slow their development. iv. Because of lack of collateral and a proven track record, SMEs tend to have relatively little bargaining power vis-à-vis creditors or customers. v. SMEs tend to shoulder a relatively heavy burden in regulation since overly bureaucratic administrative practices impose high fixed costs on individual firms. Standardisation can prevent flexibility, which is an important source of competitiveness for SMEs, not least in the case of new, technology-based firms. vi. SMEs have a limited ability to influence government policy in terms of access to special programmes, e.g., in research. As can be seen from the above, policies are sometimes the problem, rather than the solution. In fact, SME policies present some inherent problems. Financial support for industrial R&D provided in the form of tax incentives is typically less useful for SMEs. The risk of moral hazard or adverse selection in the provision of direct subsidies is also likely to be particularly serious for SMEs, creating a need for financial institutions with special expertise in micro-credit programmes and the handling of risk in the early stages of entrepreneurial development. Much of the government support provided to SMEs in the past has ignored such problems, as it has also neglected to pay serious attention to rationale. The policies of the past often served to protect SMEs from normal business pressures, leading to a dependence on government programmes that has served to diminish their competitiveness and innovativeness, ultimately leading to even greater difficulties. In today s era of intensifying globalisation and technical progress, policy makers must seriously reflect on how SME policies can work alongside open markets, or catalyse efficient markets (markets cannot be taken for given: often there is no market, or only a poorly functioning one), rather than trying to prevent or circumvent them. The opening of markets brings in its wake both greater risks and greater opportunities the task is to achieve results which maximise the opportunities. The most important in this respect include: i. The development of better equity and venture capital markets, which can improve risk evaluation and distribute resources more effectively to early stages of business. There are large differences between countries in the respect to which venture capital is channelled to early business stages, see Figure 2. ii. The diffusion, absorption and use of technology in the SME sector, including the Internet and electronic commerce. iii. The formation of SME networks. The available experience suggests that policies which are successful in promoting network development need to take a broad, well-co-ordinated

5 demand-oriented strategy, combining public procurement, regulatory reform, and measures facilitating the provision of venture capital [UNIDO 1995; Schmitz and Nadvi, 1999]. Countries and SMEs are making progress in all these areas. While SMEs are behind in Internet penetration, Figure 3 illustrates a consistent increase in different categories of firms within three countries where comparable data is available. In the European context, Figure 4 points to a very strong increase between 1997 and 1999, particularly in Internet to sell and Internet to buy. There is no panacea for improving SME performance. While OECD governments seem to have made certain reductions in their overall public support to industry over the last decade, programmes seeking to foster stronger development among SMEs are generally on the increase (OECD, 1998b). To succeed, however, governments have to engage in a systematic and permanent reform process. Continuous learning is key to improvement, and international exchange of experience and co-operation is a cornerstone in this respect. 4. Efficiency criteria Measuring the efficiency of SME policy is a far from simple exercise. As is typical in the social area, the outcome cannot be observed since one cannot know what would have happened in the absence of the measure (using a control group to compare is not always either practical or fully effective). Nevertheless, efficiency does need to be measured on the basis of appropriate criteria. Based on the OECD s analytical framework for policy evaluation, I would emphasise that programmes, to the extent feasible should be evaluated with regard to their efficiency in the following areas: i. Appropriateness: Does the programme address an objective that can be clearly related to policy rationale in terms of market, policy or government failure? ii. Superiority: Is the programme more effective than other instruments that might have achieved the same goals? iii. Systemic Efficiency: How does the policy interact with other policies? Is there efficiency or suboptimality in this respect? iv. Own Efficiency: Is the programme cost-effective in achieving its specific objectives? v. Adaptive Efficiency: To what extent do the results from evaluations feed back into policy design and implementation? Does policy design ensure a sufficient degree of flexibility enabling it to respond to the need for change? These five aspects of efficiency should be examined with a view to the appropriate objectives. There are several alternatives in this respect, including: i. Objectives of the SME programme (these should be compatible with overall government objectives and should address market failures); ii. Overall government objectives (e.g. economic growth, job creation, enhanced competitiveness, export promotion, regional growth policies, etc.); and

6 iii. Framework conditions (e.g. the tax system, overall macroeconomic conditions, regulatory environment, bankruptcy laws, competition policy, etc.) If efficiency is only considered in terms of the first of these objectives, there is a risk that policy will result in a piecemeal approach. The objective of a given programme may overlap or be inconsistent with that of another one. Programmes and policies are often contradictory, especially when they go beyond the area of SME policies in the narrow sense. The incentives confronting SMEs are influenced by a range of policy areas and the optimum solution to a particular problem may not be confined to SME policy in a narrow sense. If second-best solutions and cross-purposes are to be identified and avoided, governments must not fail to take account of the second and third objectives. This distinction between objectives reflects the role of different actors. For those involved in the actual management of a government programme, including some policy makers, the focus is entirely on internal programme efficiencies and on maximising the quality, responsiveness and delivery of the scheme. There is a need for policy makers and society stakeholders (i.e. tax payers, consumer interests) to concern themselves with incentive effects and resource allocation as well as the choice of policy instruments. 5. Evaluation issues In the last few years, many OECD countries have become aware of the importance of evaluation and taken steps to become more active in this area. Techniques and methodologies have been elaborated to measure and capture the direct and indirect effects of programmes as well as their implementation. Methodologies such as in-depth surveys, econometric techniques and cost-benefit analyses are being used either alone or in combination, depending on the nature of the programme and the scope of the exercise. In relation to evaluation methodologies, the following can be recommended: i. Cost-benefit frameworks should be applied; ii. Quantifiable, measurable criteria should be strived for in determining success; iii. Evaluations need to attempt to determine what would have happened in the absence of a programme or policy and whether a control group (individuals or firms that do not participate in the programme but which are identical in other respects) should be set up or not; iv. Evaluations need to take account of the temporal dimension of programmes and their impacts, developing retrospective ex-post evaluations, or in contrast, ex-ante evaluations depending on whether the aim is the monitoring of ongoing programmes or the forward planning of policies and programmes. v. A combination of evaluation tools and approaches (both quantitative and qualitative) is required in order to optimise the evaluation process and increase the credibility of evaluation results and, by extension, the policy recommendations that emerge.

7 Programme and policy evaluation experiences differ widely among countries, involve a large variety of evaluation instruments and concern a broad range of institutions. Based on technologyrelated assessments of evaluations undertaken in 1998 (OECD, 1998a), Figure 5 provides a broad illustration of the status of evaluation in OECD countries. The picture should be treated with caution, since the assessments underlying it are a few years old, and apply to a set of issues which are broader and only partly identical with those of SME programmes. In addition, each country s evaluation practices are to a certain extent specific. Nevertheless, this characterisation is of certain relevance for comparing the approach to evaluation in the SME-area (and especially with respect to the fostering of new technology based firms, or for programmes fostering diffusion of technology to SMEs). Some key groups of practices can be distinguished, partly reflecting institutions and country specificity. The maturity of a particular policy area within a specific country matters, for instance, since evaluation practices tend to be adopted only after a certain time lag. Another factor is attitude towards the appropriate role of public institutions, although there has been some convergence in this respect in recent years. To briefly reflect on the individual countries, Australia, Canada, the United Kingdom and the United States share many evaluation practices. These countries have a relatively strong focus on evaluation, the characteristics of which are partly coloured by tight public budgets and related efforts to rationalise government action. Although there are relatively few weaknesses in these countries, each of them has room for improvement. Denmark, Finland, France, Germany, the Netherlands, Norway, Sweden and Switzerland tend to have a receptive policy attitude towards evaluation. Strictly economic objectives and criteria in evaluation are less explicit, however, and there is limited use of evaluation as a means to improve the allocation of public funds among competing uses. In the remaining OECD countries, evaluation tends to be a more recent phenomenon. While the legal framework for evaluation often exists, evaluations have not yet become a regular fixture of policy making. The methodologies used tend to be qualitative, relying on expert advice, with limited efforts to quantify the impacts of interventions. Their use as a guide for allocating resources has so far been limited in these countries as well. While each country needs to look at its own pressing issues and identify priorities, OECD experience suggests that the following recommendations can serve as a good guide for improvement in evaluations of SME-policies in Member countries (OECD, 2000): the evaluation methodology should be designed at the same time as the policy or programme so that the necessary data and acceptance by the relevant parties of the evaluation procedures and criteria can be secured; evaluation should be user-oriented, serving the information needs of the different programme participants and clients; evaluations should use a combination of methods to satisfy different information needs; and. there is a need to ensure that the take-up of evaluation results reaches the adequate level of policy and decision making, encouraging public discussion of results and ensuring a response to evaluation results in terms of action for improvement;

8 Needless to say, evaluations are not without cost. How far can and should evaluations go? If they are too ambitious, this will result in markedly diminishing returns. Trade-offs need to be made so that those programmes and initiatives for which evaluation is most important are in fact evaluated. However, to date, evaluation remains at a fairly modest level in all OECD countries relative to the cost of the policies and programmes to be assessed. The key task at this juncture is to increase awareness of the benefits as well as the costs of not evaluating, and to encourage governments to adopt an evaluation culture. This will entail developing a critical attitude towards methodologies and outcomes of evaluations, but also an awareness of the issues at stake. 6. A few concluding remarks on implementation The evaluation tool should enable more effective adjustment in the design and implementation of policies and programmes. For this to be the case, an integrated approach needs to be taken to programme design, implementation and evaluation. How can this be achieved? Encouraging stronger entrepreneurial activity in the knowledge-based economy clearly transcends traditional delineation of political responsibility. In many cases, broader, more constructive cooperation within governments will be important for achieving more relevant and effective policy outcomes. The involvement of relevant societal stakeholders in the policy process is also important for laying the basis for adequate policy design and delivery, not least since SMEconcerns otherwise risk being neglected in the policy process. Finally, international co-operation and exchange of experience can serve to focus attention on the importance of critical examination and assessment of SME-policies. Measurement of policies is difficult in this area but it is necessary for learning, and we can all learn from others policy experience.

9 Figure 1. Employment Distribution by size class in Manufacturing (late 1990s) (0-19) (20-99) ( ) 500+ per cent 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% United Japan France Germany Spain United Korea New Czech Turkey

10 Figure 2. Venture capital investment by stage as a percentage of GDP, average Canada Netherlands Iceland United States United Kingdom Norway Belgium Finland Europe Sweden Ireland Portugal France Germany Australia (1997) Spain Italy Switzerland Greece Denmark Austria Japan (1994) Early stage Expansion Buyouts and others Source: OECD, based on private sources. Note: Caution should be used in international comparison, especially regarding buyouts.

11 Figure 3. Internet penetration rate in the business sector Denmark (+) Austria Jul-98 Mar-99 Netherlands to 9 10 to to to to to and more Source: KSV and WirtschaftsBlatt,(Austria), Statistics Denmark (Denmark), Statistics Netherlands (Netherlands).

12 Figure 4. Use of computers and the Internet by European SMEs (% of enterprises with computers and the Internet) Figure 4: Use of computers and the Internet by European SMEs ( in % of enterprises with computers and the Internet) Internet - information search Internet - to sell Internet - to buy Source: The European Business Survey, Grant Thornton International. Note : SMES are enterprises with between 10 and 250 employees.

13 Economic rationale and objectives, additionality Figure 5. Best practices and weaknesses in evaluation Evaluation of actors and institutions Use of quantitative methods in evaluation Use of qualitative methods Institutionalised rules and procedures for evaluation Regularity of evaluations Systemic approach to evaluation Policy feedback and learning Australia l l l l l l m Austria m m m l m m m m Belgium m m l m m Canada l l l l l l l Czech Republic m l m m m m Denmark l l l m Finland m l m l l l l France m l l l m m m Germany l m l l m Greece m m l m m m Hungary m l m m m m Ireland m m Italy m l m m m m Japan m m m m m Netherlands l l l m l New Zealand l l m l m Norway l l m Portugal m m l m m m m Spain m l m m Sweden m l m l l Switzerland l l l l United Kingdom l l l l l m l United States l l l l l l m m European Union l l l l l l Key: l represents strengths (best practices); m represents weaknesses; no symbol signifies insufficient information or absence of major strength/weakness. Source: OECD Secretariat.

14 References OECD, 1997a, Small Business Job Creation and Growth: Facts, Obstacles and Best Practices, Paris. OECD, 1997a, Globalisation and Small and Medium Enterprises (SMEs), Paris. OECD, 1998a, Technology, Productivity, Job Creation - Best Policy Practices, Paris. OECD, 1998b, Spotlight on Public Support to Industry, Paris. OECD, 1999, Science, Technology and Industry Scoreboard: Benchmarking Knowledge-based Industries, Paris. Schmitz, Hubert, and Khalid Nadvi. 1999, Clustering and Industrialisation: Introduction, World Development, 27(9): Schreyer, P., 2000, The Contribution of Information and Communication Technology to Output Growth: A Study of the G7 Countries, STI Working Paper, 2000/2, OECD, Paris. UNIDO. 1995a, Principles of Promoting Clusters and Networks of Small and Medium Enterprises. UNIDO. 1995b, Industrial Clusters and Networks: Case Studies of SME Growth and Innovation. 1