As a result of an increasingly global economy based on knowledge and innovation, and in the wake of the achievement of the internal market and the EMU, Europe is in the process of adjusting its policies and its economies in order to ensure competitiveness, and thus employment, sustainable growth and social cohesion. The central role played by the European research and innovation system in the transition to the knowledge-based economy is highlighted in the recent Commission Communication Towards a European Research Area. It pinpointed, in particular, the need to decompartmentalize and better integrate Europe s scientific and technological area. In this context, a benchmarking approach can help to measure the performance of the European research and innovation system, to learn about best practice in the field and to provide insights into how policies might be improved and made more effective. Moreover, in view of the strong interactions between different public policies (research, innovation, education, industrial competitiveness, etc.), there needs to be a coherent and global approach towards benchmarking across all these policy fields.. With its origins in the business enterprise where pioneering work was done comparing enterprises of similar type with a view to transferring best practice benchmarking has now spread to many other fields including that of public policy, where it is increasingly seen as a powerful strategic tool to assist policy making and to improve the quality and effectiveness of public services. understand the factors influencing performance at multiple organisational levels (national, regional, enterprise, institution etc.) in various fields of policy concern. Moreover, learning about best practice can provide them with better insights into how appropriate policies, mechanisms or tools can be developed or made more effective, while monitoring can provide feedback on progress and needs for further improvements. Using performance indicators, a benchmark is established which sets the standard or target for improvements. However, in order to raise one s performance up to this target, one must understand how the benchmark has been achieved through analysis, learning and examination of best practices., especially at the European level, in view of the diversity of national R&D and innovation systems, and the consequent difficulties of transferring best practice from one country to another. To be successful it requires the full participation of the member states, and a clear willingness for co-operation and mutual 1
information exchange. To achieve this, the process must be managed through a partnership between the European Commission and the member states. : identification of the target issue to be improved, development of specific performance indicators for that issue, comparison of own performance against the "best in the world", analysis of best practice and improvement potential, implementation of actions to improve, and continuous monitoring of results. It should be stressed that the objective of benchmarking national R&D policies is, but rather to identify best practice, provide real metrics of national efforts, and help pinpoint weaknesses or bottlenecks that can block excellent ideas or hinder the development of policies., benchmarking is already used in several key areas as a tool for improving policies in Europe: In the context of the Luxembourg Process, mechanisms have been developed for the in the European Union. Progress towards the policy commitments set out in the Employment Guidelines is systematically monitored, and measures undertaken in the National Action Plans are regularly reviewed. A system of takes place in the framework of the Stability and Growth Pact, and the monitoring of developments relevant to monetary policy in the euro area. The has been carried out since 1996, with activities covering the framework conditions underlying competitiveness, as well as the promotion of sectoral and enterprise level benchmarking. A trendchart on innovation is also in progress. In a number of other policy fields benchmarking approaches have been adopted, notably the benchmarking of, the monitoring of (Single Market Scoreboard), and the Innovative Public Services initiative involving identifying. several benchmarking exercises are in progress or have recently been completed. These are not entirely comparable neither in terms of the choice of subjects nor on geographical coverage. The references provided below are not exhaustive. Denmark would appear to be quite advanced in benchmarking both at the level of individual research organisations, and on broader RTD policy issues, comparing best practices with Finland, Ireland and Israel. Ireland is also further developing mainly quantitative indicators on education and industrial RTD. In other cases, benchmarking is carried out on particular issues. For example, Austria is about to complete a benchmarking exercise on mobility of scientists, while Norway is analysing this same subject from a primarily statistical viewpoint. Spain has undertaken a benchmarking-type exercise of the performance of a group of universities in terms of their research and training capacities, as well as a comparative analysis of a limited number of universities and technology transfer centres in terms of their capacity for exploitation of results. Germany has quite a developed evaluation system of actional RTD programmes and institutions aiming at spreading best practice. 2
Some sectoral exercises are also in progress: for example, on information and communication technologies in the Netherlands, on standardisation in Finland, or on electronic networks in Belgium. Certain other countries, such as Sweden, have recently completed benchmarking of research management, an exercise which is somewhat comparable to the one launched by the Commission research services. Greece has an on-going benchmarking exercise, mainly related to selection processes and funding of RTD organisations, which is expected to be completed by the end of the year. In the UK, several on-going actions focus mainly on the relation between science and industry. In France, a comparative analysis of innovation systems of research countries has recently been completed. Experience of benchmarking in the member states and in the Commission services has shown that, for the benchmarking process to lead to successful improvements, several operational elements are required : : in order to support the activity and overcome possible reticence production and development of key performance indicators / scoreboards for particular issues qualitative analysis of best practice, understanding of factors behind good performance in order to have a clear and up-to-date picture of policies employed, and their relation to performance in order to foster improvements and learning, and to report progress steering groups are necessary in order to set targets, choose parameters, monitor activity and streamline implementation and follow-up In order to increase the contribution of research and innovation policies to sustainable growth and creation of employment, this benchmarking exercise should cover the areas of human capital, education, science, technology and innovation. These areas are complementary and interactive. Implementing the proposed benchmarking exercise needs a. At least four structural elements are required : The first is a high level of political commitment by the member states. This can be achieved by the setting-up of a. This group may include different actors from the European research and innovation system. Sub-committees and technical expert groups could also be used where needed. The mission of this High-level Group should include : acting as a guarantor of the quality of the benchmarking process ; taking decisions on the programming of benchmarking activities and on the provision of necessary mechanisms to implement them ; 3
discussing and evaluating the results of the benchmarking exercise, and the feasibility of transferring best practice. The second is the progressive establishment of a on the performance of research and innovation systems in Europe. This should include both quantitative measures (pertinent indicators) and qualitative information (eg. on the policy or regulatory environment) that can be used to benchmark performance. Efforts should be targeted, in the initial phase, on a number of key issues of interest to European policy makers (see below). The third element concerns the required to understand the factors behind the observed performance. At the beginning this could take the form of key pilot studies, which would make use of the quantitative and qualitative data, but would go further by trying to shed light on why particular performance levels are achieved. It could include the analysis of : the specific mechanisms and processes that lead to the performance revealed by indicators; the identification of key factors prevailing in best practice; the impact of political or regulatory developments on identified processes; the transferability of identified best practice and assessment of the framework conditions required for implementation in another context (e.g. in another Member state). The fourth relates to the. This should involve presenting the results of the benchmarking process at regular intervals. (including scoreboards and pilot studies). In this context it is also foreseen that an should be produced which would integrate several complementary elements : an overview of EU RTD policy developments and implementation (as required under Article 173 of the Treaty), a presentation of the main developments in national policies, highlights of the results of the benchmarking exercise, and pertinent indicators and analyses relating to Europe s investment and performance in S&T. In addition, there should be the necessary forums for evaluation and debate of the findings, for learning how improvements can be implemented, and for providing feedback on progress. From a rapid analysis of the development of member states R&D efforts and performance, and taking into account the main preoccupations of European policy makers at this moment, the following three strategic areas emerge as promising candidates for a benchmarking exercise. In each of these three domains, an effort would have to be made to collect new and pertinent indicators, and to carry out in-depth analyses of the various S&T mechanisms and policies and how they interact with the key objectives of employment, sustainable growth and social cohesion. (economic and financial factors specific to each country, impact of tax systems, factors related to industrial and competition policies, social factors, etc.) (organisation of research : content and focus, openness of research institutes, evaluation systems, higher education related research ; funding issues : budgets devoted to funding of new projects, national programmes and public 4
research centres, advantages and disadvantages of competitive and institutional financing mechanisms) (content and forms of research undertaken by entreprises ; funding mechanisms including venture capital companies, business angels, financial and banking institutions, etc.) (cooperation between universities and industry, inter-firm cooperation, creation of start-ups by researchers, ) (development of spin-offs and start-ups in each member state, incitative best practices, needs and obstacles, ) (public instruments and mechanisms, regulatory and administrative aspects, intellectual property and patent systems,..) (efforts to encourage students to choose S&T fields of study, attractiveness of S&T professions) (brain drain, mobility, conditions of work) If the ministers agree to give their political impulse to this process with the establishment of a High Level Group, and provided initial work can be focussed on a number of key areas and indicators, then the first results of a benchmarking exercise could be ready within a year. 5