2014 TAFTIE Policy Forum Measuring innovation. Can we measure innovation? Lessons from innovation scoreboards

Transcription

1 2014 TAFTIE Policy Forum Measuring innovation Can we measure innovation? Lessons from innovation scoreboards Hugo Hollanders Brussels, November 25, 2014

2 What is an innovation scoreboard? Innovation scoreboards consist of a collection of national or regional indicators of relevance to innovation Arundel, A. and H. Hollanders, "Innovation Scoreboards: Indicators and Policy Use", in C. Nauwelaers and R. Wintjes (eds.), Innovation Policy in Europe, Edward Elgar: Cheltenham, 2008 Innovation scoreboards are an attempt to summarize innovation indicators and to compare innovation performance of countries, regions or sectors Hollanders, H. and N. Janz, "Scoreboard and indicator reports", in F. Gault (ed.), Handbook of Innovation Indicators and Measurement, Edward Elgar : Cheltenham, 2013

3 Purpose of innovation scoreboards The main purpose of an innovation scoreboard is to assist policy by summarizing a range of innovation indicators at the national, regional or sectoral level by permitting a comparison of the relative success or failure of the innovation system, or through the identification of specific aspects of the innovation system which perform well or poorly

4 Policy relevance Innovation scoreboards serve three policy needs They act as an early-warning system for potential problems If used over time, they can track changes in innovation strengths and weaknesses They can attract the interest of policy-makers and as such can help build a consensus to introduce actions and policies to improve the innovation capabilities of firms and institutions Policy use Innovation scoreboards will be most useful for policy when combined with an integrated system for evaluating innovation policies

5 Principles of scoreboard design The indicators should Be of similar importance as measures of the drivers of innovation activity Be based on reliable statistics Hold their value over time Be of relevance to medium- and long-run policy issues

6 Examples of innovation scoreboards European Commission Innovation Union Scoreboard (IUS) Innovation Union Competitiveness Report OECD Science, Technology and Industry Scoreboard OECD Science, Technology and Industry Outlook INSEAD/WIPO Global Innovation Index World Economic Forum Global Competitiveness Report World Bank Knowledge Assessment Methodology Deutsche Telekom Stiftung, BDI Innovationsindikator

7 Name Innovation Union Scoreboard Innovation Union Competitiveness Report OECD Science, Technology and Industry Scoreboard OECD Science, Technology and Industry Outlook Global Innovation Editing Institution European Commission European Commission Index Innovationsindikator Telekom Stiftung, BDI Global Competitiveness Report Knowledge Assessment Preparing Institution Examples of innovation scoreboards First Ed. Last Ed. Frequency Standard Target Number thereof structure countries of based on indicators innovation surveys MERIT Annual Yes EU Member States European Commission Biennial unknown EU Member States OECD OECD Biennial No OECD countries Selection criteria 25 6 Reasoning, Correlation analysis Composite Indicator Yes 51 None No ~ Reasoning No OECD OECD Biennial No OECD 22 None Not No countries (country available fiches) INSEAD, INSEAD Annual Yes World 84 None Yes WIPO Fraunhofer Annual Yes Germany 38 None Model, Yes ISI, and Regression ZEW, MERIT selected analysis World Economic Forum World Bank Centre for Global Competitiveness and Performance World Bank Regularly updated countries Annual Yes World 116 None Yes Yes World 148 None Yes

8 Innovation Union Scoreboard IUS monitors the implementation of the Europe 2020 Innovation Union flagship by providing a comparative assessment of the innovation performance of the EU28 Member States and the relative strengths and weaknesses of their innovation systems IUS uses 25 indicators which are grouped into 3 main categories: Enablers capture the main drivers of innovation performance external to the firm Firm activities capture the innovation efforts at firm level Outputs capture the effects of firms innovation activities Average performance is measured by a composite indicator, the Summary Innovation Index 8

9 IUS Measurement framework Summary Innovation Index (SII) ENABLERS FIRM ACTIVITIES OUTPUTS Human resources Open, excellent, attractive research systems Finance and support Firm investments Linkages & entrepreneurship Intellectual assets Innovators Economic effects New doctorate graduates International scientific copublications R&D expenditure in the public sector R&D expenditure in the business sector SMEs innovating inhouse PCT patent applications SMEs with product or process innovations Employment in knowledgeintensive activities Population aged with tertiary education Top 10% most cited scientific publications Venture capital investments Non-R&D innovation expenditure Innovative SMEs collaborating with others PCT patent applications in societal challenges SMEs with marketing or organisational innovations Medium-high & high-tech product exports Youth with at least upper secondary education Non-EU doctorate students Public-private copublications Community trademarks Employment fast-growing firms innovative sectors Knowledgeintensive services exports Community designs Sales of new to market and new to firm innovations License and patent revenues from abroad 9

10 IUS Performance groups 10

11 Differences in rankings

12 IUS International comparison Europe s performance gap with the US and Japan is closing but the gap with South Korea is increasing Europe is

13 Lessons from innovation scoreboards What are the strengths and weaknesses of the innovation measurement tools and indicators we use today? Strengths Country coverage, measuring variety of innovation activities, good coverage of inputs and throughputs, use of innovation survey data Weaknesses Partial coverage of outputs, use of innovation survey data

14 Lessons from innovation scoreboards What s the state of play on the development of new indicators? Community Innovation Survey 2014 Better coverage of public sector as a driver of innovation Public procurement Ad-hoc modules: 2012 on growth, 2014 on innovations with environmental benefits EC s Innovation Output indicator Combines 5 of the IUS indicators into one indicator measuring outputs of which one is a new indicator: Employment in fast-growing firms of innovative sectors

15 Fast-growing firms innovative sectors 2010: High-Level Panel asked to recommend new headline innovation indicator : work by OECD/Eurostat to identify the innovativeness of sectors 2013: indicator launched by EC Employment in fast-growing firms of innovative sectors provides an indication of the dynamism of fast-growing firms in innovative sectors as compared to all fast-growing business activities. It captures the capacity of a country to transform rapidly its economy to respond to new needs and to take advantage of emerging demand

16 Lessons from innovation scoreboards What has not been measured yet? Outputs, current indicators focus on Employment in high-tech and knowledge-intensive activities Exports in high-tech and knowledge-intensive activities Both of which do not necessarily have to be directly linked to innovation Sales of innovative products Data are not available for all countries and there are issues about the comparability of results across countries Commercialization activities Innobarometer 2014

17 Commercialization of innovations

18 Commercialization of innovations

19 Thank You

20 Lessons from innovation scoreboards Is there a better taxonomy and nomenclature for the innovation process? How can we develop the process of finding the exact tools to observe, monitor and describe policy relevance, data quality (international availability and cross-country comparability) and robustness of results? What could be the role of TAFTIE in developing innovation indicators and ensuring the most relevant picture of impact of TAFTIE support?