The European Commission s science and knowledge service. Joint Research Centre

Transcription

1 The European Commission s science and knowledge service Joint Research Centre

2 The Innovation Output Indicator Dániel Vértesy COIN CoP 2017 Composite Indicators & Scoreboards Community of Practice: 2nd Annual Meeting 09-10/11/2017, Ispra (IT)

3 What does the IOI measure? Innovation output: the extent to which innovative ideas can reach the market A composite index of 4 components (5 variables): Innovation Output Indicator (IOI) PCT KIABI COMP DYN GOOD SERV Published by DG RTD since 2013 Methodology developed with JRC/COIN support

4 Components PCT PCT patent applications per billion GDP (in PPS) "2016" (2014) "2011" (2009) "2015" (2013) SE CH DE DK NL AT US FR EU BE IS UK NO IT ES EE HU MT CZ LT PT TR BG HR PL EL SK LV BR RO MK KIABI Share of employment in knowledge-intensive activities in business industries (in %) "2016" (2016) "2011" (2011) "2015" (2015) CH IS MT SE UK NL US DK BE NO DE AT FR EU IT CZ EE ES EL HU HR LV PT BR BG PL SK LT RO TR MK

5 Components GOOD Exports of medium- and high-technology products as a share of total product exports (in %) "2016" (2016) "2011" (2011) "2015" (2015) HU MT DE SK CZ EUx FR UK AT EU MK SE RO US IT CH PL NL BE ES DK EE HR PT TR LT LV BG BR EL NO IS SERV Knowledge-intensive services exports as percentage of total services exports (in %) "2016" (2015) "2011" (2010) "2015" (2014) NO BR EUx NL SE DK DE UK EU CH BE FR US IS LV IT HU EE RO EL AT PT CZ BG PL SK MT ES TR MK LT HR

6 Components DYN Employment in fast-growing enterprises in the top 50% most innovative sectors as a percentage of total employment (in %) "2016" (2014) "2011" (2012) "2015" (2013) HU SK MT UK BG SE BR PL NL CZ LV NO EU DE DK FR LT PT ES CH EE AT HR RO IT BE

7 Why measure innovation output? Innovation key to Europe 2020 strategy: make the EU a dynamic, sustainable, knowledge-based ( smarter ) economy The political mandate from the European Council: Complement the Lisbon agenda s 3% R&D [input] target with output-oriented indicator(s) Aim to complement the Innovation Union Scoreboard, and its Summary Innovation Index (I), which measures overall performance, by zooming in on a reduced set of dimensions

8 High-level panel on the measurement of innovation proposed a short list of indicators: Each of them, alone, yields only a partial view of the innovation phenomenon, but together they provide if not for a perfect, at least for a clearer assessment EC (2010)

9 Why multiple indicators? Single indicator: hourly labor productivity: indirect measure Short list of indicators offering a clearer picture of the innovation phenomenon No ideal world: Existing data: not good enough Proxies based on data produced by official statistics are available Policy demand for a concise picture

10 How: a composite index or a list of indicators? Disagreement: use a composite index or not? Against: HLP s reservations: not easy to interpret or... if there is a composite, HLP advised to: keep the number of indicators limited to ensure transparency; Ensure a relative independence of components; Make possible to monitor the policies addressing the various components First solution: a complex formula of 2 indices (sectoral innovation performance, sectoral innovation dynamics) Make use of innovation coefficients computed by OECD and Eurostat s new employment in high-growth enterprises data Stakeholder reservation: Too complicated; insufficient coverage!

11 How: a composite index or a list of indicators? Political decision to go for a composite index For: Summarize an admittedly multi-dimensional phenomenon Useful for delivering policy messages (factors hampering R&I system performance) Easy to interpret Inter-service Task force (DG RTD, ECN, ESTAT, GROW, JRC, TRADE, etc.) established to develop an index from a list of 4 components

12 JRC team s role in developing the index Building indicator Improve robustness Statistical validation: checking conceptual and statistical soundness; Perform global sensitivity analyses (MC simulations, alternative weights, DEA, etc.) Computing scaling factors: Decision to address statistical imbalances across dimensions using effective equal weights

13 Updates and refinements Communication challenge: align with EIS Similar measures: avoid different message Frequency of publication: align or not? Refinements: if you want to measure change, better not change the measurement Increase country coverage (given data availability) Indicators refined

14 Results

15 The EU in international comparison US EUx BR

16 JRC - COIN Statistical Support Balancing components (Computing scaling coefficients) for annual updates

17 JRC - COIN Statistical Support Balancing components (Computing scaling coefficients) for annual updates

18 Statistical coherence CH IS MT UK US NL NOBE EU28 CZ AT DK IT FR DE EL EE HRHU ES BR LV SK PL BG PT LT MKTR RO DE UK EU28 FR HU NL DK BE US CH SK CZ MT RO IT AT BR PL EE MK LV PT ES NO BG TR EL HR LT IS SK HU MT UK PCT BG US BR PL NL LVELCZ NO EU28 DK DE MK FR LT IS TR PT ES HR EE CH RO AT IT BE SE SE SE SE DE CH DK NL AT FR US EU28 BE UKIS NO ES IT EE LT CZ BGHR HU MT PLPTEL MK TR RO SK BR LV KIABI DE EU28 FR SE UK HU DKNL BE US CH SK CZ MT RO ITAT BR PL EE MK LV PTES NO TR BG HR EL LT IS SK HU MT UK BG US BR SE PL NL LVEL CZ EU28 NO DEDK MK FR LT IS TR PT ES HR EE CH RO AT IT BE SE CH DE DK NL AT US FR BE EU28 IS UK NO IT ES EE MT CZ HU LT HRELTR BG PT LVPL MK BROSK CH IS MT NL SE UK US AT BE DK NO IT EU28 FR DE EL ESEE CZ HR HU LT BG PTLV PL BR SK TR MK RO COMP_EUR SK MT HU UK BG BR US SE PL NL EL NO LV CZ EU28 DK DE FR LTIS MK TR PT ES HR EE CH ROAT IT BE SE CH DE DK NL AT FR US BE EU28 IS NO IT EE ES CZ HR TRPTLT RO LV EL PL BG MK BR CH IS NL SE US BE DK ITAT FR DE NO EU28 EE ES EL CZ HR PT LV LT PLBRBG RO TR MK UK MTHU SK UKMT HU SK DE FREU28 SE UK DK NL HU BE CH US CZ MT ITAT SK RO BR EE PL ES MK LV PT NO TR BG HR EL LTIS DYN(imp) PCT KIABI COMP EU DYN imp N.Obs Min Max Mean Std. Dev Skewne ss Kurtosis Correlation PCT KIABI COMP EUR DYN imp

19 JRC - COIN Statistical Support Developing alternatives Alternative ways of measuring components i.e., DYN PCT Applications per million pop AT US NO FR IS BE UKEU28 AU CA IT CZ MT HU EE ES SK PL LT BG PT RO LV EL HR MK RS MX CL TR CN BR RU ZA CH NL DE DK SE KR PCT Applications per bln GDP(PPS) Year: '2016' (2014); R-squared = 0.961

20 JRC - COIN Statistical Support Robustness analyses when revising components Sensitivity analyses to reveal impact of changing modelling assumptions

21 Comparing scores: IOI vs I Moderate, positive association: I: relatively stronger influence of science and R&D-based innovation activities; IOI: relatively stronger influence of firm dynamics Innovation Output Indicator BG PL LV RO HR SE UK MT NL HU DE DK SK EU FR CZ NO IS AT BE EL IT ES EE PT LT R² = Summary Innovation Index CH

22 Comparing scores: IOI vs Employment Impacts of EIS Strong positive correlation: I: high-growth, innovative firms component receives stronger relative weight Still some differences: consider Sweden, Germany Innovation Output Indicator RO DE DK EU FR SK CZ NO AT PL BG BE EL IT LV EE ES PT HR LT NL SE CH HU R² = Summary Innovation Index: Impacts Employment (4.1) UK MT IS

23 Output vs Input: IOI vs R&D expenditure Moderate, positive association Gives a different picture Caveat: Not all innovation is R&D based; No linear relationship, considerable time lag Yet, high spenders feel not rewarded (i.e. AT) Innovation Output Indicator "2016" SE 120 UK MT NL 110 US DE HU CH EUx DK 100 SK FR NO IS 90 AT BG PL CZ BE LV EL IT 80 ES EE LT RO PT 70 HR TR R² = Gross R&D expenditures per GDP (2015)

24 Results

25 Limitations, debates Structural differences across countries: i.e., what do exports measure? Indicators affected by geography, natural resource endowments (<> smart specialization?) Economic impact of innovation output: where is it realized? transnational companies vs. national borders DYN: conceptually as well as statistically different from the rest of the indicators International comparison limited: different benchmarks Is there a validator: a single measure of innovation? HLP: hourly labour productivity

26 Challenge for innovation indicators Advocacy tool Analytical tool 15+ years of experience (think of I; GII!) Innovation policy (more than technology!) now high on the agenda innovate out of the crisis Advocacy for data improvements Contents of the rest of the reports Data challenges: Established official statistics (i.e., R&D); Improving official statistics: (i.e., CIS surveys, 20 years); Emerging data sources: experimental mainstream /admin, balance-sheet data, big data (i.e., introduction and diffusion of new products, their consumption)/ => make it useful for policy monitoring

27 THANK YOU Welcome to us at: The European Commission s Competence Centre on Composite Indicators and Scoreboards COIN in the EU Science Hub COIN tools are available at:

28 References EC (2010) Elements for the setting-up of headline indicators for innovation in support of the Europe 2020 strategy: Report of the High Level Panel on the measurement of Innovation 2010.