Conclusions of the project Features of PPPs in FPI MUTUAL LEARNING EXERCISE CARLOS MARTÍNEZ BRUSSELS, 19th OCTOBER 2016
Definition
Definition of PPPs in this MLE exercise What they are NOT: Merely project-based collaboration programmes Organisations such as research and technology organisations (RTOs, e.g., the Fraunhofer institutes), They have a market or quasi-market relationship with their customers. PPPs in our sense do not have customers: they involve partners or members.
Definition of PPPs in this MLE exercise Complex policy instruments Involving funding research and innovation activities in partnerships between the knowledge infrastructure i.e. universities and research institutes and producers i.e. mainly companies, but increasingly also state agencies and organisations such as hospitals, welfare services and so on. The companies involved may be public as well as private, on the assumption that the fact that they are organised as companies and operate in markets makes them liable to market- and other failures, which can be addressed through STI policy.
System-level issues
Pre-conditions the existence of potential industrial partners with competencies and absorptive capacity in the fields a well-developed ecosystem the extent of technological development characteristics of a country in terms of social and cultural capital A systems view regards PPPs as part of the overall policy mix of a country risk-taking programme thus a certain degree of failure at project level is to be expected. A well-designed evaluation system for the PPPs can provide feedback to policy learning
Design
Design of PPPs Design refers to all the principles and arrangements of a PPP with the purpose to efficiently and effectively fulfil its mission. It has to take into consideration aspects of subject (what?), actors (who?), functioning (how?) and span (how far, for how long, how big, how much), questions all needed to best address the mission (what for?) derived from the original drivers (why?). Other aspects : Environmental, regulatory or historical constraints (why so?). Design includes basic aspects of implementation and governance.
Design choices Centre PPPs Organised around physical or virtual research centres to extend traditional research and innovation collaboration between the knowledge infrastructure and companies Generate changes in structure and culture within the knowledge infrastructure (mainly universities). Define strategies focused on the work of the centre itself, which is executed by a consortium that works at the centre to varying degrees.
Design choices Network PPPs Focused on a community of firms and research organisations Working in concert on a common, innovation-related agenda devised by a branch of industry, a supply chain or another grouping with common economic, thematic and technological interests. Constructed to identify and address research and innovation needs across the whole community, so members of the community, outsiders or a mixture may execute their research work. There seems to be a trend for network-based PPPs to be on the rise. Examples: Swedish Strategic Innovation Programmes and Societal Challenge Consortia
Design choices Mixed model: Competence Centres (CREST 2008) 1) Strongly based in academia but having industry on their boards Swedish Competence Centres Austrian K+ Centres; 2) Virtual centres such as Some Dutch Leading Technological Institutes; and 3) Dedicated physical Centres IMEC in Belgium, Dutch Telematics Institute which aim to get real depth in expertise and facilities as a core support to the development of industry sectors
OTHER DESIGN ISSUES Top-down vs bottom-up selection of research areas Top-down: Dutch Top Sectors and Finnish SHOKs Bottom-up: Norwegian Centres for Research Based Innovation and Calls for proposals Swedish Strategic Innovation Programmes (the latter, modulated by Gvmt intervention) Consultation process. Modulation of proposals into final projects. Mixed: the Swedish competence centres programme 28 areas. Subsequent calls to fill gaps. Principal-Agent relationship Potential conflict: Norway s Centres for Research-based Innovation Clearly separated roles: Swedish Strategic Innovation Programme
Governance
GOVERNANCE In PPPs, Governance needs to address: a complex set of institutions and actors with a range of differing interests which need to establish and implement a common plan that is in their own interests, and in those of their funder.
GOVERNANCE ISSUES IN A GLANCE Adaptability (i.e. Climate KIC) Transparency & Accountability Efficient governance and management structures Balance between the private and public partners Selection: number and/or quality of proposals Keep the central partners committed Influence of political challenges Efficient administrative procedures Appropriate life-time of an instrument
PPPs evaluation and impacts Erik Arnold Group chairman Technopolis Prof University of Twente Visiting Academic, MIoIR Brussels 19 October 2016
Road map Innovation systems Implications for PPP evaluation Evaluating centre PPPs Evaluating network PPPs The long-term context Methods over time 17
PPPs are structural interventions in National Innovation Systems Demand Consumers (final demand) Producers (intermediate demand) Framework Conditions Financial environment; taxation and incentives; propensity to innovation and entrepreneurship; mobility Industrial System Education and Research System Political System Large companies Professional education, training Government Mature SMEs Intermediary Institutions Research institutes Brokers Higher education and research Governance New, technology-based firms Public sector research RTD policies Infrastructure Banking, venture capital IPR, information services Innovation support Standards and norms Source: Stefan Kuhlmann and Erik Arnold, 2000 18
Some implications of the Innovation Systems approach Business enterprises are central actors in the system Innovation activities are much more than R&D Interconnection and interdependence are at the heart of the innovation system concept Demand, not just supply, drives innovation systems National systems are internationally open Innovation systems develop in specific cultures and have unique histories their institutions co-evolve, so different systems may employ different structures to perform similar functions In policy, balance or mix is key 19
It s a system : things are interdependent Human resource production needs to match societal skill needs Problem orientation, knowledge production and exchange in the research and higher education sectors need orientation to external needs Innovation propensity is partly related to consistent rule of law, IPR regime, competitive regime ( framework conditions ) The thematic focus of the research and higher education sector needs partly to match industry structure and needs The character of consumer demand, including propensity to adopt innovations, affect rates of innovation and FDI Infrastructures need to be adjusted to enable some types of innovation 20
Implications for system-level PPP evaluation Success depends on context as well as the PPP intervention Context-dependence undermines the idea of benchmarking A PPP has to make sense within the wider policy mix Overall indications of the system performance using indicators and other information Take into account the considerations that led to the launch of the evaluated programme or, if it is a long-standing programme, changes in its objectives and expected impacts Consider other programmes that fulfil objectives close to or similar to the evaluated programme Compare programme impacts with close to or similar programmes, if such information is available or can be achieved. Consider potentially hindering factors for performance such as institutional features (a lack of or poorly performing institutions 21
The agenda-setting, system-changing character of PPPs demands a meso-level analysis Analysis of system health Meso-level bottleneck analysis and evaluation Policy development Evaluating programmes and portfolios 22
Centre PPPs: Competence centre goals, internationally Industrially relevant research of a more fundamental kind than is normal in academicindustrial cooperation High-quality scientific outputs, often inter-disciplinary Scientifically qualified people with industrially relevant skills Integrating PhD training into the centres Focusing the skills and experience of academic and industrial R&D workers in the scientific and technological domains of the centres Critical mass or de-fragmentation within academia in areas of industrial relevance Changing research culture Encouraging companies to engage in open innovation and jointly exploring more fundamental questions than normal Encouraging greater interest in and acceptance of the value of industrial collaboration within academia Producing innovations in participating companies and spin-outs 23
Some lessons from the international CC literature Big economic impacts, over extended periods of time Changing research culture in universities and companies Key effects result from integrating and changing education Producing more industrially usable PhD-holders Importance of sweat equity (ERCs) Governance, balance of power are key to success in centres Integrated programmes and centres work best Sort out a fair IPR arrangement then get on with your life the sooner the lawyers are kicked out, the better the centres work Behavioural additionality does not conquer the market failure associated with fundamental research when the high subsidy runs out, the party s over 24
Figure 14 Hypothesised impacts of the Swedish competence centres Universities Changed education Changed research behaviour and organisation New, relevant directions in education and research Other Companies People Knowledge spillovers Consumers Consumer surplus Competence Centres Public goods Knowledge Standards Labour supply Participating Companies People Intermediate knowledge products Behavioural changes Networks Innovations, IPR, etc Income 25
Sweden example: Groups of competence centre impact on companies Direct impacts on industry, through generating directly usable outputs Direct impacts through behavioural additionality, including creation of knowledge networks Economic impacts on participants Economic development of individual SMEs participating in CCs Indirect effects through adding to the firms stock of internal resources Spillovers Indirect effects, via the university system 26
Not much evaluation of Network PPPs has been done yet Overall goals are generally competitiveness and innovation-related, sometimes in the context of a grand challenge Agenda-setting, community reinforcement and other network goals are important reform or restructuring of individual institutions is not (unlike in centre PPPs) Evaluative attention is currently on strategy, start-up activities and learning Evaluation findings therefore tend to focus on quality-assurance, and validating and improving the organisational models used to run the PPPs 27
PPP evaluation issues need eventually to be considered in a long-term context Impacts Short- and long-term innovation impact Economic impact Impact on institutions, capacity Effects on education Knowledge Value Collectives Agenda setting, focusing devices Other issues Governance Learning curve Inherent conservatism of the instrument Programming Top-down, bottom-up Role in instrument portfolio 28
PPPs long lives and restructuring tasks need to be reflected in evaluation: from formative to summative 1. Early: is this PPP working? 1. Does it conform to the programme model? 2. Does it have the right governance and processes in place? 3. Is it equipped to produce and maintain quality 4. Does it appear to be sustainable? 2. Growing: is it beginning to produce good work, relevant to the stakeholders and with potential for wider impact? Is it setting new agendas, as intended? Governance? 3. Maturing: is it beginning to have visible impact beyond the stakeholder group while maintaining quality? Governance? 4. Late in life: what has it achieved (outputs, outcomes, impact)? At a good quality level? Succession/continuation? Governance? 29
Diversity of methods is necessary. Methodological focus should shift during the life cycle Intervention logic Peer review: quality; strategy Document analysis Surveys, interviews with participants and stakeholders Scientometrics Economic impact analysis Little versus big firms what is really feasible? Human capital analysis, Knowledge Value Collectives Social Network Analysis Long-term tracing (backwards and forwards) 30
Thank you technopolis group has offices in Amsterdam, Brighton, Brussels, Frankfurt/Main, Paris, Stockholm, Tallinn and Vienna 31