Impact evaluation of Interreg IVC projects

Transcription

1 European Union European Regional Development Fund Sharing solutions for better regional policies Impact evaluation of Interreg IVC projects Cahier des Charges Terms of Reference IVC evaluation 1 / 9

2 Contents 1/ Context of the Call for Tenders / Objective of the tender and target audience / Definition of needs Presentation of the tender Methodology and data Provisional timing and deliverables Contractor team composition and required competences / Project management / Confidentiality and intellectual property rights... 8 Intellectual Property Rights / Selection _ToR 2 / 9

3 1/ Context of the Call for Tenders The Interreg Europe programme is part of the European Territorial Cooperation goal of EU cohesion policy for the programming period 1. It is the successor to the INTERREG IVC programme (period ). The overall objective of the Interreg Europe programme is to improve the implementation of policies and programmes for regional development, principally of programmes under the Investment for Growth and Jobs goal and, where relevant, of programmes under the European Territorial Cooperation goal, by promoting exchange of experience and policy learning among actors of regional relevance. Interreg Europe is a pan-european cooperation programme that covers the entire territory of the European Union with its 28 EU Member States, including their insular and outermost areas, as well as Norway and Switzerland. Partners from other countries can participate at their own cost. Like for INTERREG IVC, public authorities (at national, regional or local level) and institutions governed by public law (e.g. regional development agencies, business support organisations, universities ) are eligible to receive ERDF or Norwegian funding in Interreg Europe projects. In addition to these, private non-profit bodies are eligible for funding only under Interreg Europe. The programme is financed by the European Regional Development Fund (ERDF). The managing authority (MA) of the programme is the Hauts-de-France Region, located in Lille/France. The joint secretariat (JS) is also located in Lille/France and hosted by the GEIE GECOTTI-PE, contracting authority of this tender. 2/ Objective of the tender and target audience With the discussions on the future of the Cohesion Policy underway and the programming of the future period for the Cohesion Funds starting in early 2018, the programme sees the need to collect objective evidence that the interventions funded by the programme at present bring positive change to regions participating in them. Interventions funded by the programme s predecessor, INTERREG IVC represent a great source for collecting evidence of territorial impact of interregional cooperation projects. This evidence already exists. If results were achieved early enough during the projects implementation (before 2014), the programme could be informed during the funding period of some territorial impact. This is for instance the case in the PIMMS TRANSFER project where the Walk to School good practice (encouraging pupils and their parents to walk to school instead of using the car) was transferred from Bromley (UK) to Frankfurt (DE). 15 schools finally adopted the practice in Frankfurt and, through the participation of 3,000 pupils, the CO2 emissions saved through this initiative were estimated to 40 tonnes. However, even if the vast majority of INTERREG IVC projects reported policy changes, the territorial impact of these changes occurs after the funding period and the programme therefore does not have information about that impact. A concrete illustration can be provided from the same PIMMS TRANSFER project. Based on the experience of the City of Graz (AT), the City of Maribor (SI) modified its policy for parking in the city center (extension of paying parking areas for short stay duration, revised parking price system). This policy change was reported as an achievement to the INTERREG IVC programme since it occurred during the funding period 1 The Cooperation Programme was adopted by the European Commission on the 16 June _ToR 3 / 9

4 of the programme. However, the possible territorial impact of this change (evolution in the use of cars, reduces traffic and CO2 emissions), which occurred after, was not monitored by the programme. It is sometimes only by chance that the programme is informed about such impact. This is the case in the MINI EUROPE project where a good practice called Summer Entrepreneur was transferred from Mid Sweden to Flevoland region in the Netherlands in Through a discussion with the project lead partner in a conference in September 2017, the programme was informed that the Summer Entrepreneur became a regular initiative organised in 11 regions in the Netherlands with a thousand of young people participating in the entrepreneurship coaching and training in the past years and around 20% of them registering with their local chamber of commerce as young entrepreneurs. The present evaluation should result in a more systematic collection of the available evidence of territorial impact attributable to INTERREG IVC projects and illustrate how the exchange of good practice and capacity building contribute to positive changes on the ground in regions participating in interregional cooperation projects. Moreover, the evaluation will potentially contribute to evaluation methodology of the Interreg Europe programme in particular by investigation ways to assess territorial impact (phase 2 of Interreg Europe projects). The aims of this tender is to get external expert support in: designing the impact evaluation methodology; carrying out the evaluation; and validating the results of the evaluation The target groups of this evaluation are primarily the influencers of the programme: European Commission and its geographical units European Council European Parliament Committee of the Regions Partner state points of contact and members of the monitoring committee Elected officials in the programme partner states Regional offices in Brussels In addition to those target groups, the evaluation results will be made publicly available to programme beneficiaries (lead partners and project partners of supported projects, participants in any PLP activities) and to the general public through a public executive summary. 3/ Definition of needs 3.1 Presentation of the tender The services included in this tender are organised around five tasks. Provisional budget estimated for all tasks including the option (Task 1.4 below): EUR External expert advice, training and validation The joint secretariat will design the evaluation methodology, prepare initial data and carry out the impact evaluation with support of an external expert. The focus of this tender is on the following expert support tasks: Task 1.1 Review and validate the evaluation methodology _ToR 4 / 9

5 The expert will support the joint secretariat in designing the methodological steps of the evaluation. S/he will support the joint secretariat in selecting and designing the evaluation tools such as an online survey, an in-depth phone interview questionnaire and final validation focus group. The expert will advise the joint secretariat on the use of quantitative and qualitative methods for each evaluation step. S/he will support the joint secretariat in selecting the cases for in-depth phone interviews. Task 1.2 Review and validate the results of each evaluation step The expert will check the results of analyses carried out by the joint secretariat in each evaluation step and validate it for its correctness and objectivity. An online survey among the full sample agreed on for the evaluation will be carried out by the expert and analysis of the survey results will feed into the selection of the cases for the second evaluation step the in-depth phone interviews. (min 20) Task 1.3 Arrange interviewers training and check data quality The expert will arrange a training on in-depth interviewing for the participating staff of the joint secretariat. The training will address issues such as how to record and code the information gathered during the interviews and how to avoid biased replies. S/he will ensure high quality of data collection and coding coming from the conducted interviews. Task 1.4 Run the validation focus group (Option) The expert will supporting the joint secretariat in designing the methodology for the validation focus group and in selecting the participants. S/he will then run the meeting and take the meeting s outcomes into account when carrying out Task 1.5. Task 1.5 Review and validate the final evaluation report The expert will support the joint secretariat in preparing the evaluation report. S/he will check and validate the report s findings for their correctness and objectivity. 3.2 Methodology and data Methodology The work to be carried out within the framework of the contract will cover the whole programme area (i.e. 28 EU Member States, Switzerland and Norway) and selected interventions implemented by the INTERREG IVC programme. Data The extent of the study: Initial online survey among all INTERREG IVC project partners involved in topics relevant to Interreg Europe (seven topics - except Employment, Information Society, and Natural and technological risks). (around 1500 partners) Phone interviews with partners based on the survey results: selected per kind of reported improvement and, if relevant, per country (min 20) 3.3 Provisional timing and deliverables This tender is open for bids until 15 December 2017 (5 pm, CET/ Paris time). The work is expected to start the week after. The work will have to be completed by mid-april The exact starting date will be communicated to the external expert once s/he is notified. mid-december January February March Mid-April _ToR 5 / 9

6 Methodology check Survey check/ Online survey Survey analysis/ Questionnaire check/ Interviewers training/ Phone interviews Phone interviews/ Interview analysis check/ Focus group agenda and meeting Final report check Table 1. Provisional timing to carry out the evaluation tasks. The timing in this table as well as deadlines for deliverables below are subject to changes depending among other things on the notification date. Deliverables - Deliverable 1 Methodology check Deadline: 22 December 2017 Once notified, the expert will receive a methodology note explaining the approach to evaluating the impact of the INTERREG IVC projects. The note will describe the tools selected and analyses foreseen to answer a set of evaluation questions. A draft online survey as well as a draft of questionnaire for in-depth phone interviews will be part of the methodological note to be reviewed by the expert. Suggestions for changes to ensure high quality and unbiased approach to the evaluation will be provided by the expert. A list of key changes needed in methodology will be provided by the expert as a deliverable. Other comments and adjustments will be done directly in the documents provided in an electronic form. Max number of pages: 4 - Deliverable 2 Methodology tools Deadlines: 5 January online survey 5 February phone interview questionnaire and coding guidance 16 March focus group agenda (Option) Finalised version of the agreed-on methodology tools (such as online survey, phone interview questionnaire, coding guidance, and focus group agenda) will be validated by the expert before their use in the online survey, during the phone interviews and the focus group meeting. - Deliverable 3 Online survey Deadline: 4 February 2018 The expert will carry out an online survey on the full sample agreed on during the methodology check. The content of the survey will be finalised with the joint secretariat. The data will be collected using an online platform such as surveymonkey. The survey will be open for four weeks. At least one reminder will be sent one week before the closure date. The expert will analyse the data collected in the survey and feed the findings into the phone interview questionnaire design (for example categories of questions/ answers) and case selection for the phone interviews. S/he will ensure representative case selection in terms of geography, topics and variety of territorial impacts identified by the online survey. An aggregate data report with charts and tables generated by the online tool will be submitted to the joint secretariat together with a brief additional report highlighting the survey findings and emphasising the needed adaptation in the consecutive evaluation step. Recommendations for any future similar evaluation surveys will also be part of the additional report. Max number of pages of additional report: 10 - Deliverable 4 Interviewers training Deadline: 9 February 2018 The expert will arrange a training for the selected joint secretariat staff on the in-depth phone interviewing method in order for them to carry out the interviews collecting high quality data for further analysis. The expert will brief the interviewers on the use of the questionnaire (Deliverable 2) and provide interview recording and coding guidance. Agenda for the training as well as a list of recommended reading/ material will be provided by the expert _ToR 6 / 9

7 - Deliverable 5 Phone interviews Deadline: 9 March 2018 The joint secretariat interviewers will carry out a set of in-depth phone interviews based on a standardised questionnaire (Deliverable 2). The expert will check the quality of how the interviews are carried out and recorded, and provide feedback to the interviewer(s) if adjustment is needed. The expert will draft a brief validation report on the interviewers check. Max number of pages: 2 pages - Deliverable 6 Analysis check interviews Deadline: 19 March 2018 The expert will check and validate the results of the interview analysis and the conclusions drawn from the findings. S/he will ensure unbiased and objective commentary on the findings as well as the representativeness of the cases selected for the evaluation report. Comments and adjustments will be done directly in the interview analysis provided in an electronic form by the joint secretariat. - Deliverable 7 Validation focus group (Option) Deadline: 23 March 2018 The expert will run a focus group meeting with participants selected together with the joint secretariat (max 15). Representatives of different programme stakeholder groups should be represented. A draft evaluation report will be shared with the participants before the meeting. The focus group participants will discuss the impact analysis and findings to be presented to the target group of this evaluation, as well as the recommendations for future impact evaluations (methodology and indicators). The meeting will take place in Lille. - Deliverable 8 Evaluation report check Deadline: 31 March 2018 The expert will check and validate the conclusions presented in the evaluation report as well as the case studies presented. S/he will ensure unbiased and objective commentary on the findings as well as the representativeness of the cases selected. Comments and adjustments will be done directly in the interview analysis provided in an electronic form by the joint secretariat. In the context of Deliverables requiring travels, it is important to keep in mind that travel and accommodation costs have to be included in the proposed price. A physical meeting may be called ad hoc to discuss any serious problems fulfilling any of the tasks. 3.5 Contractor team composition and required competences The contract will be awarded to an individual expert or a group of experts. The bidder will introduce the composition of the expert team as part of the methodological note. Personal CV(s) of the expert(s) to be involved in the expert advice, training and validation services will be included in the methodological note together with a list of previous relevant work. In case of a group of experts, one needs to be identified as a senior expert and the team leader. The expert(s) will have to demonstrate expertise in the following fields: - project/ programme analysis and impact evaluation - data collection and analysis, in particular in online surveys, phone interviews and focus groups - knowledge of European funds and regional policies (especially ERDF and ETC programmes) _ToR 7 / 9

8 Knowledge of the thematic fields of the programme (research and innovation, competitiveness of SMEs, low-carbon economy, environment and resource efficiency) will be an advantage. The expert(s) must demonstrate proficiency in English language (spoken and written). The expert(s) must also confirm the absence of conflict of interest that could prevent them from carrying out the services independently. The CV(s) shall be in English and use the European format (provided for under the following link): (max 4 pages) Any change in the expert team (including its number or composition) shall be immediately notified and approved by the contracting authority before starting the execution of any service. 4/ Project management The service provider shall ensure solid and robust project management, enabling its own project manager and the joint secretariat to view easily the work progress. The service provider shall inform the joint secretariat of any major challenges or activities requiring discussions with the joint secretariat or its approval. The service provider will be required to follow the practice of: - Early agreements with a detailed timetable and milestones - Regular meetings between the joint secretariat and the service provider, including physical meetings (at joint secretariat premises in Lille) and/ or online meetings - Formal recording of all agreed changes to the evaluation process, including the methods applied, modifications of timetable and human resources involved in carrying out individual tasks, including time allocated to each task. The joint secretariat may, at any time, request a fast answer to a specific point by . In this case an answer must be provided within two working days. 5/ Confidentiality and intellectual property rights Confidentiality The contractor will have to observe complete confidentiality of the information and documents brought to their attention during the evaluation process. Intellectual Property Rights The option B of the French CCAG PI (i.e. exclusive transfer of exploitation rights) applies to this tender. 6/ Selection This tender is subject to approval by the programme s monitoring committee which takes place in mid- December The tender can be cancelled, before the official notification of allocation if the monitoring committee decides so. The services provided by the expert(s) will have to be in English, which is the working language of Interreg Europe _ToR 8 / 9

9 After analysis, the contracting authority will allocate the project to the tenderer offering the best value for money. The examination of tenders will give rise to a classification of bids. The top ranking tender, that is the one offering best value for money, will be selected. In this respect, all offers will be assessed on the basis of criteria set out below: Technical quality of the proposal: Criteria Weight in % Relevance of the methodological note with regard to the tasks described in article 3.1 Relevance of task allocation (to one expert only or to different experts in a group) Quality of the CV(s) of the expert(s) involved: Excellent command of written and spoken English (C1-C2 proficiency level required). Experience in the fields of: project/ programme analysis and impact evaluation - data collection and analysis - in-depth knowledge of European funds and regional policies, especially ERDF and ETC programmes Proposal writing and management of Interreg Europe projects, programme evaluation (ex-ante, mid-term, ex-post) and the knowledge of the thematic fields of the programme (research and innovation, competitiveness of SMEs, low-carbon economy, environment and resource efficiency) will be an advantage. Price in EUR 20 The evaluators will have to demonstrate expertise in the following fields: - project/programme analysis and impact evaluation - data collection and analysis, in particular in online surveys, phone interviews and focus groups - knowledge of European funds and regional policies (especially ERDF and ETC programmes) - knowledge of the thematic fields of the programme (research and innovation, competitiveness of SMEs, low-carbon economy, environment and resource efficiency) will be an advantage. Bidders should include with their offer a timeline for delivering the expected services (including details about hours/ days to be spent on each task and budget foreseen). Travel and accommodation costs should be part of the budget. The deadline for submitting the offers is 15 December 2017 (5 pm, CET/ Paris time). The bids has to be sent by to communication@interregeurope.eu _ToR 9 / 9