Evaluation in Management of INTERREG Programmes. > dr Karol Olejniczak > Summary of the PhD results

Transcription

1 Evaluation in Management of INTERREG Programmes > dr Karol Olejniczak > Summary of the PhD results

2 Research rationality Background Evaluation key elements of all performanceoriented reforms of public management Evaluation standard of modern public management Evaluation legal requirement of all interventions co-financed by EU Since 1990-ies Evaluation Boom in Europe Few research on evaluation utilization in EU programmes In Europe, no systemic studies of functions and impact of evaluations Main research problem What is the real role of the evaluation reports in the management of EU programmes?

3 Objective and research questions Objective of the research Investigate the real impact of the performed evaluation studies on INTERREG management Main hypothesis High quality reports were useful in INTERREG management both for day-to-day management and preparation for next generation of programmes Supporting Hypotheses Forms of evaluation use: improving planning, management, accountability, learning, partnership & ownership Factors determining use: context, process, products characteristics

4 Community Initiative INTERREG Pan-European character International management system Interactions & cooperation Continuity & increasing importance Mid-term evaluation of programmes Contact with stakeholders still possible Evaluations executed for all programmes (same time, same scope) Long-term impact possible to trace Subject of the research Out of 69 mid-term reports 10 top quality reports selected (based on: BACHTLER et. al. 2005) 6 evaluation reports of the highest quality 3 case studies

5 research phases PHASE 1: Building meta-model Literature review PHASE 2: Case studies analysis Analysis of INTERREG reports PHASE 3: Conclusions Verification of the hypotheses Choice of case studies EU documents review Review of the context documents Construction of a metamodel Analysis of the reports and follow-up studies General conclusions on public management paradigms Conclusions on the analytical model Interviews with Contracting Authorities Preparing toolbox for case studies Interviews with evaluators Interviews with EC & INTERACT Secretariat Practical suggestions for CA & evaluators

6 on research hypotheses INPUT EVALUATION PROCESS PRODUCTS EVALUATION UTILIZATION FUNCTIONS High quality of the report Positive use short-term Improved planning Programmes situation Political context Philosophy of evaluators Participative approach Evaluators role animators & educators Trust between CA & evaluators Stakeholders involvement Technical know-how Revealing knowledge Wider explanatory knowledge Positive use long-term Pseudo-use Improved management Strengthen accountability Improved learning Dissemination strategy Misuse Strengthen partnership & ownership

7 related to public management paradigms Studies followed Managerialism paradigm Used by programme operators Bringing technical knowledge Making managers observations credible Analyzing simple indicators New paradigm emerges (New Public Gov.) Involving different actors and building network but methodological problems arises When public institutions learn When process is interactive When there is an institutional continuity When learners know what they want Evaluators role Fresh look (new angle) at the old issues Showing new perspective / including new voices Putting issues in a wider context Animator Advisor Educator

8 Practical suggestions For Contracting Authorities Awareness of the trade-off Management vs. Accountability The usefulness of continuous evaluation Clear expectations What counts is a process the dialogue, reporting can me less formal For Evaluators Participative approach is useful in mid-term i on-going evaluations Two way process - learn from prog. operators Take active role (animator / educator) Start with working out common definitions Keep the distance (and objectivity)

9 Research co-financed by Polish Ministry of Education and Science in the framework of PhD grant Study visit and publication co-finaced by the European Union - Transition Facility PL2005/ Publication in Polish: K.Olejniczak Mechanizmy wykorzystania ewaluacji. Studium ewaluacji średniookresowych INTERREG III, Scholar, Warszawa 2008 avaliable for download at: 9