The Impact Oriented Evaluation Approach of CEval EASY ECO 2006 Conference Improving the Quality of Sustainable Development Programs and Projects AG II 3 M+E-Approaches Prof. Dr. Reinhard Stockmann Director CEval
Concepts Input: Input of human and material resources, capital expenditures, costs e.g. expenditures in manpower, expenditures in Euro Output: Service, product of program activities e.g. number of treated invalids number of newly hired handicapped persons number of conducted environmental consultations Impact: Impacts, results of the work performed e.g. improvements of the state of health changed attitude towards handicapped persons improved ecological situation Reinhard Stockmann, Oktober 2006 2
Impact Dimensions Impact dimension Structure Process Behaviour intended +- +- +- Not intended +- +- +- Legend: + = goal consistent impact - = non consistent impact Reinhard Stockmann, Oktober 2006 3
Theoretical Conception of the CEval Approach Life-cycle Model Organizational Conception Conception of innovation and diffusion Multi-dimensional concpetion fo sustainability Evaluation Manual Reinhard Stockmann, Oktober 2006 4
Life-Course Model Temporal axis t1 t2 t3 t4 t5a - t5n tf t6 tnf Phases Project applic. Project appraisal Imple. order agreement Phases of project implementation Project hand-over End of assistance Follow-up Phase State of project after 3-5 years Planning Implementation Impact/ Sustainability Project-assistance phase Post-assistance phase t tf tnf = Point in time = End of project assistance = Following end of assistance Reinhard Stockmann, Oktober 2006 5
Impact Model System of education and training Environment Implementing agency Organizational structure System of employment Political system Goals Project Members Legal system Technology Financial resources Sociocultural system Object of analysis Reinhard Stockmann, Oktober 2006 6
Model of Diffusion External variables: Environmental factors Specific features of innovaton - Relative advantageousness - Compatibility - Complexity - Testability - Observability - Degree of design perfection - Political system - Norms and values - Labor market (demand) - System of education and (vocational) training (suppliers) - Other Diffusion: Internal variables Organizational elements -Goals - Members - Formal structure - Technology - Financial resources Dissemination of an innovation Source: As adapted from Mohr 1977: 43 Reinhard Stockmann, Oktober 2006 7
Sustainability of the program level Dimension I Type Project/ Programme oriented Attribute The target group and/or organisational medium continues using the innovations to serve their own interests and for their benefit. II output-/ performance oriented Other groups/organisations have adopted the innovations on a long-term basis to serve their own interests and their own benefit. III System oriented The innovations lead, via processes of diffusion, to an increase in the performance of the system as a whole (e.g. the health or education system). IV Innovation oriented The target group/organisational medium has a potential for innovation that enables it to react flexibly and appropriately to changed environmental conditions. Reinhard Stockmann, Oktober 2006 8
Deduction of the Evaluation Manual Life-Course Model Planning Implementation Organization- Model Sustainability: External Variables Model of Innovation and Diffusion Internal Variables -Macro- Level -Program- Level Model of Sustainability Reinhard Stockmann, Oktober 2006 9
Themes of Evaluation 1. Program and environment 2. Planning and implementation 3. Internal impacts 4. External impacts 5. Sustainability Reinhard Stockmann, Oktober 2006 10
Fields of Internal Impact Goals and their acceptance Personell Organizational Structure Financial Ressources Technology Technical Infrastructure Organizational program/ -conception Reinhard Stockmann, Oktober 2006 11
Fields of External Impact Acceptance of Goals in the Target Group Reaching of Target Groups Benefit for the Target Groups Target Group overlapping Impacts Impacts in the Field of Policy / Pracitcal Field Policy overlapping Impacts Reinhard Stockmann, Oktober 2006 12
Thank you for your attention! Prof. Dr. Reinhard Stockmann