Health promotion for disadvantaged youth. Presentation

Transcription

1 Health promotion for disadvantaged youth WP 3 Evaluation Quality Management Concept Presentation Max Saurug Alfred Strassegger Ancona,

2 Health 25 Introducing IFA Steiermark Our Organisation IFA Steiermark, seated in Graz, Austria (5 employees) was founded in IFA is a research institution working for the common good of society. Our organisation is aimed at assisting politically responsible people, decision makers in public corporations and private companies, persons involved in projects, special interest groups and so on. Research = Accompaniment + Optimisation

3 Health 25 Introducing IFA Steiermark Our Services Evaluations Short term analyses Consultation and development Our work includes questions concerning labour market and social policy, regional development, vocational education for companies and personnel and the integration of people with special needs into the labour market.

4 Health 25 Introducing IFA Steiermark Our Experience Over many years, IFA has acquired a vast wealth of experience in the planning, conception and undertaking of practical scientific investigations and in accompanying consultations about programs and measures in the social field. Through numerous all embracing national and transnational research and development projects conducted throughout Europe (more than 100), IFA is able to fall back on a wealth of both national and international experience. To learn more: steiermark.at

5 Health 25 WP 3: Evaluation 1. Quality Management Concept Quality Management Concept This quality management concept will give you detailed description of the quality management and an overview of the evaluation strategies in regard to the quality management of the project process, the project product (outputs and outcomes) and the project impact. In respect of the assessment of the quality of the Health 25 project the quality indicator identification process and its outcomes should be started as one of the next steps. from

6 Health 25 WP 3: Evaluation Quality Management Concept The objectives # 1 Literature review and survey # 2 Development of the health training and intervention programme for the target group # 3 Development of pedagogical psychological concept for the target group # 4 Synthesis of the holistic health training and intervention programme for year old people without a job and not in school education # 5 Pilot implementation

7 Health 25 WP 3: Evaluation Quality Management Concept Overview Criteria Based Evaluation For a successful venture the quality management approach has to rely on specific quality criteria. This concept relies on the following characteristics: Scientifically an methodologically correct Transparent Flexible Realistic Possible to control Targeted

8 Health 25 WP 3: Evaluation Quality Management Concept Overview Description of the model The quality management concept is based on a triangulate evaluation approach. In the frame of this approach there are three different dimensions to focus on during the evaluation: Project process Project products Project impact Only by taking into account these three dimensions the project quality can be estimated in a holistic approach. The model contains so called Critical Success Factors (CSF s) for all three dimensions, which are partly given in advance, but mostly tailored to the project.

9 Health 25 WP 3: Evaluation Quality Management Concept Overview Description of the model This triangulate Quality Management Approach implies that all three quality dimensions interdependently interfere, which allows to ensure quality through a holistic approach. Furthermore, this method is absolutely transparent and the evaluation data is accessible anytime to all the project partners. The quality management method estimates the actual status quo by covering the targets set; it recognizes deviances and suggests possible steering measures, which allows to keep the project on track from the start and to reach the project quality proposed.

10 Health 25 WP 3: Evaluation Quality Management Concept Overview QM of the project process One of the components of this quality management strategy involves the monitoring of the project in terms of its processes. In this context the following processes will be observed: Communication among partners Communication between the contractor and the partners The level of information and awareness of the partners towards the contents of the project (also goals, schedule and budget) The degree of involvement of the different partners in the project activities

11 Health 25 WP 3: Evaluation Quality Management Concept Overview QM of the project process The reports serves as monitoring tool for the coordinator as its part of the quality Management and Evaluation Strategy. The questionnaire developed focuses on the following areas: Processes during the meeting Organisation of the meeting Suggestions, recommendations and personal comments Project progress reports Implementation of the peer evaluation system

12 Health 25 WP 3: Evaluation Quality Management Concept Overview QM of the project process Criteria collection for the product evaluation Methods and instruments for the evaluation

13 Health 25 WP 3: Evaluation Quality Management Concept QM of the project products Criteria collection (CSF s) In the process of ensuring the quality of the Health 25 project all participating partners will be assigned the identification of applicable quality indicators and subsequently asked to rate these items. The partner responsible for the WP3 Evaluation of the project will collect and analyse the data provided by all organisations involved in the Health 25 project. Stage 1 Initial Phase Definition of the CSF s Stage 2 Rating the elaborated items Stage 3 Processing the contributions

14 Health 25 WP 3: Evaluation Quality Management Concept QM of the project products Evaluation methods and instruments The proposed instruments used for the evaluation of the Health 25 project are based on the experience of the evaluator in the area of quality management. In order to assess the individual CSF s of the Health 25 project it is recommended to use (mainly) questionnaires and guided Interviews (if reasonable and necessary) as methods most suitable for the evaluation purposes. The use of questionnaires along guided interviews within a partner meeting is intended to produce a combination of qualitative and quantitative information which should support the evaluation of the Health 25 project in respect of the work process and the product itself.

15 WP 3: Evaluation of Health 25 The Quality Management Concept Overview QM of the project impact Similar to the evaluation of the product quality, a method of impact evaluation is recommended; where in some case partly standardised CSF s may be useable. These CSF s will be developed similar to those in the product quality involving all partners and stakeholders. It is suggested to evaluate the impact of the product on representatives of the target groups in the partner countries at the end of the project; the Health 25 project can also be assessed in respect to its short term impact. There could/should be mentioned measurement criteria like physical measurements (baseline and at the end of the training programme) in relation to the Health Training and Intervention programme as well as (set and fulfilled) expectations, knowledge, skills, attitudes and behaviour.

16 Thank you for your attention

17 Health 25 WP 3: Evaluation Evaluation Basic approach Objective of the planned research project is the evaluation of the cooperation Health 25 on a transnational level. The central tasks of the research assignment are to analyse the implementation process and the attainment of the common set objectives.

18 Health 25 WP 3: Evaluation Levels of analysis and key questions Level 1: This concerns outputs of work packages and the Health 25 project as a whole: Specifying tasks and duties What are the central common aims of Health 25? What is each member is in charge of? How do the partners describe their roles within the transnational partnership? What are the expectations? What are the duties? Level 2: Specifying the implementation steps: What working steps, methods and tools are to be used to attain the common objectives? Level 3: Analysing the implementation process: What are the general conditions and factors that can promote and impede the cooperation process?

19 Health 25 WP 3: Evaluation Levels of analysis and key questions Level 4: Analysing goal attainment: To what extent were the set objectives (outputs/outcomes) attained? Could the expected impact on the target groups be reached? Level 5: SWOT analysis and analysis of lessons learned : What are the strengths and the weaknesses of this transnational cooperation? What is the potential for improvement? Are there any recommendations for future partnerships? In addition the project partners of the participating countries are to be assisted in evaluating their project activities (within their work packages) themselves by putting at their disposal research instruments developed for this evaluation assignment. The results from this self assessment are to be made available to the evaluator and incorporated into the evaluation as a whole. Special task: A Quality Management Handbook has already been worked out (and also distributed to the partners)

20 Health 25 WP 3: Evaluation Duration, methods and working steps 36 months (February 2011 January 2014) Both qualitative and quantitative instruments are to be used for this accompanying evaluation. Internetbased questionnaires for quantitative and partly standardised surveys are to be used.

21 Health 25 WP 3: Evaluation Supporting the lead partner and reporting Periodic written reports about the progress of Health 25 are to be presented at every transnational meeting. The evaluator is a member of the steering committee and is to support the project lead partner BBRZ Austria. The evaluator is obliged to attend all transnational meetings of Health 25. Representatives of the transnational project partnership and the lead partner are to be kept regularly upto date orally and by periodic short reports after the meetings. These reports ensure that the evaluation results can be used as a basis for project management. Written reports are to be submitted every six months as well as the final results of the evaluation are to be drawn up in a final research report.

22 Health 25 WP 3: Evaluation Working Steps Elaboration the the Quality Management Handbook (June 2011) Presentation of of the the evaluation concept at at the the transnational meeting in in Ancona/It (September 2011) Development of of questionnaires for for self assessment of of the the work packages of of the the project partners (October/November 2011) Development of of an an internet based questionnaire in in order to to evaluate the the progress of of target attainment and and the the implementation process of of the the project partners and and the the project Health as as a whole (October/November 2011) Evaluating and and reporting the the self assessment of of the the project partners in in the the final evaluation report Periodic evaluation of of the the progress of of the the project including short evaluation reports after every transnational meeting (including first first SWOT analyses) Participating observation at at all all transnational meetings; 6 qualitative interviews with leading representatives of of the the partner organisations (Ongoing) Participating in in the the 1st 1st working group meetings of of WP WP 5, 5, WP WP 6, 6, WP WP 8 Drawing up up the the Final Report January 2014 Presentation of of the the final results January 2014

23 Thank you for your attention