Measuring the Process and Impact of Children and Young People s Participation. Mererid Lewis, Emyr Williams and Stacey Jones Participation Unit

Transcription

1 Measuring the Process and Impact of Children and Young People s Participation Mererid Lewis, Emyr Williams and Stacey Jones Participation Unit

2 Aims of Presentation To raise awareness of the National Children and Young People s Participation Standards for Wales. To look at how organisations can move forward from measuring the process of children and young peoples participation to measuring the impact.

3 Guesstimates!! Trial Question- How many people had a take-away on the weekend?

4 1.How many people know how they could go about doing a self-assessment?

5 2. How many people in this workshop have registered on the Standards Selfassessment website?

6 3. How many people have completed a Self-assessment?

7 Measuring the Process - National Standards for Participation These are a good quality process to help measure and promote the participation of children and young people in making decisions, planning and reviewing any action that will affect them. 1. Information- which is easy for children and young people to understand 2. It s Your Choice enough information and time to make an informed choice 3. No Discrimination - every young person has the same chance to participate. 4. Respect- Your opinion will be taken seriously 5. You get something out of it- You will get something from the experience 6. Feedback- You will find out what difference your opinion has made 7. Improving how we work- Adults will ask you how they can improve how they work for the future

8 Measuring the Impact- What s Changed?

9 Impact Assessment Project: Background Standards and Kite-marking Measuring Outcomes A Common Framework Links to other developments: WAG Outcomes Framework, RBA and Demonstrating Success

10 A number of challenges need to be addressed: Currently No agreed indicators against which to measure effective child participation There is a need to construct universally applicable indicators in order to be able to compare initiatives in terms of the outcomes and impact they achieve. Children themselves have a role to play in determining the objectives in any initiatives in which they are involved. The outcomes of participation are often qualitative not quantitative and therefore hard to measure Long-term changes in children s lives which cannot be measured within a short time period.

11 No simple answers exist to these challenges. However, Standards and Tools with which to measure child participation will: Help define the legislative and policy environment needed to promote and respect children s right to participation Clarify who are the duty bearers and their responsibilities Enable children to gain a greater understanding of what they hope to achieve. Help identify what support and resources are needed to strengthen child participation. Provide evidence to support the case for political commitment to the realisation of children s participation rights. Encourage donors to see the benefits of investment in strategies to promote child participation.

12 Impact Assessment Project Who? Participation Consortium (Standards subgroup) Anne Crowley,Cardiff University Other inputs NYA, Save the Children

13 Outcomes, Indicators & Performance Measures Population Outcome is a condition of well-being for children, adults, families or communities in plain language Children born healthy Indicator is a piece of measurable information which helps quantify achievement of an Outcome Rate of low birthweight babies Performance Performance Measure is a measure of how well an individual service etc is working 3 types 1. How much did we do? 2. How well did we do it? 3. Is anyone better off? = customer results

14 Performance Measures (RBA model) Quantity Quality Effect Effort How much did we do? Is anyone better off? How well did we do it? # %

15 Types of Measures Found in Each Quadrant How much did we do? # Clients/customers served # Activities (by type of activity) # # # # Is anyone better off? How well did we do it? % Common measures e.g. client staff ratio, workload ratio, staff turnover rate, staff morale, % staff fully trained, % clients seen in their own language, worker safety, unit cost % Activity-specific measures e.g. % timely, % clients completing activity, % correct and complete, % meeting standard % Skills / Knowledge (e.g. parenting skills) % Attitude / Opinion (e.g. toward drugs) % Behavior (e.g.school attendance) % Circumstance (e.g. working, in stable housing)

16 Impact Assessment Project Three dimensions of change: What? FRAMEWORK changes to policy and services; changes to social and power relations and changes to the children or young people s personal development and well-being. Other dimensions of change could be added depending on the objective of the participation activity, e.g. parents, community.

17 Impact Assessment Project What? FRAMEWORK For each dimension of change a small number of indicators and suggested monitoring arrangements

18 STEP 1 Purpose and Measuring Success The first step is to decide on the participation activity/structure/mechanism that you want to evaluate the impact of and to document the purpose(s) of this example of children or young people s participation. Based on the purposes you have identified look through the Impact Assessment Framework and decide on the most appropriate indicators to be used and add others that are distinctive to the participation you are evaluating.

19 For example: if one of the purposes identified is: to increase children or young people s rights awareness then select indicators from the framework (under Dimension B.). If one of the purposes is: to enable children or young people to have a voice in shaping services, then some or all of the indicators under Dimension A. should be selected.

20 STEP 2 Fill in the specifics Agree the specific objectives of the activity you want to evaluate. For example if the participation activity relates to trying to influence a particular policy, service or practice (that a youth forum has selected, for example) record the change that the young people want to see (see Indicator A4 in the Framework on page 5). Try and be as specific as possible it will help you later when you come back to assess the impact of children or young people s participation. Young people specifying they want careers advice available in informal surroundings outside of school as well as in school and to have drop in sessions is going to be easier to evaluate in policy terms than young people specifying they want better careers services.

21 Under Dimension A: Impact on Polices/Services. As well as recording the changes that the children or young people want to see, if you are using indicator A3 record the issue(s) that the activity is hoping to raise awareness of and define who are the relevant service providers. Add any additional indicators selected by children or young people.

22 Dimension B: Impact on Social and Power Relations. If you expect or want to have an impact on broader social and power relations between adults and children/young people have a look at the suggested indicators under Dimension B. This can seem the most complicated dimension of change in the Framework. Activities designed to challenge negative stereotypes of young people, demonstrate the valuable contribution young people can make and thereby encourage adults at all levels of society and in all roles (professional, parent, neighbour etc.), to engage with, listen to and take into account children and young people s views when making decisions about things that affect them, would come under this heading. The suggested indicators under Dimension B aim to help you capture evidence of change along these lines.

23 Dimension C: Impact on personal development and wellbeing. All the indicators under this Dimension rely on before and after measurements. When looking to assess the impact of participation on the personal development and well-being of children you will need to decide on which children or young people the participation is expected to impact on usually it will be the children involved in the forum or group but you may have a wider target group. Again, the forum or group may wish to supplement the suggested indicators with one s of their own.

24 STEP 3 Data Collection Collect the BEFORE information and store it safely. Decide when you are going to collect the AFTER information. In making this decision you need to consider: At what point might you expect to see change? Changing a policy or a service or how a service is delivered can take a long time it might require a policy decision, an implementation plan, training before any changes are felt on the ground. The availability of the data some statistics are only published annually; surveys of children or young people in schools cannot be undertaken during school holidays or over examination periods. How long the group or forum and/or individual members will be around for. When you are reviewing the impact of the participation activity you will probably want to try and understand how these changes have occurred and what helped or didn t help achieve a good impact or maximum benefits.

25 STEP 4 Analysis and report The final tasks are to make sense of all the information you have collected and report on the evidence of the impact of children or young people s participation on whatever change dimensions you selected. You will then be able to make use of this report in any ways you wish including (we hope!) as part of your regular reviews of the way you work (in line with Participation Standard Improving how we work ).

26 Conwy Impact Assessment Pilot Aim: To improve and formalise existing methods for measuring the impact that the Youth Council has in general. Objectives: To introduce a robust system for measuring the difference that being involved in the Youth Council has made to the young people taking part e.g. skills, accreditation, achievement (impact on personal development and wellbeing); To introduce a meaningful method for measuring what difference the Youth Council has made to the delivery of the Children and Young People s Plan in Conwy (impact on policies/services); To explore methods for measuring how the Youth Council s involvement has had an impact on the relationship between adults/services and young people (impact on social and power relations); To ensure that Youth Council members are involved in developing these methods, and in evaluating their own work. Give priority to this objective

27 Baseline Before measure, recorded through a log book filled in by Conwy Youth Council members, during October B2: Young people s views on: (a) adults awareness of children s rights Log book question: Adults and your rights - do you feel that the adults working with you know about your rights and think that they re important? Adults and your rights Yes No Number of Youth Council members

28 B3: Young people s views on whether they can change things and influence the services and community provision that affect them (and C4) Log Book question: Your Influence - do you feel like you re able to change things and influence the things that affect you in Conwy? Your influences Yes fully Not at all Number of Youth Council members

29 In your groups. How do you currently measure impact? Where are you missing evidence? How applicable would using this framework be in your setting? What are the suggestions for improvement to the framework and/or implementation

30 Feedback

31 Contact Us- For more information about the work that we do, please contact us Participation Unit Save the Children Phoenix House 8 Cathedral Road Cardiff, CF11 9LJ Phone participationunit@savethechildren.org.uk Website-