TERMS OF REFERENCE EVALUATION OF THE IMPACT OF THE CHILD-FRIENDLY SCHOOL APPROACH IN THE EASTERN CARIBBEAN

Transcription

1 TERMS OF REFERENCE EVALUATION OF THE IMPACT OF THE CHILD-FRIENDLY SCHOOL APPROACH IN THE EASTERN CARIBBEAN 1. Purpose The CFS approach has been on-going in Barbados for three years and in Dominica for two years and there have been several anecdotal evidences of its benefit and effectiveness, which need to be verified and documented. There have been criticisms of the approach in Barbados and this has led to the renaming of the programme as Positive Behaviour Management programme rather than CFS. Also, the Governments in Barbados and in Dominica are about to take a policy-decision on scaling-up the programme in all primary and secondary schools and it is a programme imperative to evaluate its relevance, effectiveness, impact and sustainability before expanding to additional schools and countries in the Caribbean. Finally, different countries and schools within countries have adapted the CFS approach to fit the local conditions and there is a need to identify which approaches or adaptations work best, why do they work and under which conditions and what does not work and why. The evaluation is part of UNICEF s accountability for results, and it will contribute to improved decisions making ad programme delivery. As noted above, it will provide an evidence base to support advocacy, policy development and social mobilization for wider adoption of the approach. The primary audience will be national Governments in Barbados and Dominica, while the evaluation will also be shared with UNICEF s donors, the Organization of Eastern Caribbean States and other development partners in the Education sector. The conduct of the evaluation is of mutual benefit to the Ministries of Education in Barbados and Dominica and UNICEF. 2. Objectives The objectives of the evaluation are to: 1. Assess the results achieved during the programme implementation period in relation to the planned results to provide evidence of the relevance, effectiveness, and impact 2. Estimate the financial cost of implementing the CFS programme to provide evidence of affordability to national Governments 3. Identifying challenges encountered and solutions applied during the implementation process and draw lessons learned including factors that have been responsible for outstanding success or limited success. 4. Factors that can promote sustainability of the CFS approach and adoption of the approach on a national level in all Eastern Caribbean countries In pursuant of the above objectives, the following evaluation questions are being proposed. CFS Principle Child centeredness Questions -What results/effects have been achieved by the CFS initiative for children (improved learning achievement, life-skill acquisition, positive attitudes concerning health and hygiene, reduction of high risk behaviour and need for corporal punishment, reduction in drop out and repetition, increased connectedness to school, children feeling safer at school, positive change in behaviour of teachers and parents?) -To what extent is the Child Friendly School model relevant and effective in improving the educational quality of participating schools? -Is there evidence that there has been an increase in CFS school access, assistance and permanence, compared to similar schools (not participating)? -To what extent has CFS achieved an effective and high-quality learning environment? -To what extent has CFS created a health-promoting and health-seeking environment (i.e., achieved access to safe drinking water, good school hygiene, and a clean school 1

2 CFS Principle Democratic participation Inclusiveness Other important issues Questions environment)? -To what extent has CFS created a rights-based and inclusive environment in its schools? To what degree are these learning environments safe, protective and caring? -What has been the efficiency of strategies pertaining to participation, influencing educational policies, institutional strengthening and a demand focus? -What has been the inter-sectoral coordination and level of ownership of the Ministry of Education and schools? -To what extent has CFS created a gender-sensitive environment that promotes equity and equality in schools? -To what extent has CFS increased the democratic participation of students, parents, and community members and forged meaningful linkages between schools and communities to improve the child friendliness of its schools? -Are there evidences that show improved gender indicators, and gender equity? -Have the impacts of CFS achievements been at an acceptable cost, compared to other similar models? -What coordination mechanisms have been developed in order to guarantee sustainable impacts of results? -What is the added value of UNICEF s technical and financial cooperation in the implementation process for CFS, through support of national institutions? 3. Context/Background UNICEF s framework for CFS programming translates the language of the UN CRC into practical policy and operational guidelines that allow the goals of the UN CRC to be realized at all levels of any country s education system. The promotion and expanded utilization of the CFS framework contributes directly to achieving EFA and MDG goals, along with LAC Regional Meta 2021 targets, while strengthening actions aimed at the principle of inclusion supports the fulfilment of specific articles of the more recently approved UN Convention on the Rights of Persons with Disabilities. UNICEF envisions and promotes CFS models not as abstract concepts or a rigid blueprint but rather as pathways towards quality in education that reflect three key, and interrelated, principles derived from the Convention of the Rights of the Child: (i) Child-Centredness: Central to all decision-making in education is safeguarding the interest of the child; (ii) Democratic participation: As rights holders, children and those who facilitate their rights should have a say in the form and substance of their education; and (iii) Inclusiveness: All children have a right to education. Access to education is not a privilege that society grants to children; it is a duty that society fulfils to all children. UNICEF believes: A child-friendly school actively identifies excluded children and gets them enrolled in school. It regards education as every child s right and helps to monitor the rights and well-being of every child in the community. A child-friendly school acts in the interests of the whole child, which includes his or her health, nutrition and overall well-being. It cares about what happens to children in their families and communities before they enter school and after they leave it. But countries have taken a wide range of strategies to achieve the main CFS principles. The path to child-friendly (inclusive) schools may vary, but the goal is the same to ensure that all children have access to quality education and are nurtured in a child-friendly (inclusive) environment where they (all) can develop their full potential. The CFS programme is implemented under the Social Policy, Monitoring & Evaluation for Children s Rights Multi-country Programme covering Barbados and the Eastern Caribbean. Implementing agencies include the Ministry of Education in Barbados, Dominica, Antigua & Barbuda and St. Lucia. The CFS initiative was introduced to principals and teachers at a meeting of the Caribbean Union of Teachers (CUT) held in Barbados in December 2006 as a follow-up to the UN Secretary General Study on Violence Against Children. The focus was on positive behaviour management as an alternative to corporal punishment in schools. 2

3 CFS started in Barbados in 2007 at one school under the name Changing the Classroom Culture. It has been about three years of piloting and experimenting now with the approach and the approach has expanded from its initial focus on alternatives to corporal punishment to include a more holistic approach towards creating childcentred, inclusive, participatory and protective learning environment in the schools. In addition to positive behavioural management there is now emphasis placed on: Enhancing Health and family Life Education in school along with promoting healthy lifestyles; Promoting learning ready and inclusive classrooms; Promoting student Involvement in school governance and parental and community involvement in the school The approach was received with scepticism at first, but anecdotal reports from those schools that use the approach indicate that they are happy with the results they are getting. Scepticism of the approach continues to re-surface on a regular basis in the public domain. In Barbados and in Dominica, the Ministry of Education plans to scale up the CFS approach and introduce it into all schools, primary and secondary and requires documented evidence of its benefits, cost-effectiveness and relevance to the Caribbean context. The CFS approach also needs to gain the support of all its educational partners and the public. 4. Evaluation Scope, Focus, Questions and Criteria In terms of geographical scope, although the CFs is being implemented in Barbados, Dominica, Antigua & Barbuda and St. Lucia, the evaluation will however only cover Barbados and Dominica, where implementation has been longest. The primary reason for this is financial constraints. The timeframe for the evaluation is about 5 months and will occur between July and Dec Following from the objectives of this consultancy, relevance, effectiveness, impact, cost effectiveness and sustainability will be the primary Evaluation criteria to be considered. The project will also be evaluated for evidence of the results-based management (RBM) and the human rights based approach (HRBA). 5. Evaluation Questions Objectives Evaluation Criteria Questions 1.Results 3.Challenges & Lessons Learned 4.Sustainability & adoption Relevance -What results/effects have been achieved by the CFS initiative for children (improved learning achievement, life-skill acquisition, positive attitudes concerning health and hygiene, reduction of high risk behaviour and need for corporal punishment, reduction in drop out and repetition, increased connectedness to school, children feeling safer at school, positive change in behaviour of teachers and parents?) -To what extent is the Child Friendly School model relevant and effective in improving the educational quality of participating schools? -Is there evidence that there has been an increase in CFS school access, assistance and permanence, compared to similar schools (not participating)? -To what extent has CFS achieved an effective and high-quality learning environment? -To what extent has CFS created a health-promoting and healthseeking environment (i.e., achieved access to safe drinking water, good school hygiene, and a clean school environment)? -To what extent has CFS created a rights-based and inclusive environment in its schools? To what degree are these learning environments safe, protective and caring? HRBA 2.Cost Cost Effectiveness -What has been the efficiency of strategies pertaining to participation, influencing educational policies, institutional strengthening and a demand focus? 3

4 Objectives Evaluation Criteria Questions 3.Challenges & Lessons Learned Sustainability HRBA/RBM -What has been the inter-sectoral coordination and level of ownership of the Ministry of Education and schools? 4.Sustainability & adoption 1.Results 3.Challenges & Lessons Learned 4.Sustainability & adoption 2.Cost 3.Challenges & Lessons Learned 4.Sustainability & adoption Relevance HRBA RBM Cost Effectiveness Sustainability Relevance HRBA/RBM -To what extent has CFS created a gender-sensitive environment that promotes equity and equality in schools? -To what extent has CFS increased the democratic participation of students, parents, and community members and forged meaningful linkages between schools and communities to improve the child friendliness of its schools? -Are there evidences that show improved gender indicators, and gender equity? -Have the impacts of CFS achievements been at an acceptable cost, compared to other similar models? -What coordination mechanisms have been developed in order to guarantee sustainable impacts of results? -What is the added value of UNICEF s technical and financial cooperation in the implementation process for CFS, through support of national institutions? 6. Methodology Evaluation approach/limitations The evaluation approaches proposed for is a combination of goal-based, case studies and cost-benefit. The purpose of the goal-based approach is to determine whether a programme s goals have been achieved or not. It will require the use of quantitative and qualitative methods and analysis will focus on the gap between goals and achievement. The report may or may not be structured by goals and objectives. The strength of this approach is that the criteria for judging success is explicitly stated, while the weaknesses are bad objectives, a narrow scope, and side effects. Case studies will examines issues of importance to key stakeholders, using qualitative methods (document analysis, observation and interview), qualitative data analysis (patterns, categories, etc.), reports which contains description and analysis of issues. The strength of this approach is its attention to context and implementation, while its weakness is resource intensiveness. The purpose of the cost benefit approach is to judge a programme s productivity in economic terms. It does this by computing the sum of the present value of the benefits less the present value of the costs, using accounting methods and compares Net Present Value (NPV) of alternatives. The report tends to be structured by cost-benefit comparisons. Its strength is the ability to determine value for money, while its weakness is the difficulty in assigning monetary value to outcomes. The evaluation will follow UNEG norms and standards for evaluation, as well as ethical guidelines. Evaluation process The overall flow of the evaluation process will cover a sequence of six (6) key stages: Step 1: Engagement of stakeholders The objective is to foster the input, participation, and power-sharing among those with a vested interest in evaluation. This will be the very first discussions held before the evaluation team is on board, and as initial plans and Terms of Reference is developed. Step 2: To define and understand the purpose of the evaluation helps to clearly define the expected users and end use of the evaluation and what is to be evaluated. Managers of evaluation lay the purpose out in a ToR which includes a preliminary description of the programme to be evaluated 4

5 to ensure a clear understanding of its purpose, placing this accurately in the wider context. Step 3: Focus the Design This involves mmoving from the broad purpose to a logical description of what will be done and how. Questions and methods will be adjusted to achieve an optimal match that facilitates use by primary users. Step 4: collecting the Evidence The consultants will compile information stakeholders perceive as trustworthy and relevant for answering questions, ensure that adequate data is available and easily accessed, or new data that might need to be collected. Please note that credibility depends on - how the questions were posed, sources of information, conditions of data collection, reliability of measurement, validity of interpretations, and quality control procedures. Step 5: Analysis, Conclusions and Recommendations The consulting firm will process the data gathered according to accepted procedures, draw conclusions that follow logically from this and are consistent with the agreed values or standards of stakeholders and finally make logical and coherent recommendations. Step 6: Sharing and Using Evaluation Results This increases the chances that key stakeholders are aware of results of the evaluation its conclusions, recommendations and lessons where applicable. It also increases the probability that stakeholders will consider those results in decisions or actions they take and that hose involved in the process have benefited. Data collection and Information Sources A representative sample of schools from Barbados and Dominica will be selected with due consideration for the following criteria: Schools and varying durations spent implementing the CFs approach Representative sample of students in school population Secondary vs. Primary schools Data collection instruments appropriate to the methodology and evaluation question will be developed and pilots by the consulting firm. The various types of analyses to be undertaken and level of precision desires will also be included as part of the Inception report. Reference indicators, and benchmarks will be provided where applicable. The following are some information sources that exist, such as monitoring systems and/or previous evaluations. An appraisal of their quality and reliability is not currently available. Information source -Monthly reports from CFS coordinator in each country -Project proposals to UNICEF & UNESCO -School-wide Evaluation Tool -Monitoring information from needs assessment tool -Office disciplinary referrals/suspension data -Drop out data -Relatedness to M of E white paper (policy) objectives code of conduct, discipline policy of school (rewritten), special education policy -Assessment data on literacy and numeracy; special ed referral data if available -Student performance (CXC; common entrance) -Public perception of school -Subjective report of attitude change -Student perception of teacher performance -Evaluate role of the M of E thus far Quality of Information (Poor-Fair-Good-Excellent) Comments 5

6 7. Stakeholder Participation and Responsibilities The following are the specific roles and responsibilities of the Evaluation Team leader and team members, the steering committee and a Reference group. It is proposed that the Steering committee be composed of : Country Education Officers, UNICEF, UNECSO; NGOs (Child Fund Caribbean, MoE Planning Division and the lead Consultant. The Steering committee will be responsible for: Liaison with the evaluation team Providing technical guidance Co-ordinating the stakeholders involved Approval of intermediate and final products Ensuring effective dissemination to all stakeholders Primary stakeholders will include those implementing the evaluation i.e. Ministries of Education and UNICEF. Secondary stakeholders include the beneficiaries of CFS: the communities, families, students, teachers, PTAs members,,teachers Union, /CBOs, other relevant ministries. A Reference group proposed will be made up of the Regional M&E adviser, Regional Education Adviser and an expert in education from the UWI Cave Hill Campus Activity Stakeholders Roles and responsibilities Planning & -Education officers, CFS Coordinators, -Liaison - Participate in meetings to draft and review ToR design of evaluation -Donors (UNICEF UNESCO)) -Consulting team/evaluators; -Reference group -Liaison/Technical - Build Evaluation capacity, convene and provide financial support for the process. -Technical - Review ToR, meet with stakeholders to finalize the purpose, scope, focus and methodology of the Evaluation. -Technical and Advisory Data collection & -Liaison Review preliminary findings analysis Reporting Dissemination Follow-up: implementers & -Education officers, CFS Coordinators, -Donors (UNICEF UNESCO)) -Schools/Students/Teachers Communities/families, PTAs, Teachers union, NGOs/CBOs, other relevant ministries -Consulting team/evaluators; -Reference group -Education officers, CFS Coordinators, -Donors (UNICEF UNESCO) -Schools/Students/Teachers Communities/families, PTAs, Teachers union, NGOs/CBOs, other relevant ministries -Consulting team/evaluators; -Reference group -Education officers, CFS Coordinators, -Donors (UNICEF, UNESCO) -Schools/Students/Teachers Communities/families, PTAs, Teachers union, NGOs/CBOs, other relevant ministries -Liaison and technical -Participatory - Provide data -Technical and Advisory - -Technical and Advisory -Liaison Review conclusions and recommendations -Liaison and Technical -Participatory Receive feedback of Evaluation -Technical and Advisory - -Technical and Advisory -Liaison Follow up on implementation of findings and recommendations -Liaison and Technical -Follow up Political pressure, Demand change 6

7 Activity Stakeholders Roles and responsibilities -Consulting team/evaluators; Reference group -Technical and Advisory 8. Evaluation Team composition or Qualification/experience Areas of Technical competence - Specialists in Evaluation, with a background in education, Curriculum development, Psychology, Inclusive Education, with knowledge of school-wide positive behaviour management, child development and experience in conducting focus groups and key informant interviews Language proficiency - English and French Creole would be an asset Work experience - Work experience in the Caribbean, knowledge of the educational, political, social, economic climate in the region. A proven experience of having conducted successful evaluations. Other skills and competencies facilitation and team management sills, data collection and analysis skills and knowledge of various international frameworks of analysis e.g. Result-based management (RBM), Human rights and Gender. 9. Logistics/Working Conditions and Deliverables The consulting firm will be responsible for contacting focal persons, ministries, agencies and organizations in the countries, with the support of project specialists. The steering committee will also facilitate the process by sharing phone numbers and addresses of country level contacts, in addition to ensuring a smooth implementation process. The consulting firm will be responsible for provision of services (local translators, interviewers, data processors, drivers), and other needs of the research/field work e.g. office space, cars, laptops, tape recorders etc The following is the list of products to be delivered and some specifications required: The completed data set (completed questionnaires or surveys, raw data and tabulations) is the property of UNICEF and should not be used without permission. Consultant will prepare a Management Response to the Evaluation, to be discussed, finalized and posted online for follow up on the UNICEF Evaluation Database in the required format. Softcopies of all deliverables must be delivered in MS Office 2007 formats (Word, Excel or Powerpoint) Action by Consulting Firm July Aug Sept Oct Nov/Dec Deliverables 1.Consultant meets committees and x Inception report prepares Inception Report & Evaluation Plan using Evaluation Report Standards. Meetings, consultation, workshops with different groups of Stakeholders. 2.Develops Evaluation report outline x Outline of report according to Evaluation Report standards 3.Data Collection x x x 4. Data Analysis and verification of findings with key stakeholders x x Preliminary findings 5.Prepares a Draft report with x x Draft report Dissemination plan in line with evaluation Report Standards and comments from stakeholders. Presentation of preliminary findings and recommendations 6.Prepares Final report (Evaluation report x x Final report and standards) and Powerpoint Presentations presentations 7

8 Action by Consulting Firm July Aug Sept Oct Nov/Dec Deliverables targeting Policy-makers, Project managers and the Media. 7.Dissemination meeting x Presentations 10. Official Travel It is anticipated that the consultancy team will visit Barbados and Dominica to hold meetings with stakeholders, collect data, and share preliminary findings of the Evaluation. 11. Conditions of Service Prior to commencing the contract, the following conditions must be met: A Corporate Entity will be required to submit a Certificate of Incorporation for the Company as well as a profile of the individual(s) who would be undertaking the assignment An Individual Consultant will be required to submit a statement of good health, accompanied by a recent Medical Certificate which indicates that the Consultant is fit for work and travel. In addition, the Consultant is required to certify in the Health Statement that he/she is covered by medical/health insurance. The statement includes confirmation that he/she has been informed of any inoculation required for the country or countries to which travel is authorized. He/she takes full responsibility for the accuracy of the statement. 12. Recourse UNICEF reserves the right to withhold all or a portion of payment if performance is unsatisfactory, if work/outputs is incomplete, not delivered or for failure to meet deadlines. Performance indicators against which the satisfactory conclusion of this contract will be assessed include: timeliness/quality of submission and responsiveness to UNICEF and counterpart feedback 13. Property Rights UNICEF shall hold all property rights, such as copyright, patents and registered trademarks, on matter directly related to, or derived from, the work carried out through this contract with UNICEF. Application should be submitted no later than Friday 08 July 2011: Representative UNICEF UN House Marine Gardens Christ Church BARBADOS Or to Bridgetown@unicef.org ONLY SUITABLE APPLICATIONS WILL BE ACKNOWLEDGED 8