SUPPORTING THE DEVELOPMENT OF THE JUSTICE SECTOR IN ACEH (November 2007 December 2008) EVALUATION REPORT. January 2009

Transcription

1 SUPPORTING THE DEVELOPMENT OF THE JUSTICE SECTOR IN ACEH (November 2007 December 2008) EVALUATION REPORT Conducted by Adrian De Giovanni (IDLO) Nicole Hoagland (IDLO)

2 TABLE OF CONTENTS EXECUTIVE SUMMARY Page I. EVALUATION METHODOLOGY AND LIMITATIONS Evaluation Scope and Purpose Methodology Evaluation Approach Evaluation Limitations 3 II. PROGRAM BACKGROUND Problem Analysis Program Design Program Strategy Program Implementation Harmonization and Partnerships Staffing Financial Management Support to the Field 9 III. EVALUATION FINDINGS (by Project) Women s Legal Aid Services Project The Project Summary of Post-Training Evaluation Questionnaires Conclusions Community Mediation and Legal Skills Training Project The Project Evaluation Conclusions Raising Women s Legal Awareness Through Film Project The Project Evaluation Conclusions Raising Community Legal Awareness through Print Media Project The Project Evaluation Conclusion 54 IV. LESSONS LEARNED 55 V. SUMMARY OF RECOMMENDATIONS 56 Annex 1 Documents Consulted 57 Annex 2 Schedule of Stakeholder Interviews 58 Annex 3 Stakeholder Interview Questions 59 Annex 4 Beneficiary Interview Questions 60 Annex 5 Villages Visited for Evaluation 61 Annex 6 Legal Awareness Survey 62 Annex 7 List of Published Newspaper Articles 63 Annex 8 Evaluation Exercise Terms of Reference 65 Annex 9 Program Logic Model 68

3 AREAS VISTED BY THE EVALUATION TEAM IN ACEH BANDA ACEH, ACEH BESAR, ACEH BARAT, ACEH JAYA, ACEH UTARA, BIREUEN PIDIE, PIDIE JAYA 1 1 See Annex 5 for a complete list of villages visited.

4 EXECUTIVE SUMMARY This is an evaluation of the Supporting the Development of the Justice Sector in Aceh Program (the Program) funded by the Oxfam GB Partnership Programme and implemented by the International Development Law Organization (IDLO). The Program was implemented from November 2007 to December 2008 and encompassed five projects: 1. Supporting Women s Legal Aid Services (Legal Aid Project) 2. Community Mediation and Legal Skills Training (Community Mediation Project) 3. Raising Women s Legal Awareness through Film (Film Project) 4. Raising Community Legal Awareness through Print Media (Print Media Project) 5. Research on Citizen s Rights to Participate in Government Decision-Making Processes and Qanun Drafting in Aceh 2 (Qanun Research Project) The goal of the evaluation is to contribute to IDLO s on-going work to support the development of the justice sector in Aceh in order to strengthen the rule of law and good governance. The evaluation also serves as a tool for sharing knowledge and lessons learned across IDLO s global programs. The evaluation was carried out by two IDLO staff, external to the Program and based in Sydney, Australia, to guarantee a level of objectivity and impartiality. In addition to the devastating effects of the 2004 Boxing Day tsunami, which killed thousands and damaged or destroyed infrastructure and livelihoods, the people of Aceh had suffered from 30 years of violent, separatist conflict, leading to the severe isolation of the province from the rest of Indonesia. The consequences of the conflict and tsunami have resulted in the justice sector institutions lacking the necessary capacity including through a lack of properly trained formal justice service providers to protect the rights of the Acehnese community 3. Since the tsunami and the peace accord between the separatists and the Indonesian government, Aceh has continued to largely rely on customary dispute resolution methods to resolve legal disputes. Community knowledge and awareness of the formal law and justice system remains low and cultural norms continue to discourage adversarial processes of the kind found in formal justice sector 4. The lack of knowledge and awareness among adat 5 leaders of formal laws has meant that their decisions on village disputes which come before them for resolution are in many instances in breach of Indonesian law and disproportionately affect vulnerable and marginalized groups such as women and children. In February 2006, IDLO established a Project Office in Banda Aceh. The Project Office was responsible for implementing IDLO s Post-Tsunami Legal Assistance Initiative which sought to address the legal needs of the Acehnese community in the post-tsunami environment by raising community legal awareness (with a particular focus on women s legal rights) and building capacity among village leaders to conduct fair and impartial dispute resolution processes. At the conclusion of the Post-Tsunami Legal Assistance Initiative in October 2007, IDLO continued implementing legal-awareness raising activities and mediation skills trainings under the Oxfam-funded Supporting the Development of the Justice Sector in Aceh Program which concluded in December As agreed between IDLO and Oxfam, the evaluation of the Qanun Research Project was not included in this report as activities were ongoing at the time of IDLO s evaluation mission to Aceh. 3 According to the UNDP, some 105 court personnel and over 1,000 police officers lost their lives or remain missing. Thirty-four percent of justice-related infrastructure including police stations, prosecution offices, court houses and prisons, is estimated to have been gravely damaged or destroyed. CGI, Preliminary Loss and Damage Assessment, January 2005, p.64 4 UNDP, Access to Justice in Aceh: Making the Transition to Sustainable Peace and Development, 2006, Acehnese customary law and procedures.

5 Overall Conclusions The Program was, overall, highly relevant to community needs and effective in meeting its stated objectives 6 in that it contributed to the overarching goal of strengthening the rule of law and good governance in Aceh. The Program effectively worked at the grassroots level to respond to community needs in both tsunami-affected and post-conflict areas of Aceh, with a particular focus on the needs of women. All Program activities were delivered in a culturally appropriate manner and in accordance with local customs, resulting in high levels of beneficiary engagement. The majority of interviewed beneficiaries retained key information and skills learned through the Program and continue to find the acquired knowledge useful and applicable to their daily lives. Beneficiaries of the Film Project and the Community Mediation Project maintained higher levels of legal awareness when compared to those of the general public in Aceh who did not receive direct training from the Program. The Program operated according to its intended schedule and proved to be cost-effective, allowing for the implementation of additional activities beyond those originally planned. While it may be too early to accurately assess the Program s impact, evaluation findings suggest that IDLO has influenced a critical mass of actors involved in village-level dispute resolution processes and effectively raised the awareness of legal rights among women in beneficiary villages. Findings indicate that the Program may be contributing to triggering a process of change in community attitudes and practices in Aceh, promoting and protecting individuals legal rights. However, IDLO cannot expect its activities to have a lasting impact on communities without a strategy to ensure that the Program s quantifiable immediate achievements (that is, increased legal awareness and improved mediation skills among participants) are extended into the future through a process of local ownership and building beneficiary capacity to transfer acquired knowledge. In addition, the evaluation finds that the impact of the Program would be heightened if a greater link between projects existed, especially between the Film Project and Community Mediation Project in the same villages. If activities are conducted to increase the awareness of legal rights in a community (via the Film Project), it is essential to simultaneously strengthen the processes through which the community can claim these rights (via the Mediation Project). Evaluation Findings Relevance The Program was extremely relevant to the needs of communities in Aceh, confirming the validity of its objectives beyond the post-tsunami context. The topics addressed in the Program corresponded to the most significant legal issues facing Acehnese communities in both tsunami-affected and post-conflict areas, reinforcing the importance of the extension of IDLO s activities to villages in post-conflict locales. The large majority of interviewed beneficiaries confirmed that the information and tools provided through the Program were useful and applicable to their daily lives. The findings of the evaluation suggest that domestic violence is a prevailing concern in Aceh. Though the Film Project s content was appropriately modified to include domestic violence, the Community Mediation Project failed to incorporate the topic into revised modules. 7 The evaluation found that the relevance of the Program would have been enhanced had IDLO incorporated the topic of domestic violence into all project activities. 6 Except for the objective of the Supporting Women s Legal Aid Services Project, since a partnership between Oxfam and KKTGA (the beneficiary) was not established. It was decided among IDLO, Oxfam and KKTGA to pursue only one of two outputs under this project. 7 Domestic violence was identified as a key issue for training in the evaluation of IDLO s Post-Tsunami Legal Assistance Initiative ( ).

6 The evaluation also found that there is a heightened need for in-depth information and training on the formal justice system and its procedures. Incorporating information on this topic in the Community Mediation Project would have increased the relevance of the Program in Aceh. In addition, evaluation findings revealed a need for women s empowerment activities and awareness-raising on women s rights in general. By providing information on women s legal rights beyond the topics of land, inheritance, guardianship and domestic violence in the Film Project, the overall relevance of the Program would have been strengthened. Effectiveness The Program was implemented according to its intended schedule and proved to be costeffective, allowing for the extension of activities beyond those originally planned. The Program achieved its stated objectives 8 and delivered culturally appropriate activities in accordance with local customs, effectively engaging beneficiary participation. IDLO s strategy of working at the grassroots level and bringing activities directly to villages proved to be valid and highly appropriate, as was the use of a visual medium as an educational tool for audiences such as women, who generally exhibit lower levels of literacy and education. While it is difficult to accurately assess the effectiveness of print media as a tool to raise legal awareness, key legal stakeholders interviewed for the evaluation indicated that it was a valid strategy to share legal information with a broad audience in Aceh. However, readership is largely limited to the educated and those living in urban areas with access to newspapers. Evaluation findings revealed that the quality of IDLO s facilitators and IDLO s training methodology, as well as the accuracy of Program content, resulted in high levels of beneficiary engagement and led to considerable improvement in knowledge and skills among participants. The evaluation identified a high instance of practical application of information and skills acquired through the Program, indicating that IDLO s activities are positively affecting members of the general community, including vulnerable groups such as women and children. The evaluation also found that the effectiveness of the Program would have been enhanced by extending activities beyond the identified target beneficiaries by including the general community (rather than only community leaders) in the Community Mediation Project, and by raising the awareness of women s legal rights among the general male population in the Film Project. Impact and Sustainability It is perhaps too soon to assess the Program s overall impact. Moreover, it is difficult to make direct conclusions regarding the long-term effects of the Print Media Project, given the project s wide target audience and the evaluation s limited scope. Nonetheless, the evaluation determined a high instance of practical application of knowledge and skills acquired through both the Film Project and Community Mediation Project, suggesting that these activities may be contributing to triggering a process of change in community attitudes and practices. Anecdotal evidence in beneficiary communities indicated that village level dispute resolution processes are improving in regards to fair and equitable decisions, and that women now have a greater awareness of their legal rights and are taking action to claim them. Despite this evidence, the evaluation found the Program s impact would have been heightened if: 1. The Film Project and Community Mediation Project had been implemented in the same villages on a consistent basis, rather than ad hoc or when logistically convenient. A greater link between projects would have empowered women to claim their legal rights while simultaneously strengthening the capacity of village leaders to protect such rights. 8 Except for the objective of the Supporting Women s Legal Aid Services Project, since a partnership between Oxfam and KKTGA (the beneficiary) was not established. It was decided among IDLO, Oxfam and KKTGA to pursue only one of two outputs under this project.

7 2. The activities of the Film Project had been extended to the general male population. In an environment such as Aceh, where the majority of disputes are resolved at the family or community level, educating the general male population on the legal rights of women is fundamental to ensuring the protection of these rights. 3. The Community Mediation Project had included a legal awareness component for the general community, empowering individuals to hold village leaders and mediators accountable for fair and equitable decisions in accordance with the law. The impact of the Program would also have been greater with a strategy to ensure that quantifiable immediate achievements (that is, increased legal awareness and improved mediation skills among participants) were extended into the future through a systematic transfer of local ownership and strengthening of beneficiary capacity to transfer acquired knowledge and skills. The lack of a strategy in this regard was a major shortfall in the Program which should be addressed in any future programming. Summary of Recommendations Overall Program For any future implementation or expansion of the Program, it is recommended that IDLO: Establish a greater link between projects, in particular the Community Mediation and Film Projects. The projects should be implemented in the same villages on a consistent basis, raising legal awareness at the community level while simultaneously strengthening the capacity of village leaders to protect individual s legal rights through more informed and legally sound decisions. Develop and implement a strategy to facilitate sustainability of the Program by encouraging local ownership and building beneficiary capacity to transfer acquired knowledge. Community Mediation Project For any future implementation or expansion of the Community Mediation Project, it is recommended that IDLO: Include domestic violence as a topic in the training materials (or design and implement a new domestic violence program altogether); Include training and information on the formal justice system in the project modules and involve formal institutions in the project to promote a link between the formal and informal justice sectors; Extend project activities to the broader community (not just community leaders); Use a tailored approach when implementing project activities in more remote and culturally conservative areas of Aceh to ensure greater relevancy for beneficiaries; Develop and implement a strategy to facilitate sustainability of the project by encouraging local ownership and building beneficiary capacity to transfer acquired knowledge; Extend the geographical scope of the project to other areas of Aceh Film Project For any future implementation or expansion of the Film Project, it is recommended that IDLO: Include domestic violence as a topic in the training materials (or design and implement a new domestic violence program altogether); Broaden its gender-focused activities beyond land, inheritance and guardianship law to address women s rights and legal empowerment more generally;

8 Extend film screenings (or women s legal rights activities) to all community members, directly targeting the general male population. IDLO should seek to amend its Program strategy to address the existing education gap among the general male population regarding women s legal rights; Develop and implement a strategy to facilitate sustainability of the project by encouraging local ownership and building beneficiary capacity to transfer acquired knowledge; Strengthen connection between the project and other projects, in particular the Community Mediation Project for example, IDLO should screen the film in all villages where the project is implemented. Print Media Project It is recommended that IDLO consider transferring this project to a local counterpart and perhaps provide assistance at the initial stages of implementation.

9 I. EVALUATION METHODOLOGY AND LIMITATIONS 1.0 Evaluation Scope and Purpose This is an evaluation report on the Oxfam-funded Supporting the Development of the Justice Sector in Aceh Program (the Program) implemented by IDLO between November 2007 and December The evaluation assessed the Program against the following criteria: Relevance; Effectiveness; Impact; Sustainability. The evaluation was conducted between 20 November and 2 December 2008 and covered the entire duration of the Program. All components of the Program were evaluated except for the Qanun Research Project. IDLO and Oxfam agreed not to evaluate this project for the purposes of this report as project activities were ongoing at the time of the evaluation. 2.0 Methodology A Monitoring and Evaluation framework (M&E framework) was established at the commencement of the Program. To ensure a certain degree of objectivity and impartiality, the evaluation exercise for the Program was conducted by Adrian De Giovanni (ADG) and Nicole Hoagland (NH), staff members from IDLO s Sydney Office who were not involved in any stage of the Program s development or implementation. Further, the M&E framework was designed by an external consultant Evaluation Approach The evaluation approach included: Document/Report Review: Review of relevant reports and other documents of national (including Acehnese) and international organizations and agencies to assess the relevance and appropriateness of IDLO s area of intervention (See Annex 1 for a list of documents consulted). Stakeholder Interviews: One-on-one interviews with representatives from eight stakeholders holding key legal positions to obtain credible perspectives on the Aceh s legal context and identify the relevance, effectiveness and appropriateness of IDLO s activities (see Annex 2 for interviewed stakeholders). Interviews were conducted by ADG and NH with the assistance of local staff from IDLO s Aceh Project Office acting as interpreters. The same questions were repeated to all stakeholders to obtain an objective and independent perspective. The interviews were conducted using a semi-structured approach to allow stakeholders to comment without being restricted to or prompted by the priorities of IDLO staff (see Annex 3 for stakeholder interview questions). Immediate Post-Activity Evaluation Questionnaires: The Community Mediation and Film Project participants were required to complete a post-activity evaluation questionnaire immediately after IDLO trainings. The questionnaires provide valuable feedback regarding the quality of IDLO facilitators, training methodology and relevance of project content. 9 The M&E framework was designed by UNIQUEST. However, the IDLO team conducting the evaluation exercise found it necessary to include additional questions to the evaluation tools in order to collect information regarding the impact, sustainability and cultural appropriateness of IDLO s activities. 1

10 Community Scorecards: a. Beneficiary Interviews: IDLO collected feedback from beneficiaries of the Community Mediation Project and the Film Project to obtain information regarding the relevance, effectiveness, sustainability and potential impact of IDLO s activities, as well as to identify suggestions for improvement. Interviews were conducted at least three months after implementation of IDLO s projects and the data was compared with the results of immediate post-activity evaluation questionnaires to obtain a stronger indication of the assessed criteria (see Annex 4 for beneficiary interview questions). 188 beneficiaries from the two projects (110 Film Project participants from 20 villages women and 5 men - and 78 Community Mediation Project participants from 20 villages - 60 men and 18 women - were interviewed) (see Annex 5 for villages visited). b. Legal Awareness Survey: IDLO conducted a survey of general legal awareness in Banda Aceh and Aceh Besar to gather information about the results of its awareness-raising activities. The survey targeted three separate audiences: - Film Project beneficiaries; - Community Mediation Project beneficiaries; and - Random sample of the general community in Aceh. The survey aimed to identify the overall level of legal awareness and awareness of legal institutions in the community, and allowed for comparison among the three audiences, providing IDLO with some insight into the relative effectiveness of the different activities and strategies utilized in the Program. 250 persons from the general public (124 men and 126 women) were surveyed between November 20 and December 2, 2008, in addition to the 188 project beneficiaries noted above (see Annex 6 for Legal Awareness Survey). Table 1: Summary of M&E Framework Project Objective Success Indicators M&E Tools Community Mediation and Legal Skills Training (Community Mediation Project) To raise the awareness of community leaders in tsunami-affected areas of Aceh regarding legal rights in the subject matter of land, inheritance and guardianship law, and to facilitate the fair and efficient resolution of tsunami- and conflict-related legal issues at the villagelevel by building the capacity of community leaders to mediate disputes a. At least 70% of participants increase their knowledge of key elements of the training course b. Feedback regarding the relevance and effectiveness of mediation training from key legal sector stakeholders c. Participants find the training effective, relevant and appropriate d. 60% of participants interviewed provide narrative examples of the practical application of techniques and principles from the training a. Interviews with key legal stakeholders b. Legal awareness survey conducted at least 3 months after training c. Interviews with participants at least 3 months after training d. Participants pre and post training knowledge evaluation tests e. Participants posttraining qualitative evaluation of training program and facilitators 2

11 Raising Women s Legal Awareness Through Film (Film Project) To improve the understanding and awareness of women in 126 villages in Banda Aceh, Aceh Besar and among Da I scholars and selected women s groups in Banda Aceh, Aceh Besar and Bireuen, regarding women s legal rights and the avenues through which such rights can be protected a. Feedback from key legal sector stakeholders regarding the relevance, effectiveness and appropriateness of the Film Project b. Participants find Film Project effective, relevant and appropriate (immediate assessment) c. Participants continue to find information provided by the Film Project useful months (at least 3) after screening/training d. 20% of participants interviewed provide narrative examples of the practical application of information provided by the Film Project a. Interviews with key legal stakeholders b. Legal awareness survey conducted at least 3 months after screening c. Interviews with participants at least 3 months after screening of Film d. Questionnaire administered immediately after each screening and workshop Raising Community Legal Awareness through Print Media (Print Media Project) To improve awareness among communities in Aceh regarding topical legal issues, legal rights and the various legal services available to protect those rights a. Feedback from key legal sector stakeholders regarding the relevance, effectiveness and appropriateness of the articles content b. At least 20% of a random sample of population in Aceh Besar have read the articles c. 20% of people surveyed, who read the articles, find the topics relevant a. 250 legal awareness surveys conducted in Aceh Besar among a random sample of population b. Interview with key legal stakeholders 4.0 Evaluation Limitations The team experienced and/or observed the following limitations in carrying out the evaluation exercise: The selection of beneficiaries to be interviewed during the evaluation exercise was at the discretion of village leaders. This reduced the likelihood of interviewing a representative sample of beneficiaries in the evaluation exercise. For example, in the Film Project, many women leaders and wives of village leaders were interviewed while women with lower community status were not proportionately represented. The evaluation of the Community Mediation Project included interviews only with direct beneficiaries (community leaders) who may have entertained some bias in regards to their mediation skills and to the fairness of dispute resolution processes and outcomes, particularly if directly involved. The accuracy of the evaluation would have been enhanced had members of the general community been included in interviews. This would have provided additional information on the communities perception of village mediation processes and allowed for comparison with leaders feedback. The evaluation was conducted during the wet season in Aceh which made transportation extremely difficult and threatened to compromise the collection of data in some cases. The time of year and weather should be taken into account in any future evaluation exercises. 3

12 Interpreters were not professionals which may have had some effect on the accuracy of the information collected through beneficiary and legal stakeholder interviews. The budget initially allocated for the evaluation was not sufficient for the exercise to be conducted by two IDLO staff members external to the Program (the minimum staff required for the exercise considering the scope of the Program). With the consent of Oxfam, Program savings were used to cover the additional expenses. This proved to be sometimes restrictive and limited the timeframe to conduct the evaluation. The allocated budget would have been insufficient for IDLO to engage an external (non-idlo) person to conduct the evaluation. 4

13 II. PROGRAM BACKGROUND 1.0 Problem Analysis Following a series of assessment missions and extensive research, IDLO determined that the overarching impediment to the development in Aceh of effective judicial processes through which individuals can access justice and claim their fundamental legal rights, as provided by the Universal Declaration of Human Rights 10 and a number of other international human rights instruments 11, is the province s weak rule of law culture. 30 years of violent conflict, severely under-resourced justice sector institutions and a lack of properly trained judicial personnel led to a decline in the number of reported cases and an increasing reliance on customary dispute resolution 12. This trend has been exacerbated by low community awareness regarding the formal justice system and procedures, and cultural norms which discourage adversarial processes. 13 Even at the informal, customary level, low levels of education among adat 14 leaders regarding the law (both formal and informal, customary) and limited dispute resolution and mediation skills, has meant that decisions are often inequitable and do not produce lasting resolutions, disproportionately affecting vulnerable and marginalized groups such as women and children. Over time, Acehnese communities have lost confidence in the capacity of both formal and informal mechanisms to respond to their legal needs, resulting in an inability to receive effective legal remedy and access to impartial justice. According to UNDP s research this lack of confidence has created an attitude of resignation towards justice mechanisms when it comes to particular types of grievances so that, rather than exhaust one s self attempting to access justice with futile results, people prefer to submit to their fate Program Design Following the December 2004 tsunami, IDLO established a Project Office in Banda Aceh, Indonesia, to implement its Post-Tsunami Legal Assistance Initiative which ran from February 2006 to October The Post-Tsunami Legal Assistance Initiative comprised a series of law-based projects aimed as assisting affected institutions and communities recover from the effects of the tsunami. In 2007, Oxfam published a Land Rights Policy Paper series with recommendations on land rights issues in post-tsunami Aceh based on extensive, field-based research over a six-month period. Many of the recommendations in the series suggested the expansion and continuation of some of the activities IDLO was implementing under its Post-Tsunami Legal Assistance Initiative. In light of these recommendations, coupled with discussions with stakeholders and research conducted by partner agencies, IDLO produced and submitted a Program Concept Paper to Oxfam in September The Concept Paper proposed the funding by Oxfam of some of the activities IDLO was implementing under the Post-Tsunami Legal Assistance Initiative in line with the recommendations contained in Oxfam s Land Rights Policy Paper series. In due course, Oxfam GB provided funds for 10 Article 6: Everyone has the right to recognition everywhere as a person before the law. Article 7: All are equal before the law and are entitled without any discrimination to equal protection of the law. Article 10: Everyone is entitled in full equality to a fair and public hearing by an independent and impartial tribunal, in the determination of his rights and obligations and of any criminal charges against him. 11 Including the International Covenant on Political and Civil Rights; the Convention on the Elimination of All Forms of Racial Discrimination; and the International Covenant on Economic, Social and Cultural Rights. 12 According to the UNDP, some 105 court personnel and over 1,000 police officers lost their lives or remain missing. Thirty-four percent of justice-related infrastructure including police stations, prosecution offices, court houses and prisons, is estimated to have been gravely damaged or destroyed. CGI, Preliminary Loss and Damage Assessment, January 2005, p UNDP, Access to Justice in Aceh: Making the Transition to Sustainable Peace and Development, 2006, Acehnese customary law and procedures. 15 UNDP, Access to Justice in Aceh: Making the Transition to Sustainable Peace and Development, 2006, 79. 5

14 these activities which were programmed as Supporting the Development of the Justice Sector in Aceh. The Program commenced in November 2007 and included the continuation of the following three projects from IDLO s Post-Tsunami Legal Assistance Initiative: Community Mediation and Legal Skills Training Project (Community Mediation Project) extended to post-conflict areas not affected by the tsunami; Promoting Women s Legal Awareness Through Film Project (Film Project) extended to post-conflict areas not affected by the tsunami; and Raising Community Legal Awareness through Print Media Project (Print Media Project). The Program also included two additional projects: Supporting Women s Legal Aid Services Project (Legal Aid Project); and Research on Citizen s Rights to Participate in Government Decision-Making Processes and Qanun Drafting in Aceh (Qanun Research Project). The following is a summary of IDLO s Program approved by Oxfam for implementation. Table 2: Program Design Framework TITLE: GOAL: Supporting the Development of the Justice Sector in Aceh To strengthen the rule of law and good governance in Aceh C Component 1: Component 2: Component 3: O M P O N E N Strengthening the Formal Justice Sector to Enhance Access to and the Administration of Justice Strengthening Informal Dispute Resolution Mechanisms to Provide Access to Justice, Uphold the Rule of Law and Protect Human Rights Strengthening Community Legal Awareness and Ability to Access Justice T P R O J E C T Supporting Women s Legal Aid Services Community Mediation and Legal Skills Training Program Raising Women s Legal Awareness Through Film Project Raising Community Legal Awareness through Print Media Research on Citizen s Rights to Participate in Government Decision- Making Processes and Qanun Drafting in Aceh As agreed between IDLO and Oxfam, the evaluation of the Qanun Research component was not included in this report as activities were ongoing at the time of IDLO s evaluation mission to Aceh. 6

15 O B J E C T I V E No. 1 To improve the legal skills of KKTGA staff in mediation and in land, inheritance and guardianship law, and to increase the institutional capacity of KKTGA staff to develop, execute and evaluate activities implemented under the Oxfam-KKTGA partnership No. 2 To raise the awareness of community leaders in tsunamiaffected areas of Aceh regarding legal rights in the subject matter of land, inheritance and guardianship law, and to facilitate the fair and efficient resolution of tsunami- and conflict-related legal issues at the village-level by building the capacity of community leaders to mediate disputes No. 3 To improve the understanding and awareness of women in 126 villages in Banda Aceh, Aceh Besar and among Da I scholars and selected women s groups in Banda Aceh, Aceh Besar and Bireuen, regarding women s legal rights and the avenues through which such rights can be protected No. 4 To improve awareness among communities in Aceh regarding topical legal issues, legal rights and the various legal services available to protect those rights No. 5 To provide Oxfam and other stakeholders with accurate information regarding government planning, budgeting, decisionmaking and Qanun drafting processes in Aceh, and the rights of Acehnese citizens to participate in these processes 3.0 Program Strategy As outlined in IDLO s Concept Paper to Oxfam, the Program was designed to: Strengthen the capacity of the actors responsible for protecting and ensuring the legal rights of individuals at the village level, mainly decision-makers and village leaders including Geuchik (village chief), Imam Meunasah (religious scholar), Tuha Pet (elder) and a range of women, youth and religious leaders; Raise the awareness of communities on their human and legal rights, particularly with respect to the rights of women. 4.0 Program Implementation All projects were implemented between November 2007 and December 2008 with the exception of the Legal Aid Project. The full implementation of the Legal Aid Project was contingent upon a partnership between Oxfam and KKTGA through which IDLO would provide oversight support for KKTGA s activities. While KKTGA expressed interest in this output during the Program s design phase, it moved away from this position prior to implementation, stating that it did not require monitoring or advice in relation to the execution of its 2008 activities. Therefore, as agreed between KKTGA, Oxfam and IDLO, this output was not pursued. In the case of the Community Mediation Project and the Film Project, additional activities were added to the original work plans, including the extension of both projects to postconflict areas not affected by the tsunami and the delivery of 1-day Film Follow-Up Workshops and 3-day Raising the Awareness of Women s Legal Rights Workshops under the Film Project. 5.0 Harmonization and Partnerships IDLO s work and experience in Aceh under the Post-Tsunami Legal Assistance Initiative to ensured that the Program did not include activities which duplicated or overlapped with the projects of other stakeholders and was delivered in a manner consistent with Acehnese norms and customs and relevant to the Acehnese context. In this regard, the Advisory Council for the Post-Tsunami Legal Assistance Initiative, comprising representatives from the General and Shari a Courts (Mahkamah Syar iyah), the Rehabilitation and 7

16 Reconstruction Agency for Aceh (Nias Badan Rehabilitasi dan Rekonstruks, BRR), the National Legal Aid Foundation (Lembaga Bantuan Hukum, LBH), the Aceh Adat Council, the UN Development Program and the World Bank, was of great assistance. IDLO continued its partnership with the Local Governance and Infrastructure for Communities in Aceh Project (AIPRD-LOGICA) which was first established under the Post-Tsunami Legal Assistance Initiative. AIPRD-LOGICA provided both financial and logistical support for all trainings under the Program s Community Mediation Project, except from the workshops conducted in Banda Aceh and a portion of those in Aceh Besar. IDLO workshops conducted in Aceh Jaya, Aceh Utara, Bireuen, Pidie and Pidie Jaya benefited from the financial, transportation and village facilitation support which LOGICA provided. IDLO also received logistical support from the Communities and Education Program in Aceh (CEPA) during implementation of the Community Mediation Project in five sub-districts where CEPA was working: Baktiya, Muara Batu, Kuta Blang, Peusangan and Simpang Mamplam. As these were new areas of implementation for IDLO, CEPA s support in coordinating with village leaders for participant selection and organization of training venues was critical to the successful delivery of the project in these areas. IDLO also worked in partnership with AIPRD-LOGICA and CEPA to implement its 3-day Raising Awareness of Women s Legal Rights Workshops as part of the Film Project. IDLO originally planned to implement six workshops in three areas (Banda Aceh, Aceh Besar and Bireuen). However, recognizing a greater need for these workshops outside of Banda Aceh and Aceh Besar, IDLO conducted four workshops in Bireuen and two in Pidie. This required additional financial and human resources which LOGICA and CEPA provided, respectively. 6.0 Staffing Dr Erica Harper headed the IDLO Project Office Aceh at the time the Program was being developed with Oxfam. She left this post in January 2008 and was replaced by Mr Edward Wright. At all material times, the IDLO Project Office reported directly to Mr Miles Young, Program Legal Counsel and Acting Director (since September 2007) of IDLO s Asia-Pacific Regional Centre in Sydney. Other staff members of the IDLO Project Office who worked on the Program are listed in Table 3. Many had been by hired by the Office prior to the implementation of the Program. Table 3: List of Staff Members Staff Member Hired Position Edward Wright January 2008 Team Leader Ibnu Fajar April 2006 Finance and Administration Officer Desi Sriwahyuni October 2006 Finance and Administration Assistant Ernita Dewi January 2007 Legal Officer (Film Project) Muhammad Maulana March 2008 Legal Officer (Community Mediation Project) Maria Ulfa January 2008 Muzakkir Abubakar November 2006 Legal Assistant (Film and Community Mediation Projects) Legal Officer (Community Mediation and Qanun Research Projects) Helen Debora Sirait January 2008 Training Coordinator Deny Syahputra March 2008 Driver Supriadi April 2006 Driver and Film Project Assistant 8

17 7.0 Financial Management The Program was executed under the allocated budget and in some cases this allowed for the addition of activities to those originally planned. Table 4: Financial Management Program Expenditures US$ % of Total Expenditures Monitoring and Evaluation $ 7, % Operational Costs $ 69, % Staffing Costs $ 150, % Program Implementation Costs $ 167, % Program Management $ 8, % Management Overhead $ 66, % Total Expenditures $ 470, % Allocated Budget $ 484, Savings $13, Support to the Field The Project Office in Aceh received high levels of support from the IDLO Asia-Pacific Regional Centre in Sydney, Australia, and IDLO Headquarters in Rome, Italy. 9

18 III. EVALUATION FINDINGS 1.0 Supporting Women s Legal Aid Services Project Summary of Evaluation Findings The evaluation came to the conclusion that the Legal Aid Project did not meet its stated objective due to Oxfam and KKTGA not establishing the relationship required to enable IDLO to implement all of the planned project activities. Nonetheless, IDLO s training activities under the Legal Aid Project were of high quality and relevant to the needs of beneficiaries. Further, the training activities under the project were extended beyond KKTGA to include staff from other Aceh-based NGOs. However, as no follow-up data was collected beyond the immediate post-training feedback from participants, it is not possible to draw conclusions about the effectiveness, impact or sustainability of the project. 1.1 THE PROJECT Project Overview Under its Post-Tsunami Legal Assistance Initiative, IDLO worked in partnership with Kelompok Kerja Transformasi Gender Aceh (KKTGA) 17 to raise awareness in Aceh about women s legal rights. From the experience of working with KKTGA and information collected in the Post-Tsunami Legal Assistance Initiative Monitoring and Evaluation Report, (M&E Report ), IDLO identified that the demand for KKTGA s legal aid services exceeded its capacity. Accordingly, the Program Concept Paper submitted to Oxfam in September 2007 proposed for IDLO to provide further capacity building activities to KKTGA. Following discussions with KTTGA, the Supporting Women s Legal Aid Services Project (Legal Aid Project) was developed with the objective of improving the legal skills of KKTGA staff in mediation and in land, inheritance and guardianship law, and increasing the institutional capacity of KKTGA staff to develop, execute and evaluate activities implemented under the Oxfam-KKTGA partnership. However, Oxfam and KKTGA failed to establish a partnership which in turn affected the project and resulted in the project objective not being achieved. Ultimately, only one of the two outputs were pursued and completed Project Implementation Outputs Oversight support This output was cancelled. While KKTGA expressed interest in IDLO providing oversight support and advice during the Program design phase, it changed its view prior to implementation, stating that it did not require monitoring or advice in relation to the execution of its 2008 activities. It was therefore agreed between KKTGA, Oxfam and IDLO, that this output would not be pursued and a partnership between Oxfam and KKTGA was not established. Professional development support IDLO delivered three x 3-day workshops for KKTGA, one each in the areas of leadership skills, mediation, and technical legal skills training. All workshops were open to other NGOs in Aceh working in related fields. 17 KKTGA is a local NGO which provides free legal advice, counselling and representation to women in Banda Aceh and Aceh Besar and undertakes legal awareness raising activities. 10

19 Direct Beneficiaries Leadership Skills Workshop: 23 beneficiaries from local NGOs and 9 KKTGA staff; Mediation Workshop: 17 beneficiaries from local NGOs and 8 KKTGA staff; Technical Legal Skills Training Workshop: 14 beneficiaries from local NGOs and four KKTGA staff. 1.2 SUMMARY OF POST-TRAINING EVALUATION QUESTIONNAIRES Leadership Skills Workshop Table 5: Leadership Skills Workshop Results Relevance and Usefulness of Training (KKTGA Staff): Relevance and Usefulness of Training (Other NGOs): Rating % of Responses Rating % of Responses Very Good 11.1% Very Good 50% Good 66.7% Good 41.7% Average 22.2% Average 8.3% Table 5 confirms that issues targeted in the Leadership Skills Training Workshop were relevant and useful for all participants. The sub-topics covered included leadership; ownership; the form, responsibility and role of a leader; leadership typology; communication skills; presentation techniques; meeting management; team building; and social styles Mediation Workshop Table 6: Mediation Workshop Results Indicators Satisfaction of participants with workshop Issues targeted in training are relevant and useful for KKTGA Verifiable improvements in the skills of KKTGA staff in the area of mediation Training modules and information provided to participants is accurate Results 15 participants judged the quality of the workshop as high, and two as medium. Training addressed many issues faced by mediators. Participants requested further training and recommended that the training be held over five days, rather than two days. Participants suggested that workshop materials and case simulations relate more directly to gender issues. The training was very practical, with all participants gaining experience in conducting mediation role-plays and casestudies. The training was delivered by four expert trainers from the Indonesian Institute of Conflict Resolution (IICT). IICT is one of two organizations accredited by the Supreme Court of Indonesia to conduct mediation certification training Technical and Legal Skills Training Workshop Table 7: Technical and Legal Skills Training Workshop Results Indicators Satisfaction of participants with workshop Material is technically accurate Results - 100% of participants (who returned evaluation forms) stated that the workshop was useful for their work % rated the quality of the workshop as high ; % rated the quality of the workshop as medium. The facilitator was a training expert from Jaringan Komunitas Masyarakat Adat (JKMA) and training materials were developed in cooperation with legal stakeholders including judges from the Mahkamah Syar iyah. 11

20 1.3 CONCLUSIONS No follow-up data was collected beyond the immediate post-training feedback from those who received training. Accordingly, it is not possible to draw conclusions about the effectiveness, impact or sustainability of the Legal Aid Project. All of the training workshops were determined to be relevant to beneficiary needs. There was positive feedback regarding the quality of the training and topics covered. The Legal Aid Project did not meet its objective of improving the legal skills of KKTGA staff in mediation and in land, inheritance and guardianship law, and increasing the institutional capacity of KKTGA staff to develop, execute and evaluate activities implemented under the Oxfam-KKTGA partnership. This was due to KKTGA and Oxfam not being able to establish a relationship that would enable IDLO to monitor or advice KKTGA in relation to the execution of its 2008 activities. The project did, however, deliver what beneficiaries considered to be relevant training to KKTGA and staff from other NGOs in Aceh. 12

21 2.0 Community Mediation and Legal Skills Training Project Summary of Evaluation Findings The Community Mediation Project was, overall, highly relevant in relation to community needs and effective in meeting its objective. The project was able to deliver culturally appropriate activities and quality training workshops and effectively engage at a grassroots level, responding and adapting to the needs of tsunami-affected and post-conflict communities in Aceh. While it may be too soon to accurately assess its impact, the findings of the evaluation suggest that the Community Mediation Project may be contributing to a process of change in community attitudes. However, more could be done to ensure the sustainability of the project and its potential to have long-term effects on the lives of individuals and communities in Aceh these are recommended below. Summary of Recommendations For any future implementation or expansion of the Community Mediation Project, it is recommended that IDLO: Include domestic violence as a topic in the training materials (or design and implement a new domestic violence program altogether); Include training and information on the formal justice system in the project modules and involve formal institutions in the project to promote a link between the formal and informal justice sectors; Extend project activities to the broader community (not just community leaders); Use a tailored approach when implementing project activities in more remote and culturally conservative areas of Aceh to ensure greater relevancy for beneficiaries; Develop and implement a strategy to facilitate sustainability of the project by encouraging local ownership and building beneficiary capacity to transfer acquired knowledge; Extend the geographical scope of the project to other areas of Aceh. 2.1 THE PROJECT Overview The Community Mediation and Legal Skills Training Project (Community Mediation Project) was originally implemented under IDLO s Post-Tsunami Legal Assistance Initiative (February October 2007). The design of the project was informed by the results of five IDLO assessment missions to Aceh between December 2004 and January 2006, which identified the positive role village-level dispute resolution processes can play in resolving tsunami-related disputes and how strengthening these processes can result in fairer and more efficient resolution of such disputes. Recognising the continuing relevance of the Community Mediation Project and its potential contribution to the broader development of the justice system in Aceh, Oxfam International agreed to fund its continuation and extension into post-conflict areas after the conclusion of the Post-Tsunami Legal Assistance Initiative. The project content was slightly revised and adapted to the changing circumstances in Aceh. The objective of the Community Mediation Project under Oxfam s funding was to raise the awareness of community leaders in tsunami-affected areas of Aceh regarding legal rights in the subject matter of land, inheritance and guardianship law, and to facilitate the fair and efficient resolution of tsunami- and conflict-related legal issues at the village-level by building the capacity of community leaders to mediate disputes. The project sought to achieve this by providing training to a cross-section of village leadership representatives in a number of villages throughout Aceh. This evaluation report covers the entire Community Mediation Project and considers its relevance, effectiveness, impact and sustainability. To mitigate bias and ensure credibility 13

22 of evaluation findings, IDLO assessed these criteria by cross-referencing data collected from: Participants immediate pre- and post-training tests; Participants immediate post-training evaluation questionnaires; training reports; Interviews with key legal stakeholders; A legal awareness survey; and Interviews with those who received training under the project at least three months after the training Project Background Access to fair and impartial justice and effective legal remedies are fundamental human rights provided by the Universal Declaration of Human Rights (UDHR) 18 and a number of other international human rights instruments 19. The rule of law and an effective justice system serve to ensure that individuals human and legal rights are realized and guarantee individuals entitlements to legal remedies. 30 years of separatist conflict in Aceh have created a situation where justice mechanisms are far from fully developed and the stability of the rule of law has been weakened. Individuals and communities are often unable to obtain satisfactory resolutions through informal and/or formal legal mechanisms and have, as a result, lost confidence in justice providers to protect their legal rights. Strengthening the rule of law and justice systems in Aceh is contingent upon the development of a legal culture within which individuals and communities are aware of and understand their rights, value these rights, and have confidence in the mechanisms through which these rights can be claimed and enforced. Since the vast majority of disputes in Aceh are resolved at the village level through informal customary processes (adat), a first step towards strengthening the legal culture and increasing confidence in justice mechanisms is to ensure that these processes are fair, impartial, participatory and rights-based. This is especially pertinent in Aceh since: The tsunami has created a drastic increase in the number of legal issues, particularly in relation to land, inheritance and guardianship; and The end of years of conflict has gradually resulted in an increased awareness among individuals of their rights and obligations as citizens, generating higher demand for the attainment, protection and enforcement of these rights. Primary impediments to more effective informal dispute resolution processes are that village leaders and mediators often have insufficient knowledge of: The applicable law, both formal and informal; The alternative options and services available (including services provided by the formal justice sector) to resolve disputes/grievances, especially when to refer matters to the formal justice sector for resolution; and Appropriate mediation techniques to assist them achieve fair and consistent dispute resolution outcomes 20. The Community Mediation Project sought to respond by: Raising legal awareness and knowledge among community leaders in a number of tsunami-affected and post-conflict areas of Aceh, particularly in the key areas of land, inheritance and guardianship; 18 Article 6: Everyone has the right to recognition everywhere as a person before the law. Article 7.: All are equal before the law and are entitled without any discrimination to equal protection of the law. Article 10: Everyone is entitled in full equality to a fair and public hearing by an independent and impartial tribunal, in the determination of his rights and obligations and of any criminal charges against him. 19 Including the International Covenant on Political and Civil Rights; the Convention on the Elimination of All Forms of Racial Discrimination; and the International Covenant on Economic, Social and Cultural Rights. 20 UNDP, in its Report Access to Justice in Aceh: Making the Transition to Sustainable Peace and Development (2006), 51-52, 79,

23 Strengthening village leaders mediation skills, and Building capacity of village leaders to resolve disputes fairly and in accordance with legal and human rights standards Project Implementation Direct Beneficiaries Beneficiaries were a cross-section of influential community leaders who play a role in village dispute resolution, including geuchik (village chief), imam meunasah (religious scholar), tuha peut (elder) and a range of women, youth and religious leaders. The aim was to reach not only the traditional male-dominated hierarchy but to provide broad-based capacity building and empowerment to future leaders and women. IDLO established a list of participant criteria including literacy, high-school education, above 21 years of age, experience in village-level structures or organizations, and a minimum target of 25% female participation. Table 8: Composition of Mediation Project Participants Position No. of Participants Religious Leaders 330 Women Leaders 328 Youth Leaders 135 Village Leaders 648 Village Elder 440 Community Member 145 Total 2,026 Outputs IDLO originally planned to conduct: 50 x 3-day training workshops in Banda Aceh, Aceh Besar, Aceh Barat and Aceh Jaya for approximately 1,250 beneficiaries, as follow-up workshops to those delivered under its Post-Tsunami and Legal Assistance Initiative; and 10 training workshops in new target villages in Bireuen for approximately 250 beneficiaries (for a total of 1,500 direct beneficiaries). However, the actual output totaled: 33 follow-up 3-day training workshops; and 48 new training workshops conducted between April and August 2008, for a total of 81 training workshops benefiting 2,026 individuals in 320 villages. The reasons for the changed outputs were: During the first phase of implementation, it was evident that guaranteeing the attendance of beneficiaries who had previously participated in IDLO s trainings under the Post-Tsunami and Legal Assistance Initiative was difficult as village heads (geuchiks) were ultimately responsible for determining participants and rarely selected prior beneficiaries. 21 The main reasons given by village heads for this selection trend included that: - Those occupying the relevant village positions (community leaders, mediators) had changed since the 2007 training; and 21 In the first two cycles of training, the percentage of participants who had attended previous training was only 32% in Banda Aceh and Aceh Besar and 27% in Aceh Jaya. 15

24 - It was more beneficial that a wider audience receive the benefit of the training. Consequently, a decision was made that when the majority of participants were new, trainings would be based on the first-round model, rather than the follow-up model. It was possible to provide 21 additional training workshops beyond to those originally planned in Bireuen, extending to a wider number of post-conflict areas not affected by the tsunami (namely, Aceh Utara, Pidie, Pidie Jaya and Aceh Besar), since savings were made in other sections of the Community Mediation Project and because AIPRD-LOGICA provided supplemental financial support. Module Development The materials used were based on those from the Community Mediation Project under the Post-Tsunami Legal Assistance Initiative which were informed by extensive field-based research conducted in the first half of As a number of revisions and changes to the relevant law occurred since the development of these materials, IDLO organized a two-day workshop (16 _ 21 January 2008) to revise training materials to reflect changes and to include more in-depth coverage of certain topics. Workshop participants comprised IDLO staff and project trainers (plus two prospective trainers) while staff from Oxfam were unable to attend due to prior commitments. The revision of the materials was also informed by the conclusions drawn from the M&E Report and based on additional consultations with Oxfam, Mahkamah Syar iyah judges and other key legal stakeholders. Trainers IDLO employed a pool of expert local trainers consisting of legal professionals, university lecturers and Mahkamah Syar iyah judges. The trainers participated in an intensive 3-day Training of Trainers program which focused on participatory and experiential learning techniques and community facilitation skills. In addition, IDLO trainers participated in a workshop delivered by four expert trainers from the Indonesian Institute on Conflict Resolution (IICT) 22 in order to facilitate their accreditation as certified mediators by the Supreme Court of Indonesia. Out of the 14 trainers that attended the workshop, ten received accreditation. 2.2 EVALUATION Table 9: Summary of Evaluation Tools and Indicators Community Mediation Project Project Objective Success Indicators M&E Tools Community Mediation and Legal Skills Training To raise the awareness of community leaders in tsunami-affected areas of Aceh regarding legal rights in the subject matter of land, inheritance and guardianship law, and to facilitate the fair and efficient resolution of tsunamiand conflict-related legal issues at the village-level by building the capacity of community leaders to mediate disputes a. At least 70% of participants increase their knowledge of key elements of the training course b. Feedback regarding the relevance and effectiveness of mediation training from key legal sector stakeholders c. Participants find the training effective, relevant and appropriate d. 60% of interviewed participants provide narrative examples of the practical application of techniques and principles from the training a. Interviews with key legal stakeholders b. Legal awareness survey conducted at least 3 months after training c. Interviews with participants at least 3 months after training d. Participants pre and post training knowledge evaluation tests e. Participants posttraining qualitative evaluation of training program and facilitators 22 The IICT is one of two organizations accredited by the Supreme Court of Indonesia to conduct mediation certification training. 16

25 2.2.1 Relevance In order to determine the relevance of the Community Mediation Project, three sources were utilized: Feedback from interviews with key legal stakeholders; Feedback from immediate post-training questionnaires; and Feedback from interviews with a sample group of those who received training under the project, interviewed at least three months after the training. By analyzing and comparing these three sources, conclusions can be drawn on the relevance of the Community Mediation Project, intended as a combination of: The validity of the project s objectives in relation to the Acehnese context and to changing circumstances; Correspondence between the topics addressed in the project and the most significant legal issues confronting Acehnese communities; and Applicability and usefulness of the materials of the project in community dispute resolution processes. Stakeholder Interviews 23 Interviews with representatives of key legal stakeholders confirmed the relevance of the Community Mediation Project throughout all areas of Aceh. All stakeholders indicated that land, inheritance and guardianship continued to be the most common and significant legal issues facing Acehnese communities in tsunami-affected and post-conflict areas. Stakeholders also emphasized that although legal disputes resulting from the tsunami had gradually decreased in the past year, land and inheritance disputes resulting from the end of the 30-year conflict were increasing. A number of stakeholders emphasized that the high number of disputes related to land, inheritance and guardianship following the tsunami were primarily the consequence of a pre-existing lack of rule of law in Aceh generated by 30 years of isolation due to the separatist conflict. For example, the majority of legal disputes regarding land division after the tsunami arose from a general absence of land registration prior to the tsunami. Similarly, as noted by one stakeholder, 75% of land in non-tsunami-affected post-conflict areas is currently unregistered, creating significant and potential land division disputes for individuals and families returning to their villages after having fled the conflict for years. A participant from IDLO s training in Aceh Utara, an area heavily affected by the earlier separatist conflict, clearly expressed the shared needs and problems amongst tsunami and non-tsunami affected areas in Aceh, stating: emerging from 30 years of conflict and isolation in many aspects is similar to emerging from the effects of a disaster. Two stakeholders stressed that the topic of land will continue to be relevant for many years to come and noted that the changing nature of the issue may potentially generate future legal disputes and dilemmas (example, status of land allocated to tsunami survivors during relocation). Stakeholders confirmed the validity the project s objective and the project s impact on reducing the formal courts caseload. In Aceh, dispute resolutions made at the village level can be endorsed by the formal courts, provided they are in accordance with Indonesian law 24. By building the capacity of village leaders and mediators to make fair and just decisions, the project facilitates this endorsement process and reduces the number of cases brought directly to the formal courts. As stated by one stakeholder: Villages have always had their own dispute resolution processes that people traditionally refer to. It is just a question of upgrading them and strengthening the skills and capacity of mediators. Only by doing this can decisions be recognized by the formal legal system and village level dispute resolution processes continue to function, while guaranteeing justice. 23 See Annex 2 for list of stakeholders interviewed. 24 As noted by a judge from the Mahkamah Syar iyah. 17

26 However, there were some criticisms and diverging opinions among stakeholders regarding the relevance of IDLO s work. Some believed that IDLO had not given enough attention to post-conflict areas not affected by the tsunami, suggesting that IDLO prioritized training locations based on logistical convenience rather than community needs. On the other hand, one stakeholder believed that IDLO should avoid working in communities heavily affected by the conflict, suggesting these communities would be resistant to the topics addressed in IDLO s trainings. Participant Feedback Immediate Post-Training Evaluation Questionnaires On the last day of each training workshop, participants were required to complete an evaluation questionnaire. The following is a summary of responses to two key questions which indicate the usefulness and practicality of the Mediation Project: GRAPH 1 Question 1 (immediate post-training): Did the training meet your expectations, in that it will help you solve disputes you face in your village? 6% No 94% Yes While there was a 94% positive response to Question 1, it must be recognised that the questionnaire was completed immediately after the training and before participants had the opportunity to apply the skills and knowledge in practice. For this reason, IDLO interviewed participants at least three months after the training in order to obtain a stronger indication/confirmation of the practical utilization of information and skills from the training (See Graphs 3 and 4) GRAPH 2 Question 2 (immediate post-training): What was the most useful topic you learned during the training? No. of Responses Guaridanship Inheritance Land Mediation Syar iyah Islam 18

27 The responses to Question 2 clearly indicate that the information on inheritance law was the most valued, followed by information on guardianship, mediation and land law. These results reflect the responses provided by participants in the M&E Report when asked what topics they would like to receive further information on. 25 The fact that these results correspond confirms that: - Community expectations were met; - The trainings were relevant to the needs of participants; - The continued relevance over time of the topics covered in the training; and - IDLO built on community feedback from its previous M&E Report. Participant Interviews (at least three months after IDLO activities) A further confirmation of the continued relevance and usefulness of the topics addressed by the Community Mediation Project can be inferred from the feedback of a sample group of those who received training under the project, interviewed at least three months after the training occurred. This also provides an indication of the applicability of topics, considering the period of time which had passed since the trainings were conducted. Following is the summary of the responses that the sample group provided to key questions: GRAPH Question 1 (at least 3 months after training): What was the most relevant, practical and useful topic of the training?" 42.3% No. of Responses % Land Inheritance 20.5% Guardianship 10.3% 7.7% Mediation All Topics Graph 3 is confirmed that at least three months after the training, inheritance law (42.3%) remained the most relevant, practical and useful topic, followed by guardianship (20.5%), land (19.2%) and mediation (10.3%). 25 Except for Mediation, which was not considered a topic in the evaluation of the Post-Tsunami Legal Initiative. 19

28 GRAPH 4 Question 2 (at least 3 months after training): What was the second most relevant, practical and useful topic of the training? % No. of Responses % Land 13.5% 11.5% Inheritance Guardianship Mediation When asked to list the second most useful topic, 58% responded mediation, 19.2% land, 13.5% inheritance, and 11.5% guardianship. Since these results are close to polar opposite of their first preference, it can be inferred that all topics were relevant, practical and useful to participants. For example, based on Question 1, mediation seems to be of little relevance to participants (10.3%); however, in Question 2 it appears to be the most relevant topic (55.8%). This may be attributed to the interrelation among topics, a point also highlighted by beneficiaries during interviews. Although the interviewer asked participants to choose one topic as the most relevant, 7.7% of interviewees stated they were unable to do so as all were related and equally relevant, useful and practical (see Graph 3). A further indication of the interrelation among topics, is that as many as 40% (not 7.7%) of participants would have responded All topics had the interviewer not insisted it was preferable that beneficiaries select one topic only. Participant Quotes on Relevance Keude Teunom Village - This training was so important for me. It is very rare for a training of this kind to occur at the village level. This is the first legal training ever. This kind of training should be done in as many communities as possible. Merudati Village - Inheritance is such a huge issue after the tsunami. Before the tsunami and during the conflict period we used to have one or two cases a month. Now we have people coming every day to enquire about inheritance matters. This happens still, four years after the tsunami and the end of the conflict. Although the cases may be slowly decreasing, the problems related to the division of land are increasing, as they are related to inherited land. Once it has been established whether a person is entitled to an inheritance, then it is time to establish how much of the land he gets, determine the borders, etc These issues happen continuously. Alue Ambang Village After the tsunami, we had so many orphans to take care of. Guardianship and inheritance are interrelated so it is so important to receive training on both. For the well-being of orphans it is important that they get the inheritance they are entitled to. Penne Let Baroh Village - As the Imam*, I am the first person people come to solve their problems, especially related to guardianship, inheritance and land division. That is why this training was useful for me. Merudati Village - Since I am the Imam, all the community comes to me first if they have a problem. Therefore, all the topics of the training were important for me. The topics are so interrelated. It (the training) should be extended to all Imams in Aceh, since the community goes to them first. It should also be done for the general community and especially young people, who will soon become the next generation of leaders and mediators. It is important to solve disputes at the village level since the formal system can take years and the outcomes do not always provide a winwin solution. For this reason it is crucial to strengthen these village level mechanisms and make them more efficient and make sure they are fair. For this reason I am so grateful. *An Imam is an Islamic scholar 20

29 GRAPH 5 Question 3 (at least 3 months after training): What topics would you like to receive more information or further training on?" 30 No. of Responses In-depth of same topics Adat Land Marital Law Mediation Formal Law Guardianship Inheritance Women's Rights Islamic Law When participants were asked at least three months after the training which topics they would like to receive further information on, the most common response was marital law. Through informal discussions with participants, it became evident that domestic violence is a growing concern in several areas of Aceh and that village leaders and mediators have limited skills to deal with this issue. Considering the topic of domestic violence was also raised by communities in the M&E Report , IDLO might wish to include this topic in any future community mediation programming in Aceh. 26 Graph 5 also shows that participants requested more information on both adat and formal/government law. While the former confirms the results of the M&E Report , the latter represents a more than 10% increase (only 5% of interviewees in the M&E Report requested more information on formal/government law). From informal discussions with participants, the request for more information on formal/government law appeared to come from the end of years of isolation resulting from the separatist conflict and the subsequent progressive realization of the existence of formal rights guaranteed by the government. In addition, a number of mediators and village leaders stressed the importance of understanding the formal law and procedures in order to be able to deliver fairer decisions and refer disputes to appropriate fora when resolutions cannot be reached at the village level. The need to create a bridge between the formal and informal justice systems in Aceh was also emphasized by a number of stakeholders, including a prominent Mahkamah Syar iyah judge. It is therefore recommended that IDLO address this issue in future programming. 26 It should be acknowledged that IDLO added a component on domestic violence the Film Project. 21

30 Participant Quotes on the Need for Further Training Padang Klieng Village - There are a lot of cases in which people think they are right and they pursue their own ways. Then when they refer to the formal justice system, they realize they are wrong and what they are doing is illegal. That s why I think mediators should receive more training on the formal law and this training should be extended to all the community. This will save time, money and despair among the population. We, as mediators, also need to know more about the formal justice system: how it works, who to refer to for assistance and advice and to understand where our jurisdiction ends and where the formal system s jurisdiction starts. Keudah Village - I need more in-depth training in mediation to raise my confidence. In this way I can mediate more important cases, especially regarding women, and I can defend their rights. Merudati Village - Cases of domestic violence have increased dramatically. There have already been 15 reported to me this year, and I am sure there are many more that don t come out. Neuheun Village - I have seen IDLO s film on women s legal rights and I would like more training on the topics related to women that came out in the film. They are crucial in our community. Matang Kumbang Village - Since we live in a post-conflict area, people are totally unaware of their rights. They do not know what it means to be a citizen and have rights and duties. 30 years of conflict has erased this. Not only we need training, but the entire village needs training on legal rights and we need to raise the general legal awareness of people Effectiveness In order to determine the effectiveness of the Community Mediation Project, four sources were utilized: Feedback from interviews with representatives of key legal stakeholders; Feedback from immediate post-training questionnaires; Feedback from interviews with a sample group of those who received training under the project, interviewed at least three months after the training; A Legal Awareness Survey conducted among the general public in Banda Aceh and Aceh Besar and interviewed participants. By analyzing and comparing these four sources, conclusions can be drawn on the effectiveness of the Project, intended as: The extent to which the project achieved its objectives and is contributing to the aim of IDLO s overall Program; Participants have retained skills and knowledge and apply them in practice; The accuracy of the content in the mediation training materials and of the information provided; The validity of the project s strategy; The satisfaction of project beneficiaries with IDLO training methodology, facilitators and approach; and The increased legal awareness of direct project beneficiaries in comparison with the legal awareness of the general public in Aceh. Stakeholders Interviews 27 Representatives from key stakeholders agreed that IDLO s strategy of working at the grassroots/village-level was an effective and valid way to: Contribute to the goal of IDLO s Program to strengthen the rule of law in Aceh; and Achieve the objective of the Community Mediation Project especially considering the context in Aceh where villages are often situated in remote areas, with little or no access to information, services and media. As stated by one stakeholder: 27 See Annex 2 for list of stakeholders interviewed. 22

31 IDLO s training is rooted at the grassroots level and it starts at the bottom of the problem, directly in the community. All stakeholders emphasized the importance of direct awareness-raising (that is, bringing training to villages) as it allowed communities to directly raise their concerns and receive immediate face-to-face feedback. Stakeholders also noted the effectiveness of engaging local trainers with Acehnese language skills, since many villagers in remote areas do not speak Indonesian. Stakeholders agreed with IDLO s strategy to target a cross-section of village leadership representatives (such as religious leaders, youth leaders and women leaders) rather than only village chiefs, as an effective means to contribute to improving justice processes at the village level. Some stakeholders, however, believed that the project may have been more effective if formal legal institutions had been more actively involved in implementation or if trainings had provided more information on the formal legal system as this would bridge the gap between the formal and informal justice systems. Since this issue was also raised by beneficiaries, it is recommended that IDLO address the existing gap between informal and formal justice systems in future programming. Some stakeholders also commented that the project may have been more effective if it had targeted the community at large rather than community leaders only, as the topics are relevant to the lives of all village members. This, too, was echoed by a number of beneficiaries during interviews. On the accuracy of the project training material, several stakeholders said they did not feel comfortable providing an opinion as they had not analyzed the materials in detail. Of those who had reviewed the materials, all confirmed their technical accuracy. However, the strongest assurance of the accuracy of the materials was the participation of Mahkamah Syar iyah judges in their development. Participants Feedback Before and after each training workshop, participants were required to complete a multiple choice test comprising 20 questions on land, inheritance, guardianship and mediation. Both sets of questions were identical in order to assess the improvement of their knowledge on these issues. Relative to the pre-training test, 82% of participants improved their results in the posttraining tests. This was significantly higher improvement percentage relative to the 70% improvement indicator established for the project. The average improvement in participants scores was 22%, which translates to participants being correct on an average of 3.5 additional questions. Though this may seem statistically insignificant, when tested on fundamental issues, such as women being included on land certificates, the responsibility and duties of guardians or the possibility of a woman becoming a guardian, such improvement can have far-reaching results, contributing to improving the lives of people and enhancing their ability to attain fundamental rights. Table 10 (following page) provides two examples of questions related to fundamental issues asked in both pre- and post-training tests and the percentage of improvement in correct answers provided by participants: 23

32 Table 10: Examples of Pre- and Post-Test Improvements Question 1: The following are people that can be guardians. Choose the most correct answer: a. Male who has a blood a relation to the father b. Male and Female c. Female In the pre-test, 19% of participants answered this question correctly. After the training, 54% of participants answered this question correctly. Question 2: A woman (girl or widow) can registered/ legalise land property on behalf her self. For married woman its best if the land which is jointly owned is registered: a. On behalf of the wife b. On behalf of the husband c. On behalf of the husband and wife In the pre-test, 51% of participants answered this question correctly. After the training, 74% of participants answered this question correctly. An important indicator of the effectiveness of the Community Mediation Project is participants self-perceived levels of improvement in mediations skills and their increased confidence in dealing with complex disputes. Of the participants interviewed at least three months after the training, 67% had been mediators prior to the training. These participants were asked to rate their perceived level of improvement in mediation skills after participating in IDLO s training, on a scale from 1 to 3, with 3 indicating poor-level of improvement or no improvement ; 2 indicating medium-level of improvement and 3 indicating high level of improvement. Following are the results: Table 11: Participants Perceived Level of Improvement in Mediation Skills Level of Improvement Percentage of Participants 1=no or poor 0% 2=medium 18% 3= high 66% When asked to elaborate on the above responses, the general consensus among participants was that the training increased their confidence in mediating more complex disputes. Over 37% of participants stated that they are able to mediate and solve disputes more efficiently and quicker because of the training. The remarkable aspect of these responses is that all participants used these precise terms (that is, more efficiently and quicker ) to indicate their improvement, without any leading from the interviewer or interpreter. Other interviewees, instead, responded that they have been able to resolve a higher number of disputes with win-win solutions and with a higher degree of satisfaction for both parties since the training. In addition, 10% of mediators stated that they were able to reach a solution to an on-going dispute (commenced prior to IDLO s training) immediately after participating in the mediation training. A strong indication of the effectiveness of the Community Mediation Project in terms of the extent to which knowledge and skills have been retained and regularly used to solve disputes is that 69% of participants interviewed at least three months after the training were able to identify a case that either they or others had resolved using the principles and techniques delivered during IDLO s trainings. Moreover, many interviewees reportedly refer to the training materials on a regular basis and some take the materials with them to 24

33 mediations to assist should difficulties arise. Interviewees noted that these materials are useful to explain complex issues and laws to disputing parties. However, the evaluation of the Community Mediation Project included interviews only with direct beneficiaries (community leaders) who may have entertained some bias in regards to their mediation skills and to the fairness of dispute resolution processes and outcomes. Evaluating the effectiveness of the project would have been enhanced had members of the general community been included in interviews for comparison of their responses to questions with those of village leaders. Participant Satisfaction with the Quality of Facilitators, Methodology and Approach: Participants perception of the quality of course facilitators and the training methodology and approach can also provide an indication of the effectiveness of the project. Where participants do not feel comfortable with the course facilitators, where facilitators are unable to effectively communicate complex issues, or apply inappropriate teaching methods, there is less of a likelihood that participants will understand and retain knowledge, be open to learning or meaningfully participating in training activities. Further, the effectiveness of the training is likely to be compromised where participants feel that the training approach is not culturally appropriate or respectful to local customs and is a foreign imposition of concepts and/or methodologies. Post-training evaluations indicated that 94% of the participants rated the facilitators as good (highest possible rating), while 6% rated the facilitators as average and 0% as poor. The overwhelmingly positive response suggests IDLO facilitators were effective in delivering project activities and contributing to the overall objective the project. GRAPH 6 Quality of IDLO Facilitators (immediate post-training) 6% Average 94% Good When participants were asked at least three months after the training if there were any negative aspects of IDLO s training, 100% stated the training was overall extremely positive. More in-depth discussions revealed several positive trends, lending further weight to the effectiveness of the training, including: Participants felt comfortable to ask questions and participate in interactive activities; The majority valued the use of role plays and some suggested this technique should be more widely used during training sessions; The training was simple to understand for everyone, regardless of level of education and literacy; Participants generally appreciated the fact that facilitators were locals, able to communicate in Acehnese (not only Indonesian) and aware of the context and needs in villages. The majority of participants revealed they had developed a very positive relationship with the facilitators and some continue to contact them for advice. 25

34 When interviewees were asked whether the training was culturally appropriate and respectful to local customs, 100% responded positively. The general feeling among interviewed participants was that the training and concepts covered were not imposed but introduced in a respectful manner and through dialogue. Some participants mentioned a change in personal views as the training progressed. They conceded being initially skeptical about the validity of certain information presented by IDLO facilitators but came to realize by the end of the training that the new information was more beneficial to the community than some rules traditionally applied. Many participants stated that after learning such information, they had stopped (or would stop) implementing various traditional norms and procedures, such as preventing women from being legal guardians of children and the rule of Patah Titi. The Patah Titi rule, in particular, was extensively referred to by participants and is a sensitive topic in many remote areas of Aceh. Patah Titi is an Acehnese custom which precludes a child from receiving family inheritance if his/her parents predecease the grandparents. The practice has been banned by the Mahkamah Syar iyah and is no longer implemented in the formal legal system. However, many remote villages still apply Patah Titi as they are unaware of this fact and of alternative ways to deals with the issue the Patah Titi addresses. In these regards an interviewee (a village leader) stated: by improving my mediation skills and knowledge on land and inheritance topics, I no longer use the Patah Titi rule, as I have learned new and better ways to deal with land division and inheritance issue. The techniques I have learned in IDLO s training allow me to solve disputes in a fairer way, still in line with Islamic law. Although all participants interviewed believed that IDLO s training was overall positive, some provided minor critiques and suggestions on how to improve the training which are summarised below: Training should not be stand-alone and follow-up trainings should be provided on a regular basis, since laws often change and new issues arise. Training should be carried out in more comfortable places. Some participants, particularly those from the area of Aceh Utara and Pidie Jaya, lamented that training venues were too hot, stuffy and uncomfortable for such a long training. Training should be carried out separately in each village, rather than bringing together participants from different villages. Training should be extended to all village members and not only to the leaders. Many participants believed that it is important for all village members to understand the issues taught, particularly the youth who constitute next generation of mediators and village leaders. Training should be longer since Islamic law is complex and often requires a significant amount of time for explanation and analysis. Legal Awareness Survey As part of the evaluation exercise, a legal awareness survey was conducted among a random sample of 250 members of the general public in Banda Aceh and Aceh Besar. The same survey was conducted among a sample group of Community Mediation Project direct beneficiaries at least three months after the training had occurred. By comparing the results between audiences, it is possible to draw some broad conclusions on the effectiveness of the project. The survey included three basic questions on common legal issues in Aceh in the areas of land, inheritance and guardianship (See Annex 5 for the complete survey): - Can women be appointed as guardians of children? - If one spouse dies what should happen to property acquired during the marriage? - Can a land title (certificate tanah) for a house which is jointly owned by a husband and wife be jointly registered in both the husband and wife s name? 26

35 Table 12: Legal Awareness Survey Results (select questions) Audience Percentage of correct answers Question 1 Question 2 Question 3 Community Mediation Project Beneficiaries 60% 36% 79% General public 48% 48% 44% These results suggest that the direct beneficiaries of the Community Mediation Project have a considerably higher level of legal awareness than the general public in two of the above questions (on guardianship and land law). However, this is not the case for Question 2 (on women s rights to inheritance) where beneficiaries of IDLO s mediation training maintained a lower percentage of correct answers compared with the general population. However, closer analysis of the data indicate suggests that a large majority of incorrect responses was given by participants from some of the more remote post-conflict areas of Aceh which are known to hold more conservative attitudes towards customary practices (due to prolonged isolation) relative to other parts of Aceh. Therefore, it is reasonable to assume that there may have been a greater resistance to accepting the information provided in the training, particularly in regards to sensitive topics, such as the inheritance rights of women. In light of the above, it is recommended that IDLO consider a set of materials for remote post-conflict areas that are specially tailored for the context. While the results of Question 2 may suggest that IDLO s training materials may have failed to adequately cover the issue of women s inheritance rights generally, this does not account for the fact that the negative results were predominately concentrated in postconflict areas. Participant Quotes on Effectiveness Merudati Village - Although an experienced mediator (I work with the community eight hrs a day), the training really assisted me. The mediation skills I learned allow me to solve disputes, negotiate and mediate in a more sensitive manner. My skills have dramatically improved. All the cases I have mediated since the training have been solved. None have gone to the court. I speak to people better and mediate more efficiently. I am really and honestly grateful to IDLO. I would have been able to acquire this knowledge only through university. This would have been very expensive and complicated for me to do. Alue Ambang Village - Since the training, I mediate cases quicker and more efficiently. After the second training, people have an increased trust and confidence in me since I have been mediating disputes with win-win solutions. I have several cases and I feel more confident. There is more justice in the village now. Ulee Kareung Village - I was a mediator before the training, but I really didn t know the proper techniques to solve issues. I would just use common sense. But since the training, I have learned how to apply mediation techniques, and the results are incredible. I mediate much easier now, I provide a greater number of win-win solutions and people are generally happy with the solution to disputes. Things get solved much quicker now and disputes and solutions remain at the village level. Ulee Gampong Village - Before the training, I sometimes was unable to solve issues related to inheritance and property. Islamic law is very complex on this matter and some mediators follow traditions which are not applied in Indonesia anymore and which are not fair for the people. Through the training, I am now able to solve disputes that I wasn t able to before and I do it in a fairer way. Padang Klieng Village - Since the training, not only am I more efficient and quicker, but I have much more confidence in mediating more complex cases and in challenging old customs which negatively affect women. People trust me and listen to me. Keudah Village - I feel so lucky that I have received this training. Before we were not able to solve a lot of disputes that arose in the village, and now we solve them so easily, especially regarding inheritance (I work in the preparation of inheritance documents). It is such a big change. Everyone in the village over 25 should receive this training. Pande Village - I became village leader only a few months before the training. In that period I had no confidence in mediating and I had great difficulties. Then I did the training and things became much easier. Now I am confident and I solve disputes with win-win solutions much quicker. Lamnga Village - Since the training, I mediate more efficiently and quicker. I must say that since the training I have observed a higher number of cases being solved at the village level rather than going out to the formal courts. Mediation through a win-win solution is less expensive, quicker and the preferred option in the village. IDLO training has helped to consolidate this. 27

36 2.2.3 Impact To determine the impact of the Community Mediation Project (that is, its long-term effect and the extent to which it has contributed to changing people s lives) one main source was utilized feedback from interviews with a sample group of participants who received training under the project, interviewed at least three months after the training. It is difficult to fully assess the long-term effects of any activity only three months after delivery. With this caveat, the results of the evaluation reveal that the project has potentially triggered a process of change in the dispute resolution processes at the village level which may have a lasting impact on the community and the lives of individuals. The majority of participants (69%) interviewed at least three months after the training offered evidence of this by providing a number of narrative and anecdotal examples of cases that demonstrate the continued application of the information and skills learned through the project. This suggests that dispute resolution procedures and outcomes at the village level may be positively changing. The following are a selection of narrative cases that support this idea and which participants have indicated as constituting precedents at the village level. Narrative Examples of Cases from Participants Matang Kumbang Village -Because he was so enthusiastic and proud, as soon as the village leader met the interviewer, he began telling the following story: A long time before the training occurred, and especially after the tsunami, a high number of cases had not been solved fairly, as it was believed that Patah Titi* still ruled in Aceh. So many children were not receiving inheritance from their grandparents because of it. But since the training, I learned that the Patah Titi rule is not used in most of Aceh anymore and that it has been banned by the government. So, after the training I went to all of those children and families, one by one, and gave them the inheritance they were entitled to. It was so satisfying to solve all these long-standing issues and tensions in the community and to contribute to improving the lives of those children. All the families were so happy and satisfied, finally. Alue Ambang Village - When there is a problem or a dispute in the village we start discussions at the family level. If this doesn t work, the case is brought to the sub-village level. If this doesn t work it then goes to the village level, involving the village and community leaders. After that, it goes to the formal justice system. Before the training, most disputes went as far as the village level, involving directly the village leader. However, after the training this trend has changed, and the majority of disputes reach the sub-village level and are solved through mediation. This has happened because mediators are now able to solve disputes more efficiently and fairly, through a win-win solution. There is more justice in the village now. Matang Kumbang Village - There was a child in the village whose parents died. His grandparents died soon after and there was no one to take care of him. According to the Patah Titi rule the assets of the grandparents could not be inherited by the child. We (mediators and village leaders) became concerned of the well-being of the child, but could not find a solution. He needed a guardian, and we looked for one, but could not find a male who would take this responsibility, as we thought only males could become guardians. This continued for over a year. Then we attended IDLO training and we became aware that first of all the Patah Titi rule is no longer applied by the government and by many villages in Aceh. The child, therefore, received the assets of the deceased grandparents. Secondly, we solved the issue of the guardian, as through the training we became aware that women can become guardians too. A female family friend is now the child s guardian and is taking care of his property until he turns 18. Keudah Village - When my father died in the tsunami, he left everything to my mother. Shortly after my mother passed away, too and she had always told me that she wanted her assets and property to be divided equally between my 2 brothers and me. However, she never wrote a formal will stating this. My brothers took the majority of the assets and gave me only a small portion of the property. In addition, they are not sharing the money they are earning from renting some land and two houses which were my mother s property. Initially, I raised the issue with someone in the village who knew about legal issues, but I was not satisfied. Then I attended the training and I understood what I am entitled to and how I can potentially solve the issue and all the background information to support my case. I still cannot convince my brothers, as they cannot accept the equal division principle, but at least now I know what I can fight for and I know I have the right to equal inheritance, even though I am a woman. I am still pursuing the case. *Patah Titi is an Acehnese custom which precludes a child from receiving family inheritance if his/her parents predecease the grandparents. The practice has been banned by the Mahkamah Syar iyah and is no longer implemented in the formal legal system. 28

37 Cont. Narrative Examples of Cases from Participants Pande Village - An entire family made up of husband, wife and 5 children were killed in the tsunami. Initially, the property and assets of the family went to the husband s brothers and sisters, based on the fact that the husband was considered the sole owner of all the assets since he was a male. These claims were supported by the village justice system, including myself. The wife s relatives believed this was unfair and brought the case to the formal justice system. However, the formal system is fairly slow. In the meantime, the village leaders and I received the training from IDLO and we realized that the claims of the wife s relatives were plausible. In fact, it was only based on traditional practice that the husband was considered the sole owner of the family s assets. In reality, since there was no formal document establishing this, we realized that the law provides that the wife is considered owner of half of the family assets. The community leaders approached the husband s relative and suggested they give part of the inheritance to the wife s family. After several weeks of discussions and negotiations, the husband s relatives agreed to give 1/3 of the property to the wife s family. Although not an equal division, the wife s family was satisfied and withdrew the case from the formal court. Gampong Baro Village - A child in the village lost his parents in the tsunami. The relatives on the father s side claimed the child s property but were not interested in caring for the child at all. They brought the case to the village. I began mediating the case and was unsure how to solve it, as I didn t know who could become a good guardian for the child and look after the child s inheritance. It was only after the training that I discovered a female can become a child s guardian. So, I asked the mother s sister (the child s aunty) if she would agree to become the guardian. She agreed and she really cares about the child s well-being. The child is now pursuing an education and his property is safe until he is 18. I would not have known this information if I had not participated in IDLO s training. Further informal discussions with participants also suggested that the Community Mediation Project has the potential to affect power structures within communities. In this regard, all female participants interviewed for the evaluation at least three months after the training who were not mediators, stated that although some cultural barriers exist, the training had dramatically increased their chances of becoming a mediator and participating more actively in village leadership. One female participant said that since attending the training, the village leader had asked her to act as an observer/advisor on some mediated cases. Within her village, this has contributed to promoting a new attitude in regards to the role and status of women in the community. The participant also said that she is often approached for advice on disputes before they are formally submitted to the village leader for mediated resolution Sustainability To assess the sustainability of the Community Mediation Project (defined as the transfer of project ownership to the local community to ensure long-term impact) a sample of participants were interviewed at least three months after the training. Participants were asked how they have used or plan to use the knowledge and skills acquired in the future. A summary of responses showed: 23.2% had not shared the knowledge or skills at all; 38.5% had shared the information (knowledge) through casual conversations with family and friends; 29.5% intend to share the knowledge and skills only if someone comes to them with a problem; 5.8% had shared information (knowledge) during weekly religious group discussions; 5.8% had organized a formal activity to pass on knowledge and skills; 3.8% said that information (knowledge) and skills would indirectly pass to others in the community through their practice. A large portion of participants (23.2%) had not shared the knowledge and skills acquired in any way at the time of the evaluation exercise. Discussions revealed that these participants either had not considered the importance of sharing the information and skills or preferred to retain the information and skills for personal use. Nevertheless, 64% of participants believed that delivering a formal training activity would be the most effective way to pass on information and skills to the community. The majority 29

38 noted, however, that without additional support, they had neither the confidence nor the qualifications to deliver such activity. This suggests that if adequate support is provided, there exists a large group of community members willing to share information through formal activities, thereby improving the sustainability of the Community Mediation Project. It is therefore recommended that any further programming on community mediation include a strategy to, inter alia: Support participants to share their knowledge in a more systematic manner; Raise the awareness among direct beneficiaries on the importance of sharing knowledge and skills with the community; Further capacity build through activities such as a training of trainers program; or Provide an expert to travel across villages to provide direct support and assistance to develop and implement local strategies Gender Issues Women comprised 27% of participants in the Community Mediation Project. This exceeded the IDLO target of 25% participation. Gender mainstreaming was a primary concern for the project. A significant disappointment under the Post-Tsunami Legal Assistance Initiative was the failure to recruit any suitably qualified female trainers for the Community Mediation Project. This was rectified in this Program with one female trainer joining the core group of IDLO training consultants and two of the three IDLO staff working on the project (the Training Coordinator and the Legal Assistant) being female. The training content of the Community Mediation Project provided a specific focus on gender-related issues, informed by IDLO s previous research on gender and numerous interviews with local female stakeholders. For the three topic areas of land, inheritance and guardianship law, specific sub-topics were devoted to gender aspects (example, Women and Different Forms of Guardianship, Women s Inheritance Rights and Women and Children s Land Rights ). Furthermore, the training of trainers for IDLO facilitators focused on the importance of gender sensitivity and the need to create an atmosphere where women would feel comfortable engaging and participating in the workshop. The potential empowerment of women and impact on the status of women within village structures is discussed in the Impact section of this report on page CONCLUSIONS Overall Findings The Community Mediation Project was, overall, highly relevant to community needs and effective in meeting its objective of raising the awareness of community leaders in tsunami- and conflict-affected areas of Aceh regarding legal rights in the subject matter of land, inheritance and guardianship law, and facilitating the fair and efficient resolution of tsunami- and conflict-related legal issues at the village-level by building the capacity of community leaders to mediate disputes. The project was able to deliver culturally appropriate activities and quality training and effectively engage at a grassroots level, responding and adapting to the needs of both tsunami-affected and post-conflict communities in Aceh. While it may be too early to accurately assess its impact, the findings of the evaluation suggest that the project may be contributing to triggering a process of change in community attitudes and practices. However, a strategy to increase local ownership and to assist direct beneficiaries transfer knowledge and skills gained from the project would have enhanced the sustainability of the project and its potential to have long-term effects on the lives of individuals and communities in Aceh. By Assessed Criteria Relevance The Community Mediation Project continues to be extremely relevant to the needs of communities and confirms the validity of the project s objective beyond the post-tsunami 30

39 environment. The topics addressed by the project correspond to the most significant legal issues facing Acehnese communities in both tsunami-affected and post-conflict areas, reinforcing the importance of the extension of IDLO s activities to remote villages in Aceh. From an operational and logistical standpoint, the Community Mediation Project built on lessons-learned from prior implementation (under IDLO s Post-Tsunami Legal Assistance Initiative). However, the topic of domestic violence, which had been identified as a key issue in the M&E Report , could have been, but was not, incorporated into the training materials under the Program. The evaluation found that the relevance of the Community Mediation Project would have been enhanced had IDLO incorporated this issue in its activities. The evaluation also found that there is a heightened need for in-depth information and training on the formal justice system and its procedures. Beneficiaries and stakeholders interviewed for this evaluation suggested that IDLO s activities should have incorporated knowledge or skills to bridge the gap between the formal and informal justice systems in Aceh. This would increase the relevance of IDLO s activities to Aceh s present context, as villages emerge from isolation and the government continues to engage in the peaceprocess, and likely enhance the effectiveness of the Program in achieving its overarching goal of strengthening the rule of law and good governance in Aceh. Recommendations: Any further programming of the Community Mediation Project should: Include domestic violence as a topic in the training materials (or design and implement a new domestic violence program altogether; Include training and information on the formal justice system in the project modules and involve formal institutions in the project to promote a link between the formal and informal justice sectors. Effectiveness The Community Mediation Project largely achieved its stated objective and effectively contributed to improving access to justice and law remedies at the village level, increasing local capacity to solve disputes in a fairer manner consistent with Indonesian law. In this regard, IDLO s strategy of working at the grassroots level and bringing activities directly to villages proved to be valid and highly appropriate. The project was well accepted by communities and was in line with local customs and culture. Beneficiaries confirmed there was no feeling of cultural or normative imposition from IDLO in the delivery of activities. This, together with the quality of project facilitators and the delivery methodology, and the accuracy of training materials, resulted in high levels of beneficiary engagement and led to considerable improvement in knowledge and skills retention among participants. The evaluation also found a high level of practical application of the knowledge and skills acquired by direct project beneficiaries. This has had a positive affect on the communities, including in relation to vulnerable groups such as women and children. This trend, however, was not as evident in more remote post-conflict areas, possibly attributable to a greater resistance to accepting formal legal provisions and stronger adherence to traditional norms in these locales 28. Although the Community Mediation Project s approach of targeting a broad cross-section of village leadership proved to be valid, extending activities to the entire community may have enhanced the effectiveness of the project. Raising legal awareness at the community level empowers individuals to hold village leaders and mediators accountable for fair and equitable decisions in accordance with the law. Recommendations: Any further programming of the Community Mediation Project should: Extend project activities to the broader community (not just community leaders); Include training and information on the formal justice system in the project modules and involve formal institutions in the project to promote a link between the formal and informal justice sectors; 28 As noted by legal stakeholders. 31

40 Use a tailored approach when implementing project activities in more remote and culturally conservative areas of Aceh to ensure greater relevancy for beneficiaries. Impact and Sustainability While perhaps too early to accurately assess the impact of the Community Mediation Project, a number of narrative examples of the practical application of knowledge and skills from the activities suggest that the project may be contributing to triggering a process of change in dispute resolution procedures in beneficiary villages in Aceh. Project participants interviewed for this evaluation noted that such examples may serve to act as precedents or best practices for informing future dispute resolution cases in their respective communities. However, this is an assumption which relies on a number of variables, including: - Village leaders who participated in the project continuing to act as mediators for a significant length of time in order to establish practices; - Village leaders are not corrupt nor biased in their decision-making; - Village leaders pass on acquired knowledge and skills to others, particularly youth, who constitute the new generation of leaders. IDLO, therefore, cannot expect to have an impact on community processes and individuals lives without a strategy to ensure that quantifiable immediate achievements of the project (that is, increased legal awareness and improved mediation skills among participants) are extended into the future through a process of local ownership and building beneficiary capacity to transfer acquired knowledge, limiting its potential to have long-term effects on the lives of individuals and communities in Aceh. Recommendations: Any further programming of the Community Mediation Project should: Develop and implement a strategy to facilitate sustainability of the project by encouraging local ownership and building beneficiary capacity to transfer acquired knowledge; Extend the geographical scope of the project to other areas of Aceh. 32

41 3.0 Raising Women s Legal Awareness Through Film Project Summary of Evaluation Findings The Film Project was overall highly relevant to the needs of beneficiaries and effectively met its stated objective. The project delivered culturally appropriate activities tailored to the local Acehnese context, effectively responding needs of women in communities by engaging at the grassroots level. While it is perhaps too early to assess its impact, the findings of the evaluation suggest that the Film Project may be contributing to changing the lives of beneficiaries throughout Aceh. However, the lack of a strategy to promote local ownership and build beneficiary capacity to share knowledge gained from the project may affect the sustainability and impact of the project. Summary of Recommendations: Any further programming of the Film Project should: Include domestic violence as a topic in the training materials (or design and implement a new domestic violence program altogether); Broaden its gender-focused activities beyond land, inheritance and guardianship law to address women s rights and legal empowerment more generally; Extend film screenings (or women s legal rights activities) to all community members, directly targeting the general male population. IDLO should seek to amend its Program strategy to address the existing education gap among the general male population regarding women s legal rights; Develop and implement a strategy to facilitate sustainability of the project by encouraging local ownership and building beneficiary capacity to transfer acquired knowledge; Strengthen connection between the project and other projects, in particular the Community Mediation Project for example, IDLO should screen the film in all villages where the project is implemented. 3.1 THE PROJECT Project Overview The Raising Women s Legal Awareness through Film Project (Film Project) was initially implemented under IDLO s Post Tsunami Legal Assistance Initiative (February October 2007). During this time, IDLO produced the 30 minute educational film titled The Stories of Aisha, Rauda and Ainun: Protecting Women s Legal Rights in Aceh Post-Tsunami (the film). The film tracks the lives of three women, each of whom is struggling to overcome some of the more common legal issues affecting communities in Aceh in the aftermath of the tsunami. Through these narratives, the film explains the law relevant to each of the main topic areas of land, inheritance and guardianship and the possible solutions that might be found within the community as well as the formal court system. The on-going relevance of the Film Project and the recognition of its significant contribution to the broader development of the justice system in Aceh, prompted Oxfam to fund its continuation under the framework of the Program. This included extending the Film Project to post-conflict areas not affected by the tsunami. The content of the activities under the project other than the film was slightly revised to included new and pertinent topics, such as domestic violence, and workshops were added to respond to changing needs in the Acehnese community. The objective of the project was to improve the understanding and awareness of women in 126 villages in Banda Aceh, Aceh Besar and among Da I scholars and selected women s groups in Banda Aceh, Aceh Besar and Bireuen, regarding women s legal rights and the avenues through which such rights can be protected. 33

42 This evaluation covers the entire Film Project 29 and considers its relevance, effectiveness, impact and sustainability. To mitigate bias and ensure credibility of evaluation findings, IDLO assessed these criteria by cross-referencing data collected from: Immediate post-screening and workshop evaluation questionnaires; Activity reports; Interviews with key legal stakeholders; A legal awareness survey; and Interviews with participants who received training under the project at least three months after the training was delivered Background 30 years of separatist conflict in Aceh has created a situation where justice mechanisms are far from fully developed and the stability of the rule of law has been weakened. There exists a lack of community legal awareness, particularly so amongst women, who have limited knowledge of their legal rights and of the avenues through which these rights can be protected. Strengthening the rule of law and justice systems in Aceh is contingent upon the development of a legal culture within which individuals and communities are aware of and understand their rights, value these rights, and have confidence in the mechanisms through which these rights can be claimed and protected. In this context, IDLO designed the Film Project to raise awareness amongst women in Aceh of their legal rights and empower them to pursue these rights through informal and/or formal justice mechanisms. The design of the Film Project was informed by the results of five IDLO assessment missions to Banda Aceh and Aceh Besar (December 2004-January 2006) and extensive field-based research undertaken between March and May 2006 in 12 tsunami-affected villages. A key conclusion of the design phase was that women were at risk of having their legal and human rights violated. Particular concerns included that women had limited access to information regarding their legal rights; were often denied entitlements under Islamic inheritance law; were rarely recognized as land owners; and were caring for orphans without the legalized protection of guardianship status Project Implementation Phase 1 of the Film Project (1 November March 2008) included multiple screenings of the film followed by brief information sessions. Phase 2 (01 April 31 December 2008) included the delivery of 24 x 1-day Follow-Up Workshops for women who had seen the film, and six x 3-day Raising Awareness of Women s Legal Rights Workshops for women s NGOs, village leaders and Da I 30 in Bireuen and Pidie. Direct Beneficiaries The project directly benefited a total of 5,801 members of the Acehnese community (83 men and 5,718 women), including women in Aceh Besar and Banda Aceh and women leaders, women s NGO representatives and Da I officials in Bireuen and Pidie. See Outputs for a breakdown of this total number. Outputs Film Screening and Information Sessions The film was screened 132 times (exceeding the planned 126) benefiting 4,873 women (exceeding the planned 3,150) in tsunami-affected and post-conflict villages in Banda Aceh and Aceh Besar. The film screenings were originally planned to cover the same areas as the Community Mediation Project, namely villages in Banda Aceh, Aceh Besar, Aceh Jaya and Aceh Barat. 29 But does not include the development of the book The Role of Acehnese Women in Village Adat Systems, as it had not yet been published or disseminated at the time of the evaluation exercise. 30 Da I are religious teachers and orators in mosques who have a strong influence over their congregations and often lack a balanced gender perspective. 34

43 However, it was later decided, with the approval of Oxfam, that the film screenings and information sessions only be implemented in Banda Aceh and Aceh Besar since: - Travelling long distances to implement a four hour activity (film screening and information session) was determined to be an inefficient use of staff time, as in many cases the exercise would require three days to implement (one day travelling, staying overnight, implementing half-day activity; staying overnight, then retuning to Banda Aceh on the third day); - Delivering two sessions per day was not feasible as most women in rural areas work in the fields in the mornings; - The high cost of working for prolonged periods of time outside of Banda Aceh could not be absorbed by the project budget; and - Cultural difficulties exist for Acehnese women in working away from home for extended periods, having implications for IDLO female staff who were implementing the project. 1-Day Follow-Up Workshops Based on the M&E Report and discussions with the IDLO Legal Officer managing the Film Project, it was decided that the brief information sessions following each film screening should be developed into full, 1-day Follow-Up Workshops for women who viewed the film in 2007 (under the Post-Tsunami Legal Assistance Initiative). By doing this, beneficiaries would be able to build on their existing knowledge of women s legal rights and receive more in-depth information on key issues. Follow-Up Workshops comprised the re-screening of the film, more detailed discussion on women s legal rights in relation to inheritance and guardianship as raised in the film, and issues and laws relating to the new topic of domestic violence. Each workshop was attended by 30 participants drawn from six villages (five participants per village) and was conducted by two IDLO Legal Officers who are experts in Syar iyah law. During the Film Project, a total of 24 x 1-day Follow-Up Workshops, covering 144 villages and 727 beneficiaries, were implemented. 3-Day Raising Awareness of Women s Legal Rights Workshops IDLO conducted six x 3-day Raising Awareness of Women s Legal Rights Workshops for women s NGOs and Da I. Since it was determined that a large number of gender-specific programs had already been carried out in Banda Aceh and the surrounding area, it was agreed with Oxfam to shift four of its workshops from Banda Aceh and Aceh Besar to the north coast, implementing all six workshops in Bireuen and Pidie. Da I workshop participants came from Bireuen and Pidie, including from the district Da I Association of Indonesia (Ikatan Da I Indonesia) in Bireuen and the Consultative Council of Ulama (Majelis Permusyawaraten Ulama) in Pidie. The workshops for women s NGOs included participants from organizations such as Aceh Women s League, Women s Cooperation (Tjut Nayk Dhien), as well as women community leaders from the subdistricts in east and west Bireuen and Pidie. The six x 3-day workshops benefited a total of 201 participants. Publication of the Book: The Role of Acehnese Women in Village Adat Systems The book consists of general information regarding the involvement of women in the adat system of Aceh based on interviews and research conducted by an IDLO Legal Officer between April and September 2007 in four districts, eight sub-districts and 24 villages in Banda Aceh, Aceh Besar, Pidie, and Bireuen. To ensure quality, utility and legal accuracy, the book was peer reviewed by Professor Dr Misri A Muchsin, MA from IAIN Ar-Raniry University and H Badruzzaman, Head of the Majelis Adat Aceh. As the book was not finalized until the end of December 2008 and will not be disseminated until 2009, this output of the project was not evaluated. 35

44 3.2 EVALUATION Table 13: Summary of Evaluation Tools and Indicators Film Project Project Objective Success Indicators Evaluation Tools Raising Women s Legal Awareness Through Film To improve the understanding and awareness of women in 126 villages in Banda Aceh, Aceh Besar and among Da I scholars and selected women s groups in Banda Aceh, Aceh Besar and Bireuen, regarding women s legal rights and the avenues through which such rights can be protected a. Feedback from key legal sector stakeholders regarding the relevance and appropriateness of the Film Project b. Participants find Film Project effective, relevant and appropriate (immediate assessment) c. Participants continue to find information provided by the Film Project useful at least 3 months after screening/training d. 20% of participants interviewed provide narrative examples of the practical application of information provided by the Film Project a. Interviews with key legal stakeholders b. Legal awareness survey conducted at least 3 months after screening c. Interviews with participants at least 3 months after screening of Film d. Questionnaire administered immediately after each screening and workshop Relevance To determine the relevance of the Film Project, three sources were utilized: Feedback from interviews with key legal stakeholders; Feedback from immediate post-training questionnaires; Feedback from interviews with a sample group of those who received training under the project, interviewed at least three months after the training. Conclusions were then drawn on the relevance of the Film Project intended as a combination of the following: The validity of the project s objectives in relation to the Acehnese context and to changing circumstances; Correspondence between the topics addressed in the project and the most significant legal issues confronting Acehnese communities; and Applicability and usefulness of information provided by the project to the lives of beneficiaries and the Acehnese community. Stakeholder Interviews 31 Interviews with key legal stakeholders confirmed the continued relevance of the Film Project in both the post-tsunami and post-conflict contexts in Aceh. Stakeholders agreed that disputes concerning land, inheritance distribution and guardianship were the most common legal issues facing Acehnese communities. Stakeholders noted that Aceh is moving from a post-tsunami to a post-conflict environment and that while fewer disputes, particularly those brought before the formal legal system, are related to the tsunami, the topics addressed in the film remain the primary issues surrounding disputes arising from the 30 years of separatist conflict in Aceh. These comments support the extension of IDLO s Film Project to post-conflict areas. Stakeholders confirmed that women face particular challenges in protecting their legal rights throughout Aceh. For example, there is a widespread perception that women cannot be appointed legal guardians over orphans and therefore cannot access economic support 31 See Annex 2 for list of stakeholders interviewed. 36

45 through the assets inherited by the child. This places financial burdens on these women. In addition, stakeholders mentioned that women are often victims of matrimonial poverty, as widows or divorcees are commonly denied their inheritance and joint-asset rights due to lack of awareness regarding the applicable law or absence of documentation verifying joint property ownership between husband and wife. These comments alone support the gender-specific focus of the Film Project. Stakeholders also confirmed that domestic violence needed to be addressed to ensure the protection of women s rights. Stakeholders said this is a sensitive topic in Aceh and that there was a need for increased education and awareness-raising, confirming the appropriateness of IDLO adding information on domestic violence to the Film Project workshops. Stakeholders commented that a lack of community legal awareness is the main obstacle preventing individuals from claiming and protecting their legal rights. Stakeholders agreed on the importance of assisting both tsunami-affected and post-conflict communities to understand their legal rights, highlighting the fact that very few organizations are working in the access to justice sector. One stakeholder mentioned in relation to Aceh, that: there is a small focus on justice in a community where there is a general lack of legal awareness we still need more action from NGOs to raise awareness. Finally, all stakeholders who had seen IDLO s film commented that the topics covered were extremely relevant. Those who had not seen the film but were informed about its content verified that the topics were pertinent to the needs of women in Aceh. One stakeholder attended the film screening as an observer and noted that participants response to the film was highly positive, with women identifying with the situations portrayed in the film and asking questions regarding their legal rights. Participant Feedback Immediate Post-Activity Evaluation Questionnaire Feedback from participants immediately after the Film Project s activities confirmed that the topics covered are relevant to the community in Aceh. The following summaries responses to one key question indicating the usefulness of the project in relation to community needs. 3,000 GRAPH 7 Film Screenings Question (immediate post-activity): What was the most useful information provided in the film? 2,500 No. of Responses 2,000 1,500 1, Inheritance Guardianship Land Mahkamah Syar iyah Mediation Techniques Syar iyah Islam 37

46 250 GRAPH 8 1-Day Follow-Up Workshops Question (immediate post-activity): What was the most useful information provided in the workshop? 200 No. of Repsonses Domestic Violence Inheritance Guardianship GRAPH 9 3-Day "Raising Awareness of Women's Legal Rights" Workshops Question (immediate post-activity): What was the most useful information provided in the workshop? No. of Repsonses Domestic Violence Inheritance Guardianship While inheritance remained the most useful topic in the film screenings and one-day Follow-Up Workshops, a shift occurred in the 3-day Raising Awareness of Women s Legal Rights Workshops, where participants noted that domestic violence was the most useful topic. The shift may be the result of two different target audiences for the activities. The 3- day workshops were attended by women s NGO members and Da I while the film screenings and one-day workshops were attended by women in the general community. Possible reasons for the difference in selected topics may be a combination of the following: Women s NGO representatives of the former audience may have maintained a higher interest in domestic violence as it relates to their advocacy work; Da I already have fairly extensive knowledge of women s legal rights in relation to guardianship, inheritance and land ownership under Islamic law and therefore were more interested in new information provided on domestic violence. However, regardless of the reason for the above, and considering that the importance of information and training on domestic violence was also raised by beneficiaries of the Community Mediation Project, it is recommended that IDLO continue to focus its activities in Aceh on the legal implications of domestic violence. 38

47 Participant interviews at least three months after IDLO activities A confirmation of the continued relevance of the topics addressed by the Film Project can be inferred from the feedback from a sample group of participants interviewed at least three months after activities occurred. The following is a summary of responses that the sample group provided to key questions: GRAPH 10 According to Activity 40 Question 1: (at least 3 months after activity): What was the most useful information provided in the film and/or workshop? 35 No. of Resp onses Inhertiance Guardianship All top ics Land Inheritance &Guardianship Inhertiance Guardianship All top ics Land Inheritance &Guardianship Inhertiance Guardianship Inheritance &Guardianship Dom estic Violence 1-Day Workshop Film Screening 3-Day Workshop The data shows that inheritance was regarded as the most relevant and useful topic covered in the film screenings and one-day Follow-Up workshops. Participants responses reflected an identical trend in the rating of useful topics. These findings also correspond to those of the M&E Report , indicating that inheritance continues to be a key issue for women as Aceh moves beyond the post-tsunami context. Regarding the 3-day workshops, the results in Graph 10 are significantly different than those of the immediate post-activity evaluation questionnaire. It is difficult to provide an accurate analysis of these differences as the beneficiary group interviewed was not a representative sample of participants. This was the result of the unavailability of women s NGO representatives for interviews at the time of the evaluation exercise. 39

48 GRAPH 11 Overall Reponses (Film Screenings & Workshops) Question 2 (at least 3 months after activity): What topic would you like to receive more information or further training on? No. of Repsonses Land Guardianship Inheritance Women's Rights Children's Rights Child Abuse Law in General Formal Legal System Islamic Law Marital Law Domestic Violence Criminal Law Human Rights Mediation Don't Know The data in Graph 11 further indicates the continued relevance of the topics addressed in the Film Project and highlights that domestic violence is the topic which participants are most interested in receiving more information about. Though domestic violence was not explicitly addressed in IDLO s film, it was included in the film session discussions and in the film workshops. However, since the need to receive more information on domestic violence has been continually expressed by the community 32, it is again recommended that IDLO consider this need in future programming in Aceh. Additionally, a considerable number of participants expressed a desire for more information regarding women s rights beyond land, inheritance and guardianship matters. Participants said that improved knowledge in the area of women s rights was critical to their empowerment and to gaining a stronger voice in community affairs. This suggests that IDLO should consider broadening the scope of its gender-focused activities in any future programming in Aceh Effectiveness To determine the effectiveness of the Film Project, four sources were utilized: Feedback from interviews with key legal stakeholders; Feedback from immediate post-training questionnaires; Feedback from interviews with a sample group of participants involved in project activities, at least three months after the activities; Legal Awareness Surveys conducted among the general public in Banda Aceh and Aceh Besar. By analyzing these four sources, conclusions can be drawn on the effectiveness of the Film Project, being: The extent to which the project achieved its objective and is contributing to the aim of IDLO s overall Program; The validity of using film as a strategy to raise legal awareness; The accuracy of the content in the film and the information provided in the workshops; 32 Beneficiary interviews with Community Mediation Project participants also determined that domestic violence was the most requested topic for further training. 40

49 The satisfaction of project beneficiaries with IDLO training methodology and facilitators; The delivery of culturally appropriate activities; and The increased legal awareness of project beneficiaries in comparison with the legal awareness of the general public in Aceh. Stakeholder Interviews 33 Using Film as a Tool to Raise Legal Awareness: When asked specifically about the most effective strategies for raising awareness of legal issues among a wide-audience in Aceh, stakeholders emphasized that a visual medium is highly effective as it allows for the sharing of complex legal information in a user-friendly and simplified format. Stakeholders also noted the importance of using the film as an education tool to engage women in villages where low levels of literacy and formal education exist. These comments reaffirm the validity of IDLO s strategy to use the film to raise the awareness of women s legal rights in Aceh. Stakeholders emphasized the importance of direct information dissemination, like activities conducted through the Film Project. This was regarded by stakeholders as one of the most effective strategies for raising awareness, especially for rural and vulnerable groups (such as women) who may not have access to mass-media and often have an oral-culture of information sharing. Accuracy of the Film s Content: All legal stakeholders who had watched the film stated that its content was legally and technically accurate. The former Chief Justice of the Mahkamah Syar iyah provides a twominute endorsement of the film at its commencement where he notes the content s compatibility with both Islamic and Indonesian law. This gave credibility to the film and the project more generally, promoting acceptance of IDLO s work among both legal stakeholders and community members. Participants Feedback Participant Satisfaction with Project Methodology and Facilitation: Post-screening evaluations indicated that 97.1% of the participants (sample size 4,191) rated the facilitators as good (highest possible rating) while 4.4% rated the facilitators as average and 0.04% as poor. The overwhelmingly positive response suggests IDLO facilitators were effective in delivering activities and contributing to the achievement of the project s objective. GRAPH 12 Quality of IDLO Facilitators (immediate post-activity) Film Screenings & Workshops Average 4.4% Poor 0.04% Good 97.10% 33 See Annex 2 for list of stakeholders interviewed. 41

50 Participants Feedback three months after Activities When interviewees were asked directly (three after the activities) if they found the film and workshops culturally appropriate and in accordance with local customs in regards to content and delivery, 100% of the female beneficiaries responded positively, stating that the information provided in the film and the film s presentation was culturally welcome. While some of the Da I who participated in the 3-day Raising Awareness of Women s Legal Rights Workshops said some of the information was not acceptable according to their personal views and interpretations of Islamic law (that is, information on Patah Titi and women s rights to joint-assets), they felt the workshops were presented in respectful and culturally appropriate manner. The Da I emphasized that no information was imposed and that they felt comfortable discussing sensitive issues with IDLO facilitators. This feedback indicates that IDLO delivered the Film Project in a manner tailored to the needs and values of project beneficiaries. Participant Quotes on Effectiveness Miruk Taman Village - The film was so positive for us women here. I have much respect for IDLO and for the staff because IDLO is a non-muslim organization but still has respect and concern for the Islamic Community to learn about their rights. Ateuk Jawo Village - The information and the film are appropriate because they are teaching us about the law and the right way to act. It is important for the community to accept this information because it is based on Islam and it is part of our culture. Lampaseth Kota Village - Although what is shown in the film may be different than what happens in reality in our community, it is important that we learn this information because both the film and our custom are based on Islamic law but we need to make our customary law more aligned with Islamic law. Kopelma Durussalam Village - Before the tsunami, inheritance in Aceh only focused on the son, but after seeing the film, the community has changed its mind. We think the best thing is to accept this information in the future and make it part of our custom. While all participants interviewed believed that IDLO s activities were positive, some provided suggestions on how to improve the effectiveness of the project as follows: The most common suggestion was that the film screening should not be stand-alone and that follow-up workshops and trainings should be provided on a regular basis. For many women, this was the first time they had learned about their legal rights. The majority stated that such follow-up trainings might trigger a process of positive change in community practice and lead to increased protection of women s legal rights. The Film Project should be extended to all village members and not only the women. Many participants believed that it is important for both men and women to be educated on women s legal rights. This suggestion was also echoed by stakeholders. The film screening and the one-day Follow-Up Workshops were too short to fully grasp and retain the information covered. Further, more time could be allocated for discussion during the information sessions and workshops. The use of the Indonesian language in the film and by facilitators sometimes made it difficult to understand and follow the legal terminology. In certain areas where the Film Project was delivered, participants had limited formal education and only spoke Acehnese. Legal Awareness Survey Although the absence of baseline data makes it difficult to measure any increase in the legal awareness among participants, comparing the level of legal awareness among different audiences, namely, the general public in Banda Aceh and Aceh Besar and beneficiaries of IDLO s Film Project, evidence suggests that IDLO s activities have improved the understanding and awareness of women s legal rights in communities in Aceh. 42

51 IDLO s legal awareness survey included three basic questions on women s legal rights (See Annex 5 for the complete survey): - Can women be appointed as guardians of children? - If one spouse dies, what should happen to property acquired during the marriage? - Can a land title (certificate tanah) for a house which is jointly owned by a husband and wife be jointly registered in names of both the husband and wife? The percentage of correct responses from each audience is outlined below: Table 14: Legal Awareness Survey Film Project Project Activity Percentage of Interviewees Answering Correctly Question 1 Question 2 Question 3 Film Screenings (Women) 60% 65% 88% 1-Day Follow Up Workshops (Women) 90% 80% 85% 3-Day Raising Awareness of Women s Legal Rights Workshops General Public in Banda Aceh and Aceh Besar Women 100% 60% 100% Da I (men) 60% 60% 40% Women 56% 42% 45% Men 43% 49% 42% As detailed in the Table 14, the percentage of correct answers among Film Project beneficiaries is significant higher when compared with women in the general public. This increased level of legal awareness among beneficiaries supports the analysis that the project was effective in achieving its stated objective. In addition, Table 14 reveals that participants in the 1-day and 3-day workshops maintained a greater level of legal awareness when compared to those who had only participated in the film screening and information session. This indicates that combining film screenings with subsequent workshops proves to be a more effective strategy for raising awareness of women s legal rights than delivering the film screenings and information sessions alone. The lowest percentage (60%) of correct answers in Question 2 occurred among participants in the 3-day workshop. This may be attributed to the influence of Da I who are known to hold more conservative views on gender parity 34. This assumption is inferred from interviews with Da I conducted at least three months after the workshops, during which 40% of those interviewed said they did not accept formal legal provisions entitling woman to be appointed as guardians of children and 80% said they did not accept formal legal provisions entitling women to jointly share assets in marriage Impact To determine the impact of the Film Project (that is, its long-term effect and the extent to which it has contributed to changing people s lives), two main sources were utilized: Feedback from interviews with a sample group of participants conducted at least three months after the training; Feedback from interviews with key legal stakeholders. While it is difficult to accurately assess the long-term effects of any activity only three months after its delivery, the timeframe is sufficient to provide some early indication as to potential impact. In this regard, the following is a summary of the interviews with project participants at least three months after conclusion of the project: 34 Stakeholders commented on the resistance of the Da I (religious teachers and orators in the mosques) to accepting formal legal provisions in favour of women s rights. 43

52 34.3% of those interviewed had used, or were able to identify someone who had used, information from the film and/or workshops in an effort to resolve a legal issue. This exceeded the project success indicator of 20%; Of these respondents, 72.1% said that the legal issue was resolved in a satisfactory manner while 27.9% noted that the issue was either ongoing or reached an unsatisfactory resolution. The following are select stories of the practical application of knowledge acquired during the Film Project by beneficiaries: Participant Utilizes the Formal Legal System X attended IDLO s film screening in her village. The film showed how to resolve legal issues by first speaking to the village leader and if no solution could be found then to use the formal legal system. In X s family, there was a disagreement about how to distribute X s father s inheritance. X s uncle had received all of the inheritance. After trying to resolve the case with the village leader, her family brought the case before the formal courts, the Mhakamah Syar iyah. X had received information about the formal legal system and learned about the formal process through watching IDLO s film. X is now satisfied because the judge ruled according to the law and she now has her portion of the inheritance from her father. Participant Shares Information from Film and A Grandmother Receives Guardianship Rights In M s village, before IDLO s film workshop, an orphan s inheritance had been taken by the child s uncle, even though the child was being cared for by his grandmother. After the training, M and her friends knew that this was not legal. The orphan s grandmother had not attended the film screening, but knew that M had participated. The grandmother approached M with questions about inheritance and guardianship. M shared the information she had learned from IDLO s film workshop and suggested that the grandmother speak to the village leader about the situation. The grandmother went to the village leader who invited her and the child s uncle to participate in a mediation process to resolve the issue. The mediation was successful and the grandmother now has guardianship of the child and manages the child s inheritance which was given back by the uncle. Years After Parents Deaths, Daughter Fights for Inheritance Rights When L s parents died a few years ago, she did not receive any inheritance but believed that this was in line with the law. Then, L watched IDLO s film and learned that she should have received a portion of her parents inheritance. L went with a group of women in her community who had also watched the film to her father s family who had received the inheritance and informed them that this was not according to the law. L told them that she had a right to her inheritance. L s uncle agreed to go with her to the village leader to discuss the matter. The village leader recognized that though this information was new, it was according to the law and L should received her rightful portion of the inheritance. L was given her inheritance and her certificate of ownership for her father s land from the head of the village. Several women interviewed said that while information provided by the Film Project was relevant and applicable, they would not be able to access formal courts without assistance in the event solutions for their disputes were not reached at the village-level. The reasons given were: Lack of confidence in their individual capacities to use the courts; Limited trust in the formal judicial system; and Fear of excessive costs. These reasons constitute a potential obstacle to the intended impact of the Film Project. A possible way to overcome this is for IDLO to establish a partnership with a local legal aid service provider (for example, KKTGA) to assist women in pursuing their right to choose between the formal and informal justice system, ensuring their ability to access fair and impartial justice. 44

53 A common suggestion from interviewed participants, as well as stakeholders, was that the film should also be screened men who traditionally have more influence over decisionmaking processes. One participant noted that If only the woman knows her rights and the man does not, more conflict will be caused between the two. The non-receptive attitudes of men could also potentially compromise the impact of the Film Project. Accordingly, it is recommended that include separate film screenings for men in any future implementation of the project and screen the film in villages where the Community Mediation Project is implemented to maximize potential for impact Sustainability To gauge the level of sustainability of the Film Project (defined as the transfer of project ownership to the local community to ensure long-term impact) a sample of participants were interviewed at least three months after IDLO s activities. Participants were asked how they have used or plan to use the knowledge gained from the project. A summary of responses is as follows: 63.5% have shared the information through casual conversations with family and friends; 20.8% will share the information if someone approaches them with a problem; 6.35% have kept the information for their personal use and do not intend on sharing it; 9.4% have shared it during weekly religious group meetings (for women only); and 2.1% have shared the information with students (Da I, only men). Clearly, the majority of participants interviewed for the evaluation have passed on the knowledge gained from the project in an informal and ad hoc manner. However, the majority of participants indicated that sharing knowledge formally would be the most effective way to ensure community legal awareness continues to be raised, thus improving the sustainability of the project. Many indicated that they would be willing to conduct formal information sharing activities with appropriate training and support. Accordingly, this evaluation report recommends that any further programming in relation to the project include: Providing support to participants to share their knowledge in a more systematic manner; Raising the awareness among direct beneficiaries on the importance of sharing knowledge with the community; Capacity building activities such as a training of trainers program; or Providing an expert to travel across villages to provide direct support and assistance to develop and implement local strategies. 3.3 CONCLUSIONS Overall Findings: The Film Project was overall highly relevant to the needs of beneficiaries and largely met its stated objective. The project delivered culturally appropriate activities tailored to the local Acehnese context, effectively responding needs of women in communities by engaging at the grassroots level. While it may bed too early to assess its impact, evidence suggests that the project may be contributing to changing the lives of beneficiaries throughout Aceh. Finally, a strategy to promote local ownership and building beneficiary capacity to transfer acquired knowledge to the community would have improved the sustainability and potential long-term impact of the project. 45

54 By Assessed Criteria Relevance The evaluation confirmed the validity of the Film Project s objective as beneficiaries and stakeholders emphasized the continued need for legal awareness-raising activities in Aceh. It found that the subject matter covered by the project inheritance, land, guardianship and domestic violence directly corresponded to the most significant legal issues confronting women in both tsunami-affected and post-conflict areas in Aceh. While inheritance remained the most useful topic for beneficiaries, more information on women s rights in general beyond the subject matter covered in the film (inheritance, land and guardianship) was requested by beneficiaries. IDLO responded to the M&E Report by including domestic violence in the activities of the Film Project under the framework of the Program. This enhanced the relevance of the project. Nevertheless, beneficiaries overwhelmingly requested more information on domestic violence. Recommendations: Any further programming of the Film Project should: Include domestic violence as a topic in the training materials (or design and implement a new domestic violence program altogether); Broaden its gender-focused activities beyond land, inheritance and guardianship law to address women s rights and legal empowerment more generally. Effectiveness The Film Project largely met its stated objective and effectively contributed to an increased understanding and awareness of women s legal rights in beneficiary communities. IDLO s strategy of direct information dissemination and community engagement proved to be highly valid, as was the use of a visual medium as an educational tool for audiences, such as women, with generally lower levels of literacy and formal education. The effectiveness of this strategy is highlighted by the evaluation s legal awareness survey which revealed that project beneficiaries had higher levels of legal awareness relative to the general public in Aceh, in the subject matter of the project. The methodology employed by IDLO and the quality of facilitators greatly contributed to the effectiveness of the project. Stakeholders interviewed for the evaluation confirmed that the content of the Film Project was accurate and activities were delivered in a culturally appropriate manner and in accordance to local customs. While the project s gender-specific approach proved to be valid, extending activities to the entire community may have enhanced effectiveness in relation the Program s overall goal. Both beneficiaries and stakeholders noted the importance of educating men on women s legal rights in order to ensure protection of such rights. All agreed that the Film Project s activities should have included the general male population. 35 Recommendations: Any further programming of the Film Project should: Extend film screenings (or women s legal rights activities) to all community members, directly targeting the general male population. IDLO should seek to amend its Program strategy to address the existing education gap among the general male population regarding women s legal rights; Impact and Sustainability While it may be too early to draw a sufficiently accurate conclusion as to the impact of the Film Project, the findings of the evaluation suggests that the project may be contributing to a process of change in community attitudes and practices, changing the lives of beneficiaries. In this regard, 34.3% of beneficiaries interviewed had taken action or knew of someone who had taken action to resolve a legal issue based on information drawn from the project. 35 Both beneficiaries and stakeholders highlighted the importance of having separate film screenings for men and women. 46

55 However, this observation should be read in light of the following: - The project did not include the general male population, notwithstanding the recommendation of the M&E Report In an environment, such as Aceh, where the majority of disputes are resolved at the family or community level, educating the general male population on the legal rights of women is fundamental to ensuring the protection of women s legal rights. - The impact of the project could have been enhanced had there been a systematic strategy to promote community ownership and build beneficiary capacity to transfer acquired knowledge. Finally, a greater linking of projects for example, delivering the Film Project and Mediation Project in the same villages would have enhanced the impact of the projects and the overall Program by, for example, raising awareness at both the community level and leadership/decision-making level of promote and protecting women s legal rights. Recommendations: Any further programming of the Film Project should: Extend the project activities to all community members, thereby including the general male population. Develop and implement a strategy to facilitate sustainability of the project by encouraging local ownership and building beneficiary capacity to transfer acquired knowledge; Strengthen connection between the project and other projects, in particular the Community Mediation Project for example, IDLO should screen the film in all villages where the project is implemented. 47

56 4.0 Raising Community Legal Awareness through Print Media Summary of Evaluation Findings The Print Media Project had a wide target audience (potentially 450,000 people) while the evaluation was limited in its scope. Accordingly, it is difficult to make any accurate and direct conclusions as to the extent to which the project s objective was achieved. Nonetheless, feedback from key legal stakeholders and the results of the legal awareness survey indicate that the articles were relevant and informative. In addition, the project maintains a high level of local ownership as articles are written by a local journalist, reviewed by local legal experts and printed in a local newspaper. Therefore, the evaluation finds that the project has overall high utility due to the relevance of its topics, its high potential for sustainability and its costeffectiveness (an estimated 103,500 readers reached at a cost of US$330 per issue). Recommendations: It is recommended that IDLO consider transferring this project to a local counterpart and perhaps provide assistance at the initial stages of implementation. 4.1 THE PROJECT Project Overview The Raising Community Legal Awareness through Print Media Project (Print Media Project) was initially launched by IDLO in July 2006 as part of its Post-Tsunami Legal Assistance Initiative, and continued under the framework of the Program. The objective of the Print Media Project was to improve awareness among communities in Aceh regarding topical legal issues, legal rights and the various legal services available to protect those rights. IDLO sought to achieve this by: Publishing a weekly column in the Serambi newspaper entitled The Law and You in Practice (Anda dan Hukum dalam Keseharian) containing practical and useful information on topical laws and legal issues and justice sector institutions in Aceh; and Distributing copies of the articles to beneficiaries of IDLO s Mediation Project and Film Project. All articles were translated into English and copies (in both English and Indonesian) are posted on the IDLO website ( This evaluation covers the entire project. The scope of the target audience of the project and the absence of adequate baseline data allows only for a general indication as to the relevance, effectiveness and impact of the project Background The design of the Print Media Project was informed by IDLO s five assessment missions to Aceh (December 2004-January 2006) and its field based research between March and May 2006 in 12 tsunami-affected villages in Banda Aceh and Aceh Besar. A key conclusion of these activities was that the legal awareness of the general public in Aceh was very low, particularly among vulnerable groups such as women, tsunami-affected and conflictaffected groups, and those living in isolated areas. According to UNDP, levels of legal awareness relating to rights, duties and procedures within the formal and informal justice systems remain low among Acehnese. Villagers do not have information on laws that are 48