Internal Oversight Division (IOD) CONCEPT NOTE PROPOSAL EVALUATION OF PROGRAM 3: COPYRIGHT AND RELATED RIGHTS

Transcription

1 Internal Oversight Division (IOD) CONCEPT NOTE PROPOSAL V3 EVALUATION OF PROGRAM 3: COPYRIGHT AND RELATED RIGHTS 1. WHY DO THE EVALUATION? (A) BACKGROUND 1. The World Intellectual Property Organization (WIPO) is a specialized agency of the United Nations dedicated to developing a balanced and accessible international intellectual property system that enables innovation and creativity for the benefit of all. 2. For the year 2015, the Internal Oversight Division (IOD) has foreseen the Evaluation of Program 3: Copyright and Related Rights as per its 2015 Oversight Plan. Program 3 was selected after a comprehensive analysis, which was based on criteria such as: risk assessment, relevance, oversight coverage and comments made by WIPO management and Member States. In this case, a significant factor was the fact that Program 3 has not yet been evaluated. 3. Program 3 was established in 2010 as a substantive Program with its own resources. In December 2014, a new Senior Manager was appointed to take over the management of Program The Program 3 objective is to update, develop and promote the normative copyright and related rights framework and to foster an infrastructure for creativity in order to respond to the rapidly evolving technological, geo-economic, social and cultural environment, development strategies. The critical challenges for the area of copyright and related rights are those related to digital technology and the internet. In order to mitigate these challenges Program 3 has focused its work on the following aspects: (a) Normative and related work: Copyright treaties and national laws aims to balance the interests of those who create content with the public interest in having access to

2 EVAL CONCEPT NOTE 2. that content. WIPO administers multiple international treaties in the area of copyright and related rights and advises Member States at their request on the development of national copyright laws. (b) Technical assistance to developing countries and least developed countries: Program 3 leads a range of practical initiatives to assist in the effective management of copyright, and to ensure that copyright law works in action to reward creators and support cultural production. (c) Advancing the development of copyright and related rights infrastructure including the development of collective management organizations that embody transparency, accountability, and good governance practices. 5. A detailed Program 3 results framework for each biennium covered by this evaluation and the lists of international treaties on copyright and related rights can be found in Annex 1 of this concept note. The results framework has evolved over this six-year period and indicators have been refined to better reflect the Program objectives. 6. In addition, Program 3 is implementing/ has implemented various projects within the Development Agenda framework such as: (a) Strengthening the Capacity of National IP Governmental and Stakeholder Institutions to Manage, Monitor and Promote Creative Industries, and to Enhance the Performance and Networks of Copyright Collective Management Organizations. (b) Scoping Study on Copyright and Related Rights and the Public Domain. (c) Comparative Analysis of National Approaches on Voluntary Copyright Relinquishment. (d) IP, Information and Communication Technologies (ICTs), the Digital Divide and Access to Knowledge. (e) Strengthening and Development of the Audio-Visual Sector in Burkina Faso and Certain African Countries. (B) PURPOSE OF THE EVALUATION 7. The primary purpose for this evaluation is to contribute to the accountability of the organization through sampling data and information gathered from primary stakeholders 1 involved in and benefitting from the work undertaken by Program The secondary purpose is on learning lessons with a view to improving the delivery under this Program. 9. The evaluation will assess the performance of Program 3, including where applicable the technical assistance work done in collaboration with the Development Agenda and Coordination Division, the WIPO Academy, the Regional Bureaus, and the External Relations Division, among others, and generate evidence on the relevance, effectiveness, efficiency and sustainability of any achieved results. 1 A preliminary list of possible groups of stakeholders can be found in Annex 2. [We understand the sample has not yet been selected.]

3 EVAL CONCEPT NOTE 3. (C) OBJECTIVES AND USES 10. This Program 3 evaluation will provide decision-makers in WIPO with: (a) An assessment about the Program s performance, paying particularly attention to the effects of the Program results against compared to its objectives for each relevant period; (b) An assessment of the contribution of Program 3 towards the achievement of Strategic Goals I, III, IV and V and other strategic goals as appropriate; and (c) The identification of key factors and lessons that could be replicated and proposals, if necessary, for practical recommendations for follow up-actions. 11. Primary users of the evaluation results will be the Program managers and program staff implementing the various activities, as well as the WIPO Director General. 12. IOD will also share the evaluation report with key evaluation stakeholders participating in this evaluation once it has been completed and a redacted version has been prepared. A sample list of stakeholders groups has been provided in Annex As per WIPO Oversight Charter 2, the Director, IOD shall publish the evaluation report on the WIPO website within 30 days of their issuance. If required to protect security, safety or privacy, the Director, IOD may, at his/her discretion, withhold a report in its entirety or redact parts of it. (D) SCOPE 14. The period covered will be During this period, the program had a total budget allocation of 51 million Swiss francs 3 to invest in the achievement of multiple expected results, as per the relevant Program and Budget documents for each biennium under consideration ( , and ). 2. WHAT ARE WE EVALUATING? 15. The evaluation will assess activities implemented to contribute to the achievement of the Program 3 expected results, using the main evaluation criteria of efficiency, effectiveness, sustainability and relevance. 16. The evaluation will assess the questions below sorted by each criterion and taking into consideration cross cutting issues such as gender, participation, collaboration and inclusion to the extent they were identified in the program planning documents for the period in question: 2 WIPO Financial Regulations and Rules Annex I, page 5, paragraph 39 3 This includes personnel and non-personnel costs, as well as the budget after transfers for 2010/13 and the proposed budget for 2014/15. [Do you mean personnel and non-personnel resource allocations or resources expended?]

4 EVAL CONCEPT NOTE 4. (A) QUESTIONS ON RELEVANCE: (a) To which extent were the Program objectives and strategies at the various points in the period being assessed consistent with the needs of their Member States, beneficiaries and partners? and to what extent are they still relevant? (b) How were the needs and interests assessed? (c) Did the program activities in the area of development, copyright and legal advice services meet the needs of the various stakeholders groups, including gender? (d) Are the Program results contributing to the national strategies, policies in the area of copyright and related rights? (B) QUESTIONS ON EFFECTIVENESS 17. The degree of achievement of expected results contributing to the achievement of the objectives of Program 3 and Strategic Goals I, III, IV and V: (a) To which extent has the Program achieved its expected results as they have evolved over the years and contributed towards the achievement of WIPO s Strategic Goals? (b) During the implementation, were there systematic and appropriate efforts to include various groups of stakeholders? (c) How adequate is the Program design including its results framework and monitoring systems for decision-making and for measuring progress? (d) Is monitoring data collected and disaggregated according to relevant criteria such as sex, age, location and other criteria? (e) What have been the effects (positive or negative) of achieved results? (f) Have there been unexpected results of WIPO s activities? If so, what have been their key effects on the Program and achievement of WIPO s Strategic Goals? (g) What were the key strengths and weaknesses in managing the program effectively? (C) QUESTIONS ON EFFICIENCY 18. The efficient use of resources deployed to achieve results: (a) Were the activities adequately resourced (both human and financial) to implement the relevant activities, to target the relevant stakeholders, to monitor and to evaluate the program in order to deliver the expected results and objectives in a timely manner and with the requested quality? (b) Were there any factors that facilitated the inclusion of certain issues efficiently during the implementation? (c) What is the overall level of satisfaction among WIPO stakeholders including beneficiaries and Member States regarding the services provided?

5 EVAL CONCEPT NOTE 5. (d) To which extent were synergies and multiplying effects exploited within WIPO and its partners (Member States, multilateral and bilateral)? (e) How efficient was the organization and management of the Diplomatic Conferences and the Standing Committee on Copyrights and Related Rights (SCCR)? (D) QUESTIONS ON SUSTAINABILITY 19. Sustainability refers to the continuation of benefits from program results and activities after major programs and activities have been completed: (a) To which extent have the Program and partners considered sustainability criteria as part of their decisions and during implementation? (b) Did the Program activities promote sustainable changes in attitudes, behaviors or economic development through the implementation of copyright legal frameworks, copyright infrastructure, and creative industries? (c) Have there been program results and activities with a likelihood of continued long-term benefits? (d) To what extent do stakeholders have confidence that they will be able to build on the changes promoted by the Program? (e) What have been our partners commitments towards making program results and activities sustainable over time? 3. HOW ARE WE EVALUATING? 20. The evaluation will consist of three phases namely: Design and Review Phase, Field Phase and Reporting Phase. (A) EVALUATION DESIGN AND DESK REVIEW PHASE 21. During this phase, the evaluation team will analyze the consistency of the Program results framework for each biennium that is assessed. The evaluation team will review relevant Program documents 4, as well as documents shaping the Program strategy. On the basis of the information collected the evaluation team in consultation with Program 3 staff will: (a) Assess the quality of the design of the results based framework for each period assessed. (b) Revise the proposed evaluation questions presented in this concept note or propose a set of evaluation questions together with explanatory comments. (c) Develop evaluation questions, sub-questions and interview protocols following the appreciative enquiry methodology. 4 A list has been initiated in Annex 3 but it needs to be further complemented.

6 EVAL CONCEPT NOTE 6. (d) Prepare a list of internal and external stakeholders in collaboration with Focal Points. (e) (f) Further, define the evaluation scope and sampling strategies. Revise the proposed work plan for the finalization of the first phase. (g) Undertake a systematic review of relevant available documents, and conduct interviews with Program staff. (h) Prepare evaluation tools including questionnaires and interview protocols. 22. The evaluation team will submit its work plan with an indicative list of people to be interviewed, surveys to be undertaken, dates of visit, itinerary, and name of team members in charge to the Program focal points. 23. The evaluation team will report on preliminary findings and conclusion made during this phase as part of the draft evaluation report. (B) FIELD PHASE 24. The evaluation team will take necessary measures to ensure adequate contact and consultation with, and involvement of, different internal and external stakeholders in a participatory manner. To achieve this, the evaluation team will apply the appreciative enquiry method as part of this evaluation to identify the Program strengths, build on what works, and identify how it works to be able to replicate successful practices in future work. The evaluation team will also work closely together with key internal and external stakeholders during their entire assignment. 25. The evaluation team will triangulate (cross-validate) primary and secondary data; as well as design all evaluation tools such as surveys and semi-structure interviews using the appreciative enquiry method. 26. The evaluation team will use the most reliable and appropriate sources of information and will harmonize data from different sources to allow ready interpretation. 27. The work plan will be developed in a way that is flexible enough to accommodate any lastminute difficulties in the field. If any significant deviation from the agreed work plan or schedule is perceived as creating a risk for the quality of the evaluation, this will be discussed with the Chief of the Evaluation Section. 28. At the end of the field phase, the evaluation team will summarize its fieldwork, discuss the reliability and coverage of data collection, and present its preliminary findings in a meeting with the Program staff. In case field missions are undertaken as part of this evaluation then preliminary findings will be presented to national stakeholders, as well. (C) REPORTING PHASE 29. This phase is mainly devoted to the preparation of the draft final report. The evaluation team will prepare an evaluation report following the United Nations Evaluation Group (UNEG) Norms and Standards and its quality will be assessed by the Director IOD using the UNEG Quality Checklist for Evaluation Reports. The evaluation team will make sure that its assessments are objective and balanced, findings are accurate and verifiable, and its recommendations realistic.

7 EVAL CONCEPT NOTE A draft report approved by the IOD Director will be presented to the Focal Points and Program managers and circulated for comments to the Focal Points. Program managers and focal points will need to provide their comments within 10 working days. Based on comments expressed the Program staff through the focal points, the evaluation team will revise the draft report. Comments on the substance of the report may be either accepted or rejected. In the latter instance, the evaluation team will explain the reasons in writing and if requested by the commenters, attach the rejected comments to the final report in an annex. 31. Once the report is finalized, the IOD Director will share the report with the WIPO Director General, the Independent Advisory Oversight Committee (IAOC) and the External Auditor 5. As per WIPO Oversight Charter 6, the Director, IOD shall publish the evaluation report on the WIPO website within 30 days of their issuance. If required to protect security, safety or privacy, the Director, IOD may, at his/her discretion, withhold a report in its entirety or redact parts of it. (D) KEY ISSUES THAT COULD LIMIT THE EVALUATION PROCESS 32. The evaluation will happen during preparations for the next meeting of the Standing Committee on Copyright and Related Rights, which will take place between June 29 and July 3. This might limit the availability of program implementers. The evaluation team will consult program staff on the evaluation period to make sure that evaluation is not affected by other parallel work, and will make any necessary adjustments to the schedule for the evaluation to accommodate the commitments of the program staff. 33. The quality of information gathered will depend on the identification of the most relevant external and internal stakeholders. The evaluation team will prepare a list of stakeholders using Program information. Program staff will need to assist in the preparation of the stakeholders list and assure the quality and relevance of information. 34. The success of any country mission will depend on the support provided by the national counterparts. The more active the national counterparts are the better the evaluation results will be and vice versa. 5 In accordance with paragraph 30 of the Internal Oversight Charter. 6 WIPO Financial Regulations and Rules Annex I, page 5, paragraph 39

8 EVAL CONCEPT NOTE HOW WILL THE EVALUATION BE MANAGED? (A) MANGEMENT AND DELIVERABLES 35. Mrs. Julia Engelhardt will be the evaluation task manager. Mr. Claude Hilfiker, Chief Evaluator, will assure that the evaluation is undertaken according to the UNEG Norms and Standards. Mr. Thierry Rajaobelina, IOD Director, will assess the quality of the evaluation results and outputs, and approve any draft and final version of the evaluation report. Based on the above, the following deliverables shall be produced: (a) A revised version of the Concept Note Proposal including stakeholders list, list of program activities and sample activities for the evaluation, evaluation tools such as questionnaires and interview protocols. (b) A draft English written evaluation report with findings, conclusions and if necessary recommendations for improvement. (c) Final report edited for publication. (B) TIME TABLE 36. The main evaluative steps and tentative deadlines for the deliverables will be: (a) Design and desk review phase including revision of the present Concept Note Proposal (collection completed on April 17, 2015). (b) Field phase including any field mission preparation and including interviews in WIPO and at national level (June 2015). (c) Reporting phase including management response on any recommendations (i) Delivery of draft report on End of July, 2015 (ii) Delivery of the final report on End of August, 2015 (d) Follow up on recommendations if any beginning of September 2015.

9 EVAL CONCEPT NOTE 9. ANNEX 1: RESULTS FRAMEWORK PROGRAM 3- COPYRIGHT AND RELATED RIGHTS Expected Results Performance Indicators Baselines Targets I.1 Enhanced cooperation among Member States on development of balanced international normative frameworks for IP and agreement on specific topics on which international instruments are agreed Progress towards agreement on current issues on the SCCR agenda Two Treaties adopted in biennium 2012/13 (as to date). Two Diplomatic Conferences convened during the biennium 2012/13 One Diplomatic Conference convened and one Treaty adopted in the biennium 2014/15 I.2 Tailored and balanced IP legislative, regulatory and policy frameworks No. of countries that have ratified the Beijing Treaty No. and % of countries that have provided positive feedback about WIPO s legislative advice None Data not available 30 countries 15 countries III.2 Enhanced human resource capacities able to deal with the broad range of requirements for the effective use of IP for development in developing countries, LDCs and countries with economies in transition IV.2 Enhanced access to, and use of, IP information by IP institutions and the public to promote innovation and creativity No. of ratifications to the WIPO Internet Treaties No. of countries that have ratified the Marrakesh Treaty on the VIP No. of governments and CMOs signing an agreement with WIPO to develop a new transparency, accountability and governance quality assurance standard % of creators satisfied with the training provided on using copyright data and information for the efficient management of their copyright % of participants positive rating of the usefulness of copyright related capacity building meetings and workshops No. of Member States that take initiatives to improve their use of the copyright and related rights system to exploit the economic potentials of their cultural works and products No. of trusted intermediaries (TIs) and rights holders (RHs) having joined the TIGAR system network, including from developing countries and LDCs 181 (end 2012) 190 None n/a Not yet started 60% 20 countries Four governments and six CMOs 70% satisfactory rate 70% agree or strongly agree 60% of countries that requested technical assistance 60% of countries that received assistance 19 TIs and 40 RHs 24 TIs and 46 RHs (cumulative) No. of copyright protected works distributed among TIs and made accessible to VIPs across borders through the TIGAR system network % of individuals satisfied with the capacity building support on copyright infrastructure 400 1,000 cumulative Data not available 60% No. of institutions using GDA Use of the WIPO Collective Management Reference Database No. of legal instruments, guidelines, statements of principles other than Treaties agreed to or endorsed by the stakeholders concerned in areas such as IP and Sports and Copyright in the Digital Environment Data not available One (WIPO Review of Contractual Considerations in the Audiovisual Sector) 100 users One

10 EVAL CONCEPT NOTE 10. Expected Results Performance Indicators Baselines Targets IV.4 Enhanced technical and knowledge infrastructure for IP Offices and other IP institutions leading to better services (cheaper, faster, higher quality) to their stakeholders V.2 Wider and better use of WIPO economic analysis in policy formulation No. of governments and CMOs signing an agreement with WIPO to re-engineer WIPOCOS No. of CMOs in developing countries and LDCs participating in regional and global networks facilitated by WIPO % of governments that report positively on the improved effectiveness (and governance) of copyright offices and other institutions in the country Use of WIPO economic studies on copyright by governments and NGOs in decision-making Not yet started 0 (zero) Four governments and five NGOs 80% satisfactory rate 80% of countries that received technical assistance Eight governments or NGOs using WIPO studies governments or NGOS using WIPO studies (cumulative) National initiatives to develop further statistics on the creative industries based upon WIPO s work in the field 10 national initiatives 15 national initiatives (cumulative) MAJOR RISKS AND MITIGATION STRATEGIES Risk(s) to the Program achieving its Results Mitigation plans in place or under implementation Technology and Innovation: Inability of the copyright system to adapt to rapid technological and media changes threatens to undermine the relevance of the copyright system. Continuation of awareness-raising activities and initiatives to demonstrate the importance of copyright and to confirm its value and 'fit' in the evolving technological landscape. INTERNATIONAL TREATIES ON COPYRIGHT AND RELATED RIGHTS (a) (b) Beijing Treaty on Audiovisual Performances (not yet in force) Berne Convention for the Protection of Literary and Artistic Works (c) Brussels Convention Relating to the Distribution of Program-Carrying Signals Transmitted by Satellite (d) Geneva Convention for the Protection of Producers of Phonograms Against Unauthorized Duplication of Their Phonograms (e) Marrakesh Treaty to Facilitate Access to Published Works for Persons Who Are Blind, Visually Impaired, or Otherwise Print Disabled (not yet in force) (f) Rome Convention for the Protection of Performers, Producers of Phonograms and Broadcasting Organizations (g) (h) WIPO Copyright Treaty (WCT) WIPO Performances and Phonograms Treaty (WPPT)

11 EVAL CONCEPT NOTE 11. ANNEX 2: LIST OF STAKEHOLDERS The list below needs to be elaborated in collaboration with Program 3. Stakeholders names and contact details will be provided by the Program Managers. The classification below is just an example. Relevance of Name Institutions Contact Details Stakeholders WIPO staff members who in one way or another have collaborated with the Program Member States and Observers participating in the SCCR and the Beijing and Marrakesh Diplomatic Conferences Copyright Offices working with WIPO National counterparts for key projects such as DA projects Public Institutions working with WIPO Copyrights and Performance Associations, NGOs Beneficiaries of major projects Collective Management Organizations benefiting from or working directly with the Program Others to be specified

12 EVAL CONCEPT NOTE 12. ANNEX 3: LIST OF DOCUMENTS TO BE REVIEWED Here is a list of the initial documents that will be reviewed as part of this evaluation. WIPO Program and Budget documents for 2010/11, 2012/13 and 2014/15 WIPO Program Performance Reports for 2010/11, 2012/13 and 2014 Reports from the SCCR for the period under evaluation SCCR meeting documents Diplomatic conference meeting documents Development Agenda progress reports and evaluation reports Other reports will need to be specified during the discussions with the Program