TERMS OF REFERENCE SOUTH AFRICA FINANCIAL SECTOR DEVELOPMENT AND REFORM PROGRAM INDEPENDENT PROGRAM EVALUATION. I. Background

Transcription

1 TERMS OF REFERENCE SOUTH AFRICA FINANCIAL SECTOR DEVELOPMENT AND REFORM PROGRAM INDEPENDENT PROGRAM EVALUATION I. Background 1. The South African financial system contains a highly developed and well-capitalized banking and financial sector which, however, caters mainly to the advanced segment of the South African economy. This first-world financial sector is highly concentrated relying on competition among four large banks and exists side by side with a developing economy very similar to those found throughout Sub-Saharan Africa, which is severely underserviced. The challenge is therefore how the financial sector can promote economic growth, and expand products and services to meet the needs of the population that make little use of the financial system and the small and medium sized enterprises (SMEs) that do not utilize the banking system and capital markets to obtain funds for growth, while at the same time safeguarding financial stability. 2. To address this challenge, the South Africa Financial Sector Development and Reform Program (FSDRP) was launched in July 2014 with an initial contribution from the Swiss State Secretariat for Economic Affairs (SECO) and is scheduled to complete in June The objective of the FSDRP is to assist the Government in strengthening financial stability and improving financial inclusion, through analytical and advisory services. It is expected that advancing the soundness and inclusion of the financial sector will support the development needs of the two main pillars that focus on: (i) financial sector regulation, integrity and stability and (ii) financial inclusion, as discussed below. a. Pillar I: Financial Sector Regulation, Integrity and Stability 1. Strengthening crisis management and resolution: The overall objective of this component is to advise the Government regarding the strengthening of the financial institutions resolution regime and crisis management system that is in line with the FSB s Key Attributes, enabling the authorities to respond effectively in times of crises, as well as a sound deposit insurance system in line with the IADI guidelines, contributing to financial stability and protecting depositors. 2. Strengthening market conduct regulation and supervision: The objective of this component is to advise the Government on strengthening market conduct regulation and supervision, contributing to a stable and sound financial sector that protects consumer s rights. 3. Strengthening the development and regulation of capital markets: The overall objective of this component is to advise the Government in strengthening regulatory and oversight capacity for capital markets development including brokerage firms regulation contributing to a stable capital market and a stronger enabling environment, as well as developing new instruments for infrastructure finance and supporting the development of the financial system. 4. Reviewing the AML/CFT regulatory framework: The overall objective of this component is to provide analytical and advisory services to the government that will

2 contribute to a strengthened AML/CFT framework that gives due consideration to financial inclusion aspects, supporting a sound financial sector. b. Pillar II: Financial Inclusion 1. Innovative retail payment instruments and delivery channels leveraged to deepen financial inclusion: The overall objective of this component is to provide analytical and advisory services to the government to strengthen the legal, regulatory, and policy framework for enabling the development of innovative retail payment instruments and services, resulting in increased usage of accounts and electronic payments to facilitate a deepening of financial inclusion. 2. SME Finance: The overall objective of this component is to advise the Government in improving access to credit for SMEs, therewith contributing to SME development and growth, and strengthening the legal framework for movable asset lending through advice on the establishment of an online centralized movable asset registry. 3. Debt rehabilitation and insolvency provisions for individuals: The overall objective of this component is to provide analytical and advisory services to support the improvement of the legal and institutional framework for debt rehabilitation and personal insolvency, effectively responding to the challenge of over-indebtedness. II. Scope of Work 3. The World Bank is seeking an evaluation firm to carry out a review of the FSDRP. The main objective of the assignment is to carry out an independent assessment of the design and delivery of the different components under the FSDRP, review associated log frame, including theory of change, intervention logic, indicators, baselines and targets. The main motivation for this evaluation is to draw lessons learned and receive forward looking guidance for preparing a second phase of the program. The firm will identify any implementation course corrections which are necessary to attain FSDRP objectives, to ensure relevance, and increase where possible its effectiveness, efficiency and sustainability.. 4. The consulting firm will deliver an independent program evaluation (IPE) report detailing findings from the desktop review and field mission, including a response to the evaluation questions listed below, lessons learnt and recommendations. In addition, the consulting firm will prepare a presentation summarizing the results of the IPE. a. Relevance The IPE should address the issue of relevance and rationale in the context of implementation of the program, as well as governance and management thereof. 1. Is the program addressing priority areas based on the government s strategic focus and the potential to achieve impact? 2. To what extent did the two pillar architecture (financial stability and inclusion) of the program enhance the program s relevance? Were the two pillars complementary to each other and

3 3. The program has to date concentrated on local issues pertaining to financial stability and inclusion in the country. Given South Africa s important role in the region, how could a regional dimension be built into a second phase of the program to further strengthen the reach and relevance thereof? b. Effectiveness The issue of effectiveness should be addressed in the context of the extent to which the program has achieved its objectives and the major factors that influenced the achievement or non-achievement of the objectives, taking their relative importance into account. Key questions include: 4. What is the progress towards meeting the TFs stated objectives and how likely is the program to make a significant contribution toward achieving them at outcome level? 5. How effective is design and usage of log frame for tracking performance and monitoring intended outputs and outcomes of TF? 6. To what extent have the programs developed the capacity of the beneficiaries in a structured approach? With regard to the measurement of capacity development, has the development of capacity under the programs been measured? If not, what has to be put in place to measure it in the future? Present ideas how to include this in a logframe for a potential next project phase. c. Efficiency The IPE should address efficiency in the context of the extent to which the program can be justified by its results, taking alternatives into account. Benchmarking to similar programs may be considered where appropriate. Key questions to be answered include: 7. Is value for money provided by activity implementation, especially with a focus on costs of individual interventions in relation to their outcomes and the likelihood of achieving them? 8. How is the efficiency regarding a) reporting; b) governance, and c) risk monitoring. 9. What are challenges and risks encountered at the program and intervention levels? Where there any considerable differences between pillars and components? d. Sustainability The issue of sustainability of individual interventions should be tackled by addressing the following questions: 10. What is the likelihood of the achieved results to be maintained after the end of the program (be it reforms, studies, technical assistance)? 11. Are the activities designed in way that they can be maintained in the longer term? For the future, how can we do even better in this regard? How can be ensured that the supported regulations/reforms are really implemented?

4 12. What features would the program need to incorporate in order to ensure sustainability of interventions pursuant to program completion? e. Impact The objective of the impact assessment is not to quantify impact but rather to assess the World Bank s theory of change, which assumes that the interventions under the TF offer clear benefits to final beneficiaries. 13. What are the positive or negative impacts of the TFs operations? Are there any unintended outcomes? 14. Please assess the World Bank s theory of change and suggest improvements for a second phase of the program III. Deliverables 5. The main deliverable is a clear and concise IPE report detailing: (i) scope of the evaluation and evaluation methods, (ii) description of the development intervention, (iii) presentation and interpretation of the factual evidence in relation to the evaluation questions, (iv) lessons that may have implications for future development interventions or may be relevant for wider application, and (v) recommendations for improvements of the program and for a potential second phase. The table below outlines the consultant s specific deliverables. Outputs Date Deliverable 1: Outline of IPE report March 31, 2017 Deliverable 2: Findings workshop June 30, 2017 Deliverable 3: Draft IPE report July 31, 2017 Deliverable 4: Final IPE report August 31, 2017 Deliverable 5: Presentation of IPE report October, 2017 The above due dates may be subject to change, as circumstances necessitate. IV. Evaluation Method 6. The evaluation will primarily make use of qualitative evaluation methods, namely: a. Desktop research of the available documentation SECO and the World Bank will put all the relevant information on the project at the disposal of the evaluation team. SECO will particularly provide the following information: decision notes and credit proposals, contract documents, semi-annual reports. The World Bank will make available expert mission reports, conceptual work produced under the

5 project, all relevant information regarding budget and use of financial resources, outputs from activities. Based on the desktop research, the evaluation team will produce an outline of the IPE report, detailing: i) their understanding of the evaluation questions and propositions of adjustments to these questions if considered necessary, ii) the methodology for assessing the two pillars of the program and iii) an outline of the report. b. In-depth and semi-structured interviews of key stakeholders The consulting firm will interview the teams at the World Bank currently / formally in charge of the implementation of the programs, selected SECO staff including country office staff, as well as key stakeholders in South Africa. 7. Further, the evaluation will adhere to evaluation best practice including the Development Assistance Committee (DAC) guidelines on quality standards for development evaluation. IV. Reporting and Travel Arrangements 8. The consultant will report to Gunhild Berg (Senior Financial Sector Specialists, Finance and Markets Global Practice). The assignment will require travel to South Africa, with the first visit planned for April All travel arrangements will be agreed and coordinated with the World Bank team. V. Timeframe 9. The assignment is expected to take approximately 60 days, starting in February 2017 and concluding on October 31, VII. Qualifications 10. The consultants should be senior experts with significant experience in the evaluation of financial sector programs, including: Proven track record in the evaluation of financial services programs (financial stability and financial inclusion), specifically financial sector regulation and market conduct, capital markets, AML, payment systems, SME finance, debt rehabilitation and personal insolvency; Experience with the evaluation of legal reforms in a country/countries which have undertaken changes to financial sector legal frameworks; Demonstrated ability to identity gaps, propose and implement solutions for financial sector clients; Proven knowledge of evaluation methods relevant to the ToRs; Knowledge of the South African financial sector; Experience in working with donor organizations; Excellent interpersonal and negotiating skills and ability to interact with staff at all levels; Strong writing and presentation skills; Fluency in English.

6 Annex 1: List of Assessment Criteria Criteria RELEVANCE Projects/Programmes are addressing important development issues of the partner country and of concerned beneficiaries Projects/ Programmes are aligned with national priorities and policies of partner country and partner institutions Projects/ Programmes are consistent with SECO priorities and focus on its comparative advantage Interventions are coordinated with other project/programs in the concerned sector and are complementary EFFECTIVENESS Objectives achieved at output level (as defined in logframe) Objectives achieves at outcome level (as defined in logframe) Objectives achieved at impact level (as defined in logframe) EFFICIENCY The approach is efficient for achieving the intended results Projects/Programmes. management, monitoring (logframe) and steering mechanisms are in place and effectively used for the efficient implementation of the activities The chosen Implementation Modality ensures high competence and performance Projects/Programmes. are cost-effective SUSTAINABILITY Results (outputs / outcomes / benefits will last beyond / continue after projects / programme s closure Local institutions/capacities have been strengthened to sustain results Financial sustainability has been achieved Assessment categories Categories Highly satisfactory Satisfactory Unsatisfactory Highly unsatisfactory Acronym HS S U HU

7 Annex 2: Standard Format for Evaluation Reports The format for the presentation of the evaluation given here is to be considered as minimum standard. Content page Acronyms and abbreviations Acknowledgements and disclaimer Executive summary Introduction Purpose and objectives of the evaluation Scope of the evaluation, short statement on the evaluation methods used Description of the development intervention Context of the intervention, including policy and institutional context Description of the intervention and the intervention logic and the implementation arrangements Findings Presentation and interpretation of the factual evidence in relation to the evaluative questions. Conclusions Assessment by the evaluators of the intervention results against the expected results (as identified at the planning stage or as reconstructed by the evaluators). Lessons learned Lessons that may have implication for the future of the development intervention or may be relevant for wider application. Recommendations Proposals for improvements for the client and users of the evaluation. Annexes: - TORs - List of stakeholders consulted - Detailed description of the evaluation process and methodology: description of the evaluation process, the methodology used (including any limitations of this method), information sources (including any data issues), stakeholders participation and consultation.

8 Annex 3: FSDRP Program Document Annex 4: DAC Guidelines