ECOWAS MONITORING AND EVALUATION MANUAL

Transcription

1 ECOWAS MONITORING AND EVALUATION MANUAL 1

2 INTRODUCTION Objective of the Manual The Economic Community of West African States (ECOWAS) is a regional integration body established by the Treaty signed in Lagos on 28 May 1975 in order to raise the living standard of its peoples, and to maintain and enhance economic stability, foster relations among Member States 1 and contribute to the progress and development of the African continent. In order to achieve these objectives, ECOWAS seeks to promote cooperation and integration designed to culminate in the establishment of a West African economic union. The ECOWAS Vision 2020 contends that the improved wellbeing of the West African peoples is contingent upon the continuous evaluation of the performance and impact of the services and activities organized by ECOWAS to address the development challenges of the region. This ECOWAS monitoring and evaluation manual seeks mainly to formalize and harmonize the procedures and tools for monitoring and evaluation and for a critical appreciation of progress made in the regional integration process. It also serves as a framework for launching the automation and operationalization of the monitoring and evaluation system. The monitoring and evaluation system has, a centralized mechanism for collecting, processing and analysing information on all the regional integration actions undertaken by the main stakeholders involved in the implementation process. These are the ECOWAS Member States and institutions, and such actions may be financed through ECOWAS own funds or by development partners. The monitoring and evaluation system described in this manual is a learning process, based on stakeholder practices as relate to the different structures of the organisation. It is therefore quite flexible and simple, putting existing tools to use, and capable of taking complementary tools on board during implementation. The Scope of Monitoring and Evaluation In order to redirect the role of ECOWAS to its primary mission of regional integration and sustainable economic development, its institutions were restructured in 2006 with a view to improving the Community s organizational structure and its management systems and procedures. As part of this evolution, the Authorities adopted the ECOWAS Vision 2020² in June 2007, which marked a decisive point in the process of transforming the ECOWAS of States into an ECOWAS of Peoples. The Strategic Plan was developed for the realisation of Vision Based on the ECOWAS fundamental principles 2 (Article 4 of the Treaty), and an analysis of the prevailing circumstances, the (regional) Strategic Plan identifies the following six strategic pillars: Peace, security and good governance; 1 Benin, Burkina Faso, Cape Verde, Cote d Ivoire, The Gambia, Ghana, Guinea, Guinea Bissau, Liberia, Mali, Niger, Nigeria, Senegal, Sierra Leone, and Togo. 2 Equality and inter-dependence, solidarity and collective self-reliance, promotion of a peaceful environment, respect and protection of human and peoples rights, promotion and consolidation of a democratic system of governance, transparency, and a just and equitable distribution of the costs and benefits of regional cooperation and integration. 2

3 Cooperation and development; Competition and fairness; Economic and monetary integration; Institutional capacity; International prospects. It is on the basis of these development pillars selected by the Community that six strategic priorities (or objectives) were set, covering a five-year period beginning from These constitute open regional projections, designed to determine the general direction that the Community should take for the realization of Vision The priorities are the following: 1. To promote good governance and improve on the Mechanism for Conflict Prevention, Peace-keeping and Security; 2. To promote a competitive business environment; 3. To support development and cooperation in the region; 4. To strengthen economic and monetary integration; 5. To strengthen institutional capacities; 6. To strengthen the mechanism for integrating into the global market Each of these strategic priorities is underpinned by strategies capitalizing on strengths, eliminating weaknesses, taking advantage of opportunities and avoiding threats identified by the situation analysis. They are then presented as regional strategic objectives and thereafter broken down into regional strategic sub-objectives. The different Institutions draw on these objectives and sub-objectives for the definition of their respective objectives on the basis of which medium-term plans of action are formulated, backed by a monitoring and evaluation plan and a performance assessment framework. The latter requires at least the following items of information on each programme or project: Objective, Expected Outcome (or Product), Target, Performance Indicator, Assessment Methodology (including frequency of assessment), Officer in Charge and Information Source (or Means of Verification). These elements can be classified at two levels: the strategic level (corresponding to medium-term strategic objectives) and the operational level (corresponding to short-term activities and initiatives). This information is recorded in the reference table used as a performance assessment framework, and it must be readily available for each programme, project or activity. The monitoring and evaluation mechanism outlined in this manual is an inventory of practices and activities to be undertaken in support of the different operational programmes for implementing the strategic priorities mentioned above, in order to ensure their realization within the defined time frame. Institutional Framework There are three main groups of stakeholders involved in regional integration. These are the Member States, ECOWAS Institutions and development partners. Depending on the needs and circumstances, any one of these groups of stakeholders may mobilise other stakeholders or beneficiaries such as the people, civil society organizations, individuals, enterprises or economic and political groupings, etc, for production, distribution, consumption or advocacy purposes. 3

4 Under the provisions of Article 7, paragraph 3 (b) of the Treaty, the power to monitor and evaluate progress made towards regional integration is vested in the Authority of Heads of State and Government. In its capacity as the supreme organ of the Community, the Authority is to oversee the functioning of Community Institutions and follow-up implementation of Community objectives. The Authority is assisted in this task by the ECOWAS Council of Ministers, which is responsible for the functioning and development of the Community (Article 10, paragraph 3), and by the ECOWAS Commission which, is responsible for the submission of a report on the activities of the Community to all meetings of the Authority and Council (Article 19, paragraph 3e), and submission of proposals and preparation of such studies as may assist in the efficient and harmonious functioning and development of the Community (Article 19, paragraph 3h). For the purposes of this function, the Commission, in the person of its President, uses the reports of the other Institutions and the technical committees. At the level of each Institution or structure involved in the implementation of the regional integration agenda, the Head of Institution, Departmental Heads, Directors and Division Heads, Programme Officers, and focal points are all involved, in their respective capacities, in the functioning of the monitoring and evaluation system. The Strategic Planning Unit, in view of the specific roles it plays - planning upstream, coordination of the programming processes, ensuring the required coherence among the operational activities and their links with the ECOWAS strategic objectives the Unit is a key partner in the monitoring and evaluation process. Of equal significance is the control system of the Community which is made up of the Financial Controller, the Internal Auditor and the office of the External Auditors. Concepts and Definitions Three monitoring and evaluation functions are addressed within the framework of this manual, namely, control, monitoring and evaluation. The first two functions are generally referred to as monitoring-control, or simply monitoring. However, the control function constitutes the bedrock of any monitoring and evaluation system. The control function is exclusively concerned with the implementation status of work programmes for a given period. Monitoring uses the information provided by control to determine the extent to which planned actions and investments have been executed in accordance with the approved schedule, allocated resources, and expected outcomes. Three factors are thus taken into account: the effectiveness of the tools used in terms of organizational management and technological procedure; the amount of financial, human and material resources mobilized; and the quantity and quality of outputs. Evaluation, for its part, is an exercise of limited duration. It is designed to achieve a systematic and objective assessment of the relevance, performance (effectiveness and efficiency) and the success of a programme at a given point in its implementation, and on completion (see annex for the interrelations and the complementarity between the two functions of monitoring and evaluation, as well as the concept of results-based monitoring and evaluation). Characteristics of the ECOWAS Monitoring and evaluation System - A broad-based participatory process aimed at inculcating in all stakeholders a sense of ownership and commitment without which on-the-ground implementation would be difficult; 4

5 - Whereas the monitoring function is internal to the organisation, evaluation may be both internal and external (in the interests of achieving the objectivity and qualified expertise needed to generate an independent opinion). The internal evaluation mechanism within the organisation is used mainly for the purposes of performance assessment in terms of progress towards the attainment of desired objectives; - Transparency in methodology: the system must be based on the use of standardised tools in order to facilitate effective data exploitation and analysis; - Neutrality with regard to implementation of programmes: in order for the M&E service to provide objective opinion on the implementation of programmes, it is not involved in the execution of programmes; rather the M&E service provides support and assistance to the services in charge of implementing programmes. - Responsibilities: the functions of the different stakeholders involved in the running of the system must be clearly defined. The principal functions of the Monitoring and evaluation Unit are: (i) to verify and collate periodic reports (including annual reports) emanating from the other technical units; (ii) and to manage the information system and its data bank. Content of the Manual Apart from an introduction containing background information, definitions and annexes, the manual comprises the following four chapters: 1. The monitoring and evaluation system 2. Procedures and tools for programming and monitoring 3. Procedures and tools for rendering account and producing reports 4. Evaluation procedures. Chapter I defines the expectations of the different stakeholders involved in the monitoring and evaluation process, and identifies its objectives. The chapter also presents the monitoring and evaluation model, with a focus on institutions and the mechanism s information system. Chapter II describes the specific procedures and tools used for information programming, organisation and management, while Chapter III addresses the issue of report preparation and management at each level of the monitoring and evaluation mechanism. The fourth and last chapter of the manual focuses on the procedures and criteria ensuring high performance in programme implementation by the different stakeholders in the ECOWAS regional integration process. An Annex containing a detailed breakdown of the different operational tasks to be undertaken is appended to the manual. 5

6 PART ONE THE ECOWAS MONITORING AND EVALUATION SYSTEM 6

7 THE ECOWAS MONITORING AND EVALUATION SYSTEM Introduction The monitoring and evaluation system for all ECOWAS institutions is a flexible mechanism capable of harmonising the different systems internal to each ECOWAS Department, Institution and structure. It is designed to ensure optimal centralisation of data collection procedure at the different sectoral, decentralised levels, and to provide a better indication of the overall progress recorded in the regional integration process. Consequently, every unit participating in the system may adopt additional tools specifically designed for their needs, for the purposes of reporting on their activities to other partners. The tools described in this manual constitute the bare minimum to which all stakeholders in the ECOWAS regional integration process are required to subscribe. The mechanism presented in this part of the manual is designed to enable other stakeholders to comply with the prescribed procedures. All structures involved in the implementation of the monitoring and evaluation process are required to ensure compliance with these procedures in the discharge of their respective obligations, particularly as pertains to the timely production and availability of information on their areas of activity or sectors of intervention. This part of the manual is divided into sections under the following subject headings: i) Framework for the Application of the Prescriptions of the M&E Manual; ii) Institutional Framework; iii) Expectations of the Different Stakeholders and Information Needs; iv) Organisation of the Mechanism at the Level of Stakeholders in the ECOWAS Monitoring and Evaluation System. This chapter identifies the stakeholders involved in the monitoring and evaluation system, and based on their respective expectations and the nature of their information needs, the most appropriate institutional mechanism is outlined to ensure its smooth functioning and the realization of the monitoring and evaluation objectives. The chapter also spells out the application framework for the monitoring and evaluation manual, with a focus on the information-generating process which underlies the effective functioning of the system. Framework for the Application of the Monitoring and evaluation Manual The monitoring and evaluation manual is applicable to all the programmes, plans, projects and activities implemented within the framework of the ECOWAS regional integration process by the Institutions (including financial institutions 3 ) and the Member States, or any other structure or partner commissioned for that purpose by the Institutions or the Member States. The scope of intervention covers all the areas of regional integration contained in the ECOWAS Treaty and all the related documents (Protocols, Supplementary Acts, Decisions, Regulations, Guidelines, Declarations, Recommendations, Legal Opinions and Reports of the Authority or of Council), irrespective of the nature of the resources or the modalities for their financing. 3 such as EBID 7

8 The major stakeholders in the monitoring and evaluation system are: the Authority of Heads of State and Government, the Council of Ministers, the ECOWAS Commission and other ECOWAS Institutions (as well as their respective staff) and the Member States. Under the provisions of Article 6, paragraph 2 of the ECOWAS Treaty, all the Institutions shall perform their functions and act within the limits of the powers conferred on them by this Treaty and by the Protocols relating thereto. With regard to the monitoring and evaluation function, the President of the ECOWAS Commission (assisted by the Vice President) coordinates the activities of all the Institutions of the Community, while the Heads of Institution and Commissioners, are responsible, within their respective sectors of activity, for the control and monitoring of the correct execution of ECOWAS integration policies as well as the implementation of decisions taken by the organisation s decision-making bodies. Directorates and Divisions are responsible for the monitoring of programme, project and activity implementation, while Programme Officers and focal persons (in the Institutions and Member States) are responsible for implementation at the operational level. The monitoring and evaluation system of the ECOWAS regional integration activities acknowledges the complementary role played by other supervisory bodies within the Institutions, especially those of Financial Control and Internal Audit, with regard to the preparation of periodic reports on the budgetary, financial and accounting management of the Community (Regulations C/REG.32/12/07 and C/REG.27/12/06), and the assessment of risks and checks to be put in place (Regulations C/REG.32/12/07 and C/REG.26/12/06), as well as the annual audit of the institutions of the Community (Article 75 of the revised treaty.) The organizational information (and feedback) mechanism comprises the strategic and steering stakeholders, the day-to-day decision-making authorities and operational stakeholders responsible for data generation. The Monitoring and evaluation Unit has a cross-cutting support and coordinating role in the data management and reporting system. The programming, implementation and reporting process is supported by consultations and meetings of stakeholders from various sub-systems of the organizational mechanism. Objectives of the Monitoring and Evaluation System In specific terms, the ECOWAS monitoring and evaluation system seeks to: Establish a reputation for generating accurate data which can be used in the preparation of the different activity reports of the ECOWAS Commission and other Institutions of the Community; Monitor the progress of activities throughout programme implementation in order to be in a position to take timely corrective measures for observed shortcomings or deviations; Learn useful lessons from previous activities for better definition and implementation of future activities; Determine as systematically and objectively as possible the relevance and effectiveness of activities undertaken as well as their effects and impact on target groups. 8

9 Expectations of the Different ECOWAS Stakeholders Expectations regarding monitoring and evaluation are dependent on the level of responsibility held by the interested party in the institutional structure of the organisation, and correspond to well-defined functions. In accordance with the hierarchy of responsibility, and the concomitant chain of accountability, each level will always depend on the level below for the information it needs to fully perform its functions. Consequently, the front line implementation level comprised of Programme Officers in the Directorates and similar services, as well as focal persons and other stakeholders, in whom the bulk of ECOWAS goods and services are invested, play a key role in generating the necessary information for control and surveillance purposes within the ECOWAS programme monitoring and evaluation framework. 9

10 Table 1: Expectations of Stakeholders and Corresponding Monitoring and evaluation Functions STAKEHOLDERS EXPECTATIONS Monitoring and evaluation FUNCTION Authority of Heads of State and Government and Council of Ministers Monitoring of the conformity of strategic guidelines with regional policies. Validation of annual reports Decision-making on vision and strategy. President of the ECOWAS Commission Coordination of programming and monitoring of strategic guidelines. Programme evaluation and audit. Coordination of operations programming and monitoring and evaluation actions within Departments and specialized Institutions. Coordination of strategic surveillance. Decision-making within limits of powers delegated by the highest authorities. Coordination of the monitoring of the implementation of recommendations resulting from the evaluation process. Commissioners/Heads of Institution Coordination and day-to-day management decision-making within their Departments. Supervision of the outcomes of programmes from each Directorate and Division, and submission of reports thereon to the decision-making bodies. Monitoring and Evaluation Unit Coordination of data management and operations programming. Data collection and centralisation. Recording, analysis, reporting and dissemination. Reporting to the President through the Vice President. Directors and Heads of Division Control and supervision of programme and project activities within their Directorates. Centralisation of data within the Directorate Monitoring of programme implementation within the Directorates and preparation of summary reports. Reporting to Departmental and Institutional Heads. Programme Officers Operational planning and application of tools for collection of data on activities undertaken. Monitoring of implementation of activities & activity reports, reporting to Directors/Unit Heads Focal persons from Member States Monitoring of progress and data collection at the Community level. Transmission of data to appropriate operational Divisions. 10

11 Information Needs The following information needs would have to be supplemented and improved upon at the various levels of responsibility, as application of the prescriptions of the manual advances, in order to adapt them to changing contexts and new strategic guidelines. Table 2: Information Needs for the Purposes of Monitoring and evaluation according to Different Levels of Responsibility Level of Responsibility Information Needs for Monitoring and evaluation Purposes Authority of Heads of State and Government and Council of Ministers Progress made in implementing the regional integration policy; President and Vice- President of the ECOWAS Commission Progress made in achieving the strategic results expected of the ECOWAS Commission; Medium-term prospects for achievement of the results expected from the ECOWAS Commission in terms of strategy, actions and resources; Level of compliance with the guidelines set by the Council of Ministers and the Authority of ECOWAS Heads of State and Government Commissioner and Head of Institution Directors and Division Heads Programme Officers Focal persons in Member States Progress made in achieving the results expected of the Department or Institution; Medium-term prospects for achievement of the results expected of the Department or Institution in terms of strategy, actions and resources; Progress made in achieving the results expected of the Directorate or Division; Medium-term prospects for achievement of the results expected of the Directorate or Division in terms of strategy, actions and resources; Overall technical and financial implementation status of activities; Technical implementation status of ongoing annual work programme budget (AWPB) according to Division or programme; Budgetary implementation status of the AWPB; Implementation status of contracts within the framework of ongoing AWPB; Difficulties in complying with procedures during AWPB implementation and lessons learnt for future activities; Impacts made within the project framework (particular case studies) Technical and financial implementation status of work programmes according to components, as defined for the implementation of ongoing AWPB; 11

12 Stakeholders Organizational Mechanism The mechanism is such that all the stakeholders within the monitoring and evaluation system are regularly informed on the implementation status of planned activities, direct and indirect impacts in terms of benefits, risks, opportunities and possibilities for interventions (decisions) in order to strengthen and/or redirect some of the observed trends. It comprises three levels: strategic and steering stakeholders; stakeholders in the central sub-system of the decisionmaking body; and operational stakeholders in charge of data generation. The strategic and steering level includes the President of the ECOWAS Commission (assisted by the Vice President). S/he is responsible for decision-making at the highest level, based on the performance analysis and strategic surveillance information prepared by the Monitoring and evaluation and Strategic Planning Units on the strategic monitoring of ECOWAS activities. The central sub-system of decision-making comprises the Commissioner or Head of Institution assisted by his/her Directors and Division Heads. It is responsible for the production and analysis of summary data for its area of competence. It sees to the centralization of departmental data and information from the Directorates on implementation and the status of work plans, thus contributing to the operational decision-making process. The operational stakeholders responsible for development of the database are mainly programme or project managers and the focal persons in Member States. They are responsible for the production of primary data for summary purposes. The information chain is such that the results are shared among all the stakeholders within the required time-frame. It forms an integral part of the institutional monitoring and evaluation framework. Data may be disseminated through meetings, workshops or other appropriate channels. (See information chain diagram) 12

13 Diagram 2: The Monitoring and Evaluation institutional Framework Actors responsible for Strategic Orientation and Direction Authority of Heads of State and Government Council of Ministers President of the Commission Assisted by the Vice President M&E Unit Strategic Planning Unit Actors of central sub-system of decision-making Heads of Department (Commissioners) Heads of Institution Directors Directors Acteurs du système opérationnel de production des données Programme Officers And Heads of Service Civil Society Focal Points of Member States Other implementing actors 13

14 Part Two Programming and Monitoring Procedures and tools 14

15 Introduction The monitoring and evaluation mechanism depends principally on the accuracy of the information needed for sound programming, and on information that can be used to produce a reliable account of the execution of planned activities. This requires that a standard set of procedures and tools is made available to the different stakeholders. The tools presented in this manual are developed from the harmonization of the various practices and tools existing within the ECOWAS institutions. The procedures and tools used are determined by the different stages in the methodology for the execution of the monitoring and evaluation function. Taken in sequence they are: i) Programming; ii) Preparation of the annual work programme budget presented as quarterly or monthly plans; iii) Data collection, processing, analysis and dissemination procedures; iv) Report preparation procedures within the monitoring and evaluation framework; and v) Evaluation procedures. The following are the points which will be given priority consideration in this part of the manual: - Programming procedures (PTAB, PTM, PTT); - Data collection procedures; - Data processing and analysis procedures A. Programming Programming is the annual projection of the activities to be implemented and budgetary resource requirements in order to achieve expected results. This exercise is participatory and contractual (cf. Organogramme 2). It involves the preparation of the Annual Work Programme and Budget (AWPB) of the Institution, based on the strategic plan and logical framework presented as Quarterly Work Plans (QWP) at the level of the Departments. The AWPB is a summary document presenting the major intervention areas of specific Departments and Institutions, and the required budget allocation for a particular year without presenting a corresponding detailed implementation strategy. At the end of the first semester, the implementation of the two first QWPs for the year should facilitate the preparation of a half-year report on ECOWAS activities, indicating the level of performance achieved after mid-term implementation of the annual work plan. At the end of the fourth Quarter, the same exercise will enable the Monitoring and evaluation Unit to present an annual evaluation and to prepare the annual general report of the ECOWAS Commission. The Report will serve as the basis for the preparation of the annual work plan for the following year. The activity planning process is described below. It aims at ensuring that all stakeholders, irrespective of their level, contribute to the identification of activities to be implemented as well as the implementation strategy. The annual work plan contains the following: Identification of activities and target objectives for each Directorate, indicating their link to the priority objectives of the strategic plan; 15

16 Description of target activities and objectives; Budgeted activity programmes for each strategic area; Timeframe of activities covering a twelve-month period; Officers in charge of each activity and their possible collaborators; Financing plan of activities for each financing source; Contracting plan, as appropriate. Procedure for the Preparation of the Annual Work Plan and Budget (AWPB) The AWPB preparation process is defined according to the stages described in the following diagram: Diagram on AWPB Preparation Process a. Identification of Annual Strategic Priorities 16

17 This activity can take the form of a workshop attended by the Head of Institution and his/her immediate collaborators, namely, Directors (including the Director of Finance), the Monitoring and evaluation Unit and the Strategic Planning Unit. The activity is aimed at redefining the major guidelines for the following year in conformity with the logical framework of the Institution and its strategic plan. The outcomes of the activity will enable the Commissioners and Heads of Institutions to start work on the preparation of the work plan for their respective Departments. This activity should be carried out early in May of the year (n-1) in strict conformity with the planning cycle. The conclusions of the workshop should be disseminated as a Circular from the Office of the President of the ECOWAS Commission, to all Departments and Heads of Institution. The Circular will contain the following information: Main objectives for the coming year; Financial resources to be mobilized for all Departments and Institutions; AWPB preparation timetable. Upon receiving the Circular, each Commissioner or Head of Institution will apprise his/her collaborators of the AWPB preparation guidelines. All necessary measures will be taken to ensure preparation of the respective budgets in conformity with the AWPB sample. A deadline will be given for submission of the documents. b. Preparation of the Annual Work Plan of each Department It is the backbone of the planning process. Based on the above-defined strategic guidelines, each Commissioner or Head of Department will organise his/her Department s preparation of its annual work plan. The needs of all levels will be expressed, and a workshop organised on the preparation of the work plan based on the model presented in the operational guidelines of the manual. It should be recalled that all actors in the Department need to take ownership of the work plan prepared at this stage. This activity should take place fifteen (15) calendar days after the annual strategic priority guidelines have been established. The participation of the Strategic Planning and Monitoring and evaluation units and the Finance Department is required in this regard. c. Consolidation of the Departmental Work Plans Each Department is required to submit its AWPB to the Strategic Planning Unit for consolidation and verification of the coherence of activities, implementation timetables contained therein, and their overall conformity with priority strategic guidelines. Consolidation of AWPBs is to be carried out for the annual work plan of the year (n) during the month of July (n-1), and under normal circumstances, within five (5) calendar days of receipt of each Department s AWPB. d. Presentation of Annual Work Plan and Budget After finalisation of the consolidated AWPBs, the results are submitted to the Offices of the President, Vice-President, Commissioners and Heads of Institution for comment. When such comments have been taken into account, the first draft of the ECOWAS AWPB is ready for validation by all major actors at all levels of the institutional hierarchy. 17

18 e. Organisation of an AWPB Arbitration and Validation Workshop A validation workshop on the ECOWAS AWPB will then be organised, involving all heads of Institutions, Departments and Directorates. The workshop will be attended by at least one staff member representing each level of the programming process and all the socioprofessional staff categories of the Institution. This is designed to ensure that the AWPB takes due account of all the concerns expressed, beginning from the lowest levels of the Institution. The workshop will take place no later than the end of August of the current year (n-1). A copy of the validated AWPB is submitted to the Monitoring and evaluation Unit. f. Dissemination of the AWPB After validation of the AWPB, the recommendations and proposals emanating from the workshop will be incorporated into the final version by the Strategic Planning and Monitoring and evaluation Units. The final version will be reproduced and distributed to all institutions and departments, and also transmitted to all institutions maintaining relations with the ECOWAS Commission. Each Head of Department or Institution is required to draw up a schedule for the distribution of his/her Department s AWPB. This may be done through mini-workshops organised at the level of each Directorate, or according to programme level. However, each Commissioner or Head of Institution may decide the most appropriate means of distribution for his/her Department s AWPB with a view to ensuring its ownership by all the actors responsible for its implementation. Presentation of AWPB Preparation Tools - Logical framework showing the strategic level of the activities of each Department and their links with the organisation s priorities - Multi-year intervention framework; - AWPB presentation framework. Table 1: Presentation of Annual Work Plan of Activities LOGICAL FRAMEWORK Department/Institution N o Strategic Results Indicators Projections over a 3-year Period (Target figures) Structure in Charge 1 N N+1 N

19 Table 2: Presentation of Annual Work Plan of Activities Results Activities Tasks PERIOD Officer in charge Costs g. Preparation of the Quarterly Work Plan The annual work plan prepared and validated by each Department is presented in a quarterly work plan to facilitate implementation and monitoring of the activities with a view to optimising the results to be achieved. This activity may be implemented on a quarterly basis, preferably at a two-day workshop (held at 6-month intervals) to review activities implemented during quarter n-1 and quarterly planning n. Preparation of the work plan for the next quarter must take account of the strengths and weaknesses observed during implementation of the previous plan. Quarterly work plans should be available at least fifteen (15) calendar days before the beginning of the quarter. Copy should be sent to the service in charge of M&E. The quarterly plan preparation framework is contained in Annex 4. h. Preparation of the Monthly Work Plan The monthly work plan issues from the quarterly work plan. It is designed to inform the main officers in charge in each Department of the major tasks to be implemented during the month. The monthly work plan showing projections for the following month (see sample contained in the operational guide) should be available no later than five (5) days before the end of the current month. 19

20 Table 2: Model of Monthly Work Plan (Annex 5) MONTH OF: Implementation Period Activities Operations/ Tasks Physical Indicators Target Figures Start End Implementing Officer Monitoring Activities: Data Collection, Processing and Analysis Procedures and Tools The efficiency of a monitoring and evaluation system hinges on its mechanism for the collection, processing, analysis and dissemination of accurate and transparent information to all parties in order to ensure appropriate decision-making aimed at achieving the set objective. Data collection for the purposes of monitoring and evaluation may adopt two strategic approaches, namely, a passive approach (involving computerised collection or automatic relay of data from bottom to top) and an active approach (involving random visits by members of the M&E Unit to corroborate or verify certain sources). The Global Approach to Data Collection The Passive Approach: This is characterised by the centralisation of information at different points in the chain of transmission between the base (focal points) and the Department. In order to function optimally, this approach must be backed by an information management system capable of centralising all the different data bases. It can also function manually through the transmission of data collection sheets from the point of implementation of an activity to the M&E Unit. Information is transmitted from the base to the top, in other words, from the focal points in Member States through the programme officers to the different levels of the organisational hierarchy responsible for the preparation of reports. The Active Approach: For large organisations the size of ECOWAS the active approach is considered archaic since it is virtually impossible for the M&E Unit itself to collect necessary data through periodic field visits to fill in the data sheets. Nonetheless, the approach does have the advantage of giving the M&E team an opportunity to conduct field visits to verify the accuracy of the data transmitted. In the light of the above, it is clear that data collection for monitoring and evaluation purposes adopts a basically passive approach backed by a computerised network which incorporates the different existing information management systems of the Community institutions, and at 20

21 the same time leaves room for adoption of active collection methods for onsite verification of the accuracy of samples of specific information. Data Capture, Processing and Analysis The process of data capture, processing, analysis, distribution and dissemination is mainly a function to be carried out at the level of the Directorates and Divisions, each one of which will devise its own collection procedure on the basis of the information items specified by the monitoring tools. For the purpose of producing the different reports the ECOWAS monitoring and evaluation system is divided into the following four sub-systems (Annex 6): - The data collection sub-system - The data processing sub-system - The report preparation sub-system - The analysis sub-system. The information collected at the various decentralised points will be processed and consolidated at the M&E Unit which will conduct analyses in respect of strategic outcomes, projections, and progress against the background of the priorities set for the year and also the strategic plan. The analyses will be collated within the performance reports prepared by the M&EU with a view to facilitating the strategic decision-making at the level of the Departments, and subsequently, at the level of the Presidency of the ECOWAS Commission. a. Data Collection Sub-system i. Data Capture Data capture is the first stage of the data collection sub-system. As implementation proceeds, data on the implementation process is culled from the periodic AWP status of implementation reports compiled at the base of the data transmission pyramid, i.e. at the level of the Programme Officers and focal persons of the Member States. The data collected at this level pertains to those activities whose indicators of measurement are clearly defined in the annual work plan. Data on the activities being implemented must be captured by all actors on a daily basis as implementation proceeds. This is an important stage in the data collection process. Any delay in capture at this level could affect the quality of the information generated as well as the reliability of the monitoring and evaluation system. ii. Data Collection Data collection is the stage of information retrieval in a form that will facilitate processing, analysis and decision-making in compliance with strategic guidelines. The data collected comprises indicators showing how implementation of activities is in line with the projections contained in the annual work plan, and compliant with the results-based management chain. Collection is effected by the Programme Officers, using collection tools, scoreboards and progress reports. The collection period varies with the nature of the information to be 21

22 collected and the life cycle for implementation of the activities. The collection method may consist in routine collection, surveys, diagnostic studies or operational research. Field visits: The monitoring and evaluation teams will conduct routine or ad hoc field missions to ensure that the information transmitted by the focal points faithfully reflect developments in the field, to assess the level of execution, and identify specific difficulties encountered in order to make a report to the top hierarchy so that urgent measures and activities could be undertaken (field mission template). b. Data Processing Sub-system (Data Quality Review) Data processing aims to upgrade the quality of the raw data collected to the level of the approved terminology prescribed in the strategic guidelines and programmes with a view to facilitating analysis. Processing also aims to ensure that the information collected meets the basic stipulated conditions, or to select the combinations most likely to yield information on a specific indicator. One of the results expected at this stage is to generate an accurately completed measurement indicator framework. Processing therefore presupposes quality control, which takes place at two levels and requires that both data collection and quality control tools should be completed in full. The first level of control is carried out by the direct supervisor of the agent responsible for transmission of the data. The supervising unit concerned will verify the coherence of the data transmitted to ensure that it is complete and contains no contradictions. The second level of control entails supervision, inspection and meetings with the Divisions and Directorates concerned. When supervising or conducting inspections, the M/E teams will verify samples of the data collected at each level to ascertain their quality. c) Data Analysis and Synthesis Data analysis aims at ensuring and evaluating the level of compliance with indicators, and progress made towards achieving results. It entails the comparison of projected and actual values in order to establish the level of performance achieved. Data generated during the execution of activities is therefore analysed with a view to evaluating effectiveness and efficiency. Analysis is directed more towards the results and impacts of the actions of the Institution in terms of compliance with the guidelines laid down by the Council of Ministers and the Authority of Heads of State and Government. Analysis should also help reveal changes in the indicators. A report of findings is prepared on the process and transmitted to the more senior levels of the management structure of each Institution. The different tools for data synthesis presented below may be deployed by using the database available to the monitoring and evaluation unit. 22

23 Part Three Reporting Procedures and Tools 23

24 Introduction From the methodological point of view, production of reports constitutes the final phase of the monitoring and evaluation process and completes the analysis undertaken. Its objective is to prepare the account to be given to the high authorities. Preparation of the report on schedule will enable the top-most ECOWAS authority to make timely decisions for the revision or continuation of the regional integration strategies. The third part of the manual is devoted to the procedures, tools and time-schedules to be observed for the purpose of producing the following reports: i) monthly reports (internal); ii) quarterly progress reports; iii) mid-year performance reports; and iv) annual performance reports..the target objective is to create a simple framework for the preparation of periodic reports at all levels of the management chain of the Institution, for transmission to the decision-making authorities of the organisation, namely, the Council of Ministers and the Authority of Heads of State and Government. The procedures described below relate to the following reports: Monthly reports (for internal use of the division, directorate or department); Quarterly progress reports; Mid-year performance reports; Annual performance reports. At the operational level, the above-mentioned reporting system is directly coordinated by the Monitoring and evaluation Unit. The reporting system provides the inputs for other important ECOWAS reports, including the interim and annual reports of the President of the Commission. The methodology for preparation of each report is structured in line with the following: Report drafting stages, Deadlines for provision of required information, Responsibilities at each stage of the process. a. Monthly Report The monthly report is a progress report on the activities implemented during the year. In other words, it is a monthly report of the activities implemented. The report must be as comprehensive as possible and must be in compliance with the implementation indicator for the activity. The report covers all scheduled monthly activities for each level of responsibility. Each Programme Officer is responsible for the preparation of monthly reports as progress continues on the activities in his/her charge, as indicated in the Monthly reporting framework. 24

25 Table 2: Sample Monthly Report SAMPLE MONTHLY REPORT IDENTIFICATION OF UNIT OF DIRECTORATE REPORT FOR THE MONTH OF... Physical progress report Financial progress report Code Activities (operations or tasks) Physical indicators Projectio n for period Achieveme nt during period Monthly implement ation rate Cumulative implement ation rate Projection for period Achievemen t during period Monthly implement ation rate Cumulative implementatio n rate Comments

26 A monthly activity report generally includes the following items: Activity code, identical to the code in the work plan; Parameters including the operations and tasks contributing to the implementation of activities identified in the monthly work plan; Indicators, identical to those contained in the monthly work plan; Physical progress report indicating the physical implementation status of the activities. At the beginning of every month, each Programme Officer prepares an indicative work plan. This tool is used to report progress of his/her activities for the particular month and cumulatively. The objective is to ensure that, ultimately, the last report presents the global status of activities, and thereby facilitate the appraisal of the Programme officer executing those activities; Projection for the period: this indicates the projections contained in the monthly work plan. Every effort should be made, wherever possible, to give a numeric value in relation to the indicator; Implementation level for the period: this is the evaluation of the extent to which the monthly projection has been attained; Implementation rate for the period: this is the ratio of implementation to projection for the period; Cumulative implementation rate: this represents overall progress in the activity since commencing implementation of the annual action plan. It is calculated by dividing the sum of implemented values at the end of a specific period by annual projection reflected by the indicator. At any point in the year, this rate reflects real achievement rate of scheduled activities for the purposes of appropriate decision-making; Financial progress report: this is a means of evaluating the effectiveness and efficiency of activities. A correlation must be established between activities executed and the financial resources mobilised and consumed. The presentation of the financial progress report is identical to the physical progress report, except that it is providing information on the financial data; Observations: this is the narrative part of the report which provides specific information on tasks executed during implementation of the activity and any other major factor which should be taken into account during the decision-making process. Observations may reflect challenges encountered or may be recommendations. Preparation of the monthly activity report requires timely submission of necessary information by the various centres of responsibility within a free flowing information system in which all actors play their roles in compliance with stipulated deadlines, formats and resources. The table below provides information on monthly report preparation procedures.

27 Table 3: Monthly Report Preparation Procedures N Stages Deadline Officer in charge 1 Preparation of the monthly report by Programme Officer No later than the last day of the month All Programme Officers 2 Consolidation of monthly reports at the level of the Directorates No later than 2 days after the end of the month Directors 3 Centralisation of reports of Directorates and Divisions at the level of the Departments Upon transmission of the report Directors and heads of division 4 Preparation and transmission of monthly consolidated Departmental report No later than 5 days from the end of the month Commissioners and Heads of Institutions b. Quarterly Progress Report The quarterly progress report presents an initial analytical report which facilitates early and appropriate strategic decision-making. In addition to reporting on the implementation level of the ECOWAS Commission programmes and activities, it reveals the challenges encountered and makes projections for the following quarter. It facilitates identification of bottlenecks which could delay implementation of the activities, and the formulation of appropriate remedial measures. In this regard, it is a useful tool for the decision-making process and the definition of clear strategic guidelines to ensure attainment of objectives set within the limits of available resources. In specific terms, the quarterly progress report carries out the first analysis focused on revealing the major impacts of the Institution s actions, and based on the first qualitative available information on the implementation of activities. The analysis is used to evaluate the scope of its actions. The report adopts the format presented below. It is a tool which may be improved upon in response to the specific needs expressed by the management of the ECOWAS Commission, the other Institutions or even partners. 27