Project evaluation: Summary report Supraregional (BMZ budget item international cooperation with regions): Global Partners of Germany/Alumniportal

Transcription

1 Corporate Unit Evaluation Project evaluation: Summary report Supraregional (BMZ budget item international cooperation with regions): Global Partners of Germany/Alumniportal Deutschland II Project number: CRS Purpose Code: Project objective: Sectors not specified German international cooperation institutions have increasingly sought to harness the skills of alumni who have studied in Germany. Project term: September 2014 to December 2016 Project volume: Commissioning party: Lead executing agency: EUR 7 million German Federal Ministry for Economic Cooperation and Development (BMZ) Not defined since the programme is funded from the International Cooperation with Regions for Sustainable Development (ICR) budget item Implementing organisations (in the partner country): Not defined: ICR budget item Other participating development organisations: Target groups: German Academic Exchange Service (DAAD) Goethe-Institut Alexander von Humboldt Foundation Alumni from Germany: The target group mainly comprises experts and managers from overseas who have become professionally and emotionally attached to Germany or German institutions due to the formative phases they have spent living, training, learning or forging contacts in Germany. These include scholarship holders, participants in training courses, graduates from German universities or organisations, graduates from German courses of study or training, language course participants, former employees of German organisations and returning experts.

2 Page2 Project description The programme relates to the strategies of the Millennium Development Goals (MDGs), especially the goal to develop a global partnership for development. Implementation of the MDGs is contingent on a new form of global cooperation in which developing and donor countries give priority to development. To do so, a system is being set up that makes changes in cooperation verifiable, transparent and sustainable. Emerging actors, new forms of cooperation and a growing consciousness of global challenges are the defining features of modern development cooperation (DC) and international cooperation (IC). As this trend emerges, a new network of contacts is being built among the various actors. Germany alumni are mainly experts and managers from other countries and various sectors, with different ranks and positions, who received initial and continuing training with and through GIZ and its predecessor organisations, and with/through DAAD, Goethe-Institut and the Alexander von Humboldt Foundation. Most of these alumni returned to their countries of origin and have forged close connections with the institution that supported them and with Germany itself. Alumni from Germany are part of German DC/IC s network of contacts. Talks with numerous representatives of German organisations, alumni networks and cooperation partners have shown that German institutions in Germany and the partner countries are not sufficiently aware of the potential offered by alumni who have studied in Germany. So far, German DC/IC has shown too little interest in the innovation potential of these experts, which could be harnessed in many fields of international cooperation, either to resolve specific development problems in the alumni s countries of origin, or to tackle global development issues. German international development institutions have so far shown hardly any demand for the skills of these alumni (core problem). In order to boost demand by German DC/IC institutions for the skills of these alumni and their networks, actors need support in the form of information and contacts both on the supply and demand side, and of platforms for establishing contacts. The performance capacity of alumni from Germany and of alumni networks also needs to be strengthened, as does their ability to act, in their local context and in terms of international networking too. The module objective was formulated as follows: German international cooperation institutions have increasingly sought to harness the skills of alumni who have studied in Germany. The level of ambition of the formulated module objective appears appropriate on the whole. Its appropriateness can also be judged by the way in which the module objectives indicators are quantified and qualified. The indicators agreed with BMZ are: 1. The proportion of alumni who are placed each year in exchange formats that are relevant to development policy increases by 20% per year. (Baseline value: 50, target value: 60) 2. Alumni representatives were actively involved in 50 programmes/measures operated by GIZ, DAAD, the Goethe- Institut and the Alexander von Humboldt Foundation; 60% of the measures are relevant to development policy. (Baseline value: 10, target value: 50) The defined indicators appear suitable for measuring the achievement of the module objective. Their baseline and target values have been defined, though no timeline has been established for the second indicator. However, it can be assumed that this will have been achieved by the end of the programme. Some of the wording is not sufficiently accurate. For example, it is not clear which proportion is meant in module objective indicator 1 (and the term was therefore replaced with number for the evaluation). This notwithstanding, the indicators largely meet the SMART criteria. In view of the considerable effort that will be required for such a global programme, though, the defined values appear rather low. The indicators form the basis for assessing effectiveness. A results model was defined for the programme. This shows how the fields of activity (outputs) logically relate to each other and to the module objective in material and technical terms. The model shows clearly that the hypothesis is located outside the programme (impact) ( alumni and their networks reinforce the results of international cooperation. ). The overarching result presented in the results model forms the basis for assessing impact. Three fields of activity are designed to achieve impact: Field of activity 1 enshrines collaboration with alumni and networks in the participating German institutions. Field of activity 2 supports alumni in organising themselves within networks and carrying out projects independently. Field of activity 3 provides a virtual community environment on Alumniportal Deutschland to support the abovementioned fields of activity. The results model is based on hypotheses on two levels in the three fields of activity: To strengthen demand by German DC/IC institutions for the skills of these alumni and their networks, actors need support as regards information and contacts both on the demand (field of activity 1) and supply side (field of activity 2), and platforms need to be provided to establish contacts (field of activity 3). The results logic was not adjusted in the course of the programme since this did not appear necessary.

3 Page3 The measures related to the Alumniportal Deutschland are carried out jointly by the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ), the Alexander von Humboldt Foundation, the German Academic Exchange Service (DAAD) and the Goethe-Institut (GI). The cooperation partners are linked up by a cooperation agreement on the Alumniportal. Basis for assessment of the OECD-DAC criteria: To determine the TC measure's overall rating, calculate the average of the individual ratings of the five OECD-DAC criteria: points: very successful points: successful points: rather successful 8 9 points: rather unsatisfactory 6 7 points: unsatisfactory 4 5 points: very unsatisfactory Individual and overall rating of the OECD-DAC criteria: Relevance: Effectiveness: Impact: Efficiency: Sustainability: 14 points - very successful 13 points - successful 12 points - successful 13 points - successful 12 points - successful Overall, the TC measure is rated successful with a total of 12.8 out of 16 points. Relevance (Are we doing the right thing?) The programme is highly relevant and meets the criteria of the ICR budget item. It aims to solve the core problem, which was defined as German international development institutions have so far shown hardly any demand for the skills of alumni. The programme systematically addressed and analysed this problem in relation to the target group of alumni (potential, interests, needs). It devoted less attention to the needs, interests and wishes of the German IC institutions. The programme is located within the ICR budget item and therefore explicitly has no partners in partner countries (global approach). It does not relate to specific sectors either (cross-sectoral approach). Although the programme is located within the ICR budget item and therefore explicitly has no sectoral approach nor needs to consider any binding issues and standards, it is nevertheless clearly in line with internationally relevant strategies, especially SDGs 4 (Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all) and 17 (Strengthen the means of implementation and revitalize the global partnership for sustainable development). The programme is also in harmony with BMZ strategies (the BMZ Education Strategy, BMZ s Africa Policy, the BMZ Education for Sustainable Development strategy in connection with the 2030 Agenda and the BMZ Charter for the Future/Global Partnerships). The relevance of the TC measure is rated very successful with 14 points.

4 Page4 Effectiveness (Will we achieve the programme s objective?) The programme achieved all of the objectives agreed in the commission as defined by the module objectives indicators. It can therefore be assessed as effective. Objectives indicator Target value according to the offer Current status according to the project evaluation The number of alumni who are consulted each year in exchange formats that are relevant to development policy increases by 20% per year. Alumni representatives were actively involved in 50 programmes/measures implemented by GIZ, DAAD, Goethe- Institut and the Alexander von Humboldt Foundation ; 60% of the measures are relevant to development policy. 20% increase Fully achieved Through the activities of the alumni liaison offices, 21 alumni were deployed as advisors, members of panel discussions or support persons ; in 12 webinar series on the Alumniportal and the careers fair in Viet Nam, some 20 alumni were deployed as experts; 191 alumni assumed an active role (several times) and 145 assumed an active role (once) at events (= 13% of all respondents). If partners own alumni measures are also taken into account, the figure is several times higher (DAAD alone carried out 219 events involving alumni in 2015). 50 programmes/measures Fully achieved The programme held 68 events with alumni participation (in 2015) that were all relevant to development policy. The evaluation team comes to the conclusion that objectives indicators 1 and 2 will probably be fully achieved by the end of the project. The project is rated successful in terms of effectiveness with 13 points. Overarching development results (impact) (Are we contributing to the achievement of overarching development results?) The programme can be said to have contributed to a number of overarching (development) policy changes. These include: The creation of a positive image of Germany (image-boosting). The creation of a public exchange and discussion forum (participation). Support for other IC/DC programmes and continuation of the results generated by other programmes (from which the alumni graduated) Promotion of foreign trade by making experts available to German companies Support for the lifelong learning approach and for South-South and South-North cooperation Soft diplomacy Support for global partnerships (e.g. Sustainable Development Goal (SDG) 17) Promotion of synergies in international alumni work by cooperating with various partners It was already clear when the programme was planned that very few changes could be expected at the level of frame conditions. This also appears to apply to the future. If programme financing is considered as a frame condition, it must be said that the overarching results cannot be maintained in the long term if the programme receives no further support (since programmes of this kind cannot as a general rule achieve financial sustainability).

5 Page5 The programme is based on hypotheses related to long-term results. These hypotheses appear to be correct both retrospectively and looking forward. However, the programme should analyse the needs of the demand side more closely in future. The programme unites four partners (GIZ, DAAD, the Alexander von Humboldt Foundation and Goethe-Institut), supports their institutional alumni work and places it on a common basis. To this extent, the programme is fully complementary to these outputs and to those of the individual partners. The programme has already served as an example for other, similar projects and has advised these (e.g. Alumni Portal France/Campus France). Beyond this, since the portal has 132,000 users in 190 countries, it can be expected to have broad impact. The impact of the TC measure is rated successful with 12 points. Efficiency (Are the objectives being achieved cost-effectively?) Assessing the programme s efficiency presents somewhat of a challenge given that there are virtually no comparable programmes and the monetary value of its impact is more or less impossible to determine. It would not be sufficient to assess efficiency merely by determining the funds spent per alumnus/alumnae. The programme achieved its module objectives indicators and thus the planned results with the intended level of funding. It must be taken into account that the programme continues to operate the Alumniportal Deutschland as a web platform, which is continuously developed and expanded, and for which editorial support is provided. None of this is reflected in the module objective or the corresponding indicators. At the same time, operation of the web platform generates very high costs, also due to rapid technological development, and is a prerequisite for the programme and for cooperation between the German partners. Because the organisations involved run the Alumniportal together, the necessary instruments and digital offerings related to alumni work only have to be developed once, not four times over as was previously the case. Other partners such as the German Federal Foreign Office benefit from this because they do not need to develop their own solutions and can make use of the centrally developed instruments for digital alumni work. If the Alumniportal were discontinued, the cooperation partners would have to set up their own new platforms. This would mean initial investments in the six-digit range for each partner, plus the relevant follow-on costs. Apart from that, many alumni are also alumni of other German institutions, and would therefore have to be members of different platforms or choose between them. The cooperation partners are very committed and contribute more than could be asked from them based on their contractual obligations. That increases the available resources at no additional cost, and provides certain leverage for the programme s objectives thanks to external resources. Seen as a whole, today the programme reaches significantly more alumni worldwide with a growing number of formats, contents and events and a lower annual budget (average of EUR 3.8 million/year from ; average of EUR 3.0 million/year from ) (Alumniportal members: December 2014: 106,941; May 2016: 132,314 (+24%)). GIZ continuously coordinates activities with the programme partners (DAAD, Goethe-Institut, and the Alexander von Humboldt Foundation) at both strategic and operational level. All partners contributed more to the programme than was financed by it, and used synergies wherever possible. The programme collaborated extensively with projects (especially those of GIZ) in the partner countries, and made use of existing personnel, e.g. as liaison officers. Coordination of the programme partners constitutes one of the key outputs and results of the programme itself. Stakeholders confirm that the steering structure is appropriate and worked well. It can therefore be presumed that the programme is efficient, even if this is not reflected in the insufficiently quantified module objectives indicators in relation to the funds used. The efficiency of the TC measure is rated successful with 13 points.

6 Page6 Sustainability (Are the positive results durable?) The programme does not have a partner structure as defined under the Federal Republic of Germany s bilateral cooperation, particularly since it is financed from the ICR budget item. Sustainable changes cannot therefore be achieved among partners, nor can they be perpetuated. It can, however, be presumed that many of the instruments introduced and tested (alumni fairs, careers fairs, webinars, etc.) will be sustainably used by the programme partners on the German side (who are to be seen as partners in this case) and can therefore have a sustainable impact. It can also be assumed, though, that the majority of the positive effects (see impact) will not continue in the long term without the programme and above all without the Alumniportal (which requires ongoing maintenance and support), because new alumni are constantly joining the ranks. The programme nevertheless attempted to develop exit strategies and examined several financing models, none of which were promising. The sustainability of the TC measure is rated successful with 12 points. Published by Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH Registered offices Bonn and Eschborn Friedrich-Ebert-Allee Bonn, Germany T F Dag-Hammarskjöld-Weg Eschborn, Germany T F E I info@giz.de