STRATEGIC AND OPERATIONAL MARKET RESEARCH DURING

Transcription

1 EVALUATION January 2004 ITC s PROGRAMME ON STRATEGIC AND OPERATIONAL MARKET RESEARCH DURING EVALUATION REPORT Mission Member: Mr. Sandy Cuthbertson

2 The designations and the presentation of material in this report do not imply the expression of any opinion whatsoever on the part of the International Trade Centre UNCTAD/WTO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. This document has not been formally edited by the International Trade Centre UNCTAD/WTO (ITC) The views expressed are those of the author ITC/EV/1/04

3 i Contents Glossary ii 1 Introduction 1 2 Assess continuing relevance of the programme s objectives 4 Evolution of MAS 4 Compatibility with ITC objectives and goals 10 3 Assess quality of outputs 15 Indicators of quality 15 4 Assess effectiveness, outcomes, impact and sustainability 28 Effectiveness 29 Sustainability 31 5 Conclusions and recommendations 41 A Terms of reference 46 B People interviewed for the review 53 References 55 Boxes, charts and tables 1.1 MAS staff numbers Terms of reference at a glance Subscription costs to MAS tools Evolution of MAS at a glance A tax on imports is a tax on exports Reviews of selected client studies Effectiveness, outcomes and impact International trade databases 35 B.1 People interviewed for the review 53

4 ii Glossary ACP APEC AVE CEPII CTS DPMD EU GTAP GTIS IDB IMF ITC JITAP MAS MATRADE MNS MOU OCIPED RAU African, Caribbean and Pacific Group of States Asia Pacific Economic Cooperation Ad valorem equivalents Centre d études prospectives et d information internationales Consolidated Tariff Schedule Division of Product Market Development (ITC) European Union Global Trade Analysis Project ( Purdue University) Global Trade Information Services Integrated Database International Monetary Fund International Trade Centre Joint Integrated Technical Assistance Programme to selected Least Developed and other African Countries ( ITC, UNCTAD, WTO) Market Analysis Section (ITC) Malaysia External Trade Development Corporation Market News Service (ITC) Memorandum of Understanding Omani Centre for Investment Promotion and Export Development Research Analysis Unit

5 iii TIPS TRAINS TOR TPO TSI UNCTAD UNSD USAID WB WEF WITS WTO Trade and Investment Policy Strategy Centre (RSA) Tariff Analysis and Information System (UNCTAD) Terms of Reference Trade promotion organisations Trade support institutions United Nations Conference on Trade and Development United Nations Statistical Division United States Agency for International Development World Bank World Economic Forum World Integrated Trade Solutions World Trade Organization

6

7 1 1 Introduction This review of the International Trade Centre s (ITC) Strategic and Operational Research Programme conducted by the Market Analysis Section (MAS) has been initiated as part of ITC s regular programme review cycle. However, the review is timely as it follows a period of rapid growth and change by the section both within ITC operations and in the external environment for the delivery of trade related technical assistance. Some of the most relevant aspects of recent MAS development and growth are set out below. MAS has grown from an initial team of 25 people in 1999, when it was established, to 38 at the end of 2003 in an overall ITC establishment of about 240 people (see table 1.1). Some two thirds of MAS staff are contracted employees where continuation of contract depends on revenues generated from donor project support and purchase by government agencies of capacity building and trade information services. Of an estimated total budget for 2003 for the section of around US$2.4 million, around US$1.4 million will be obtained from these sources. The core products or services provided by MAS include database analytic support tools, capacity building services, client studies, market news services, inquiry/reply service and market briefs. These products/services have been integrated into web-based systems amenable to being found and used by a wide range of people. While MAS clients and partners include traditional ITC clients in trade support institutions (TSIs) an expanding base of clients and partners is being reached including: Other international agencies including the World Bank (WB) and the International Monetary Fund (IMF); Academic policy analysts; The World Economic Forum; Trade policy analysts in government; and

8 2 1 INTRODUCTION Trade negotiators. ITC s parent bodies: the World Trade Organization (WTO) and the United Nations Conference on Trade and Development (UNCTAD), are closely involved in the production and use of core MAS outputs. ITC has established a revolving fund to manage the resources generated externally by MAS. 1.1 MAS staff numbers Year L staff G staff Consultants a P staff Total a The reduction between 2002 and 2003 reflects replacement of short-term consultants by longer-term appointments so that staff inputs actually increased, even though numbers fell. Source: MAS records 2003 Relevant examples of changes occurring in the external environment for delivery of trade related technical assistance are set out below. Rapid changes in information technology and increasing access to that technology in developing and transition economies have made web based delivery of trade related technical assistance much more feasible. Increasing awareness and emphasis on transparency of trade policy and trade flows has increased the demand for systems that provide transparency. Transparent market access conditions is a necessary prerequisite for efficient and fair international trade for at least three reasons: efficient trade regimes require that participants know the rules; fair trade requires that buyers and sellers in developing countries not be disadvantaged by their limited access to knowledge about conditions of trade; and reform in international trade requires that people be aware of the obstacles to trade currently existing and the costs of those obstacles. The opportunities arising from these developments are well recognised and there is increasing competition in the supply of database services of the kind developed by MAS from both private providers and from government and international agencies.

9 3 1 INTRODUCTION There is increasing cooperation across international agencies encouraged by demands from donors and clients for such cooperation. The terms of reference (TOR) requires the review to assess: the continuing relevance of the programme s (strategic and operational market research) objectives; the quality of the services; effectiveness, outcomes, impacts and sustainability; and to formulate recommendations. The TOR specifies seven precise tasks that effectively map into these three core assessment requirements. See chart 1.2 and Annex A. 1.2 Terms of reference at a glance Assess continuing relevance: analyse evolution assess changing needs examine impacts on beneficiaries Methodology: field visits to Oman, Malaysia and South Africa one month at ITC headquarters telephone survey of users Assess quality of outputs: assess ITC comparative advantage assess technical knowledge and resources needed Assess effectiveness, outcomes, impact and sustainability assess expenditure required by ITC examine scope relevance and impact of R&D Formulate recommendations Annex B lists people interviewed in the course of the review.

10 4 2 Assess continuing relevance of the programme s objectives The TOR specifies that the review: Assess the continuing relevance of the programme s objectives and related strategies and activities as stated in the last ITC Business Plan, with regards to the needs of developing countries and economies in transition and taking into account ITC s corporate objectives, priorities and orientations The TOR also notes that MAS is responsible for carrying out the bulk of ITC activities under the Strategic and Operational Market Research Programme. In this context it is useful to appreciate the way the current portfolio of MAS services has evolved to meet technical assistance needs and ITC objectives. Evolution of MAS MAS was established in 1999 following amalgamation of the Trade Inquiry Development Unit (TIDU), the Market News Service (MNS) and the Research Analysis Unit (RAU) in the Division of Product and Market Development (DPMD). The RAU had previously built up a portfolio of single client studies that included substantial coverage of China issues. Its work built on ITC s traditional preferred access to Comtrade raw data initially involving microfiche and subsequently through CD-ROM distributed as PC-TAS. (PC-TAS contains five years time series of import and export statistics covering 200 countries and territories and broken down in 3,800 products according to the Standard International Trade Classification [SITC revision 3]. The Harmonised System [HS] presents 5,300 products viewable at the 2 and 6 digit level.) Country Map Following the establishment of MAS, a web based country approach called Country Map was developed and launched in Country Map is built around techniques applied to Comtrade data to derive indicators of

11 2 ASSESS CONTINUING RELEVANCE OF THE PROGRAMME S OBJECTIVES 5 competitiveness, revealed comparative advantage, product champions and export performance. Some 184 countries are now covered. Country Map s development reflected: RAU past experience delivering client studies based on access to Comtrade raw data; the history of producing trade statistics and trade related indicators, at first on microfiche then subsequently as PC-TAS in the Trade Information Section of DPMD; and demands from within ITC for a country focussed web based analytical information source; Access to Country Map is free of charge. TradeMap At about the same time building on RAU experience MAS developed TradeMap, which was designed, to provide users with a web based analytical tool to help them use Comtrade data. Subsequently information on market access was added to TradeMap, which is now an on-line database on global trade flows and market access barriers. It covers over 5,000 products and 200 countries. TradeMap became available in It is sold on a country basis. The price varies depending on level of use and stage of development of the country. For partners in a developing country with unrestricted use (national franchise) the price is US$36,000 license fee plus service fees of US$5,000 per year. The number of partners has grown from 1 in 2000 to 37 in 2003 with another 20 single user franchises. These figures do not include the substantial number of countries where access through TradeMap USAID applies to September Table 2.1 summarises subscription costs to MAS tools.

12 6 2 ASSESS CONTINUING RELEVANCE OF THE PROGRAMME S OBJECTIVES 2.1 Subscription costs to MAS tools For internal use only one user location a TradeMap Product Map TradeMap and all Product Maps Full access in one year All 72 P maps per year One P map per year Per year LDCs 1, ,000 Developing 3,000 1, ,000 Developed 6,000 3, ,000 For dissemination to users in own country All 72 maps b First year Annual renewals First year Annual renewals First year Annually from year 2 LDCs 18,000 1,000 from year 4 3,000 1,500 20,000 1,500 from year 2 Developing 36,000 5,000 from year 2 6,000 3,000 40,000 6,000 from year 2 Developed 20,000 15,000 from year 2 12,000 6,000 25,000 20,000 from year 2 One P Map first year and annually thereafter 500 1,000 2,000 a For internal us in multiple locations please contact trademap@intracen.org b Limit to 50 sub-users. For additional sub-users: $50 per company per year. Unlimited number of storefronts for the subscribing TSI Source: MAS records 2003 Market Access Map Work commenced on Market Access Map in June 2001 and an initial version was launched in Washington towards the end of that year. Market Access Map reflects constructive cooperation between UNCTAD and ITC in that it both builds on and adds to UNCTAD s Tariff Analysis and Information System (TRAINS) database. When MAS started working with the TRAINS group at UNCTAD, 125 countries were covered and about half the data related to before At present, 165 countries are covered and all data is post The key features of Market Access Map are as follows: market access data including tariff, specific tariffs, antidumping duties, tariff quotas plus ad valorem equivalent calculated on a bilateral basis for over 165 countries with at least 2000 year currency; inclusion of over 300 preferential agreements; and web based access by early TRAINS is a unilateral depiction of market access conditions, which presumes a most favoured nation (mfn) environment. With the drift to preferential trading arrangements and consequent variations in access conditions across countries, the bilateral representation in Market Access

13 2 ASSESS CONTINUING RELEVANCE OF THE PROGRAMME S OBJECTIVES 7 Map is needed to capture these now widespread departures from mfn arrangements. Market Access Map is currently sold as a country specific version on CD-ROM at US$5,000 with total sales of around 70. Product Map The fourth database developed in the early years of MAS is Product Map. Product Map integrates a range of traditional ITC products and makes them accessible on a website. These products include components of the amalgamated MAS namely the trade Inquiry/Reply service, the MNS and Market Briefs. Product Map also contains space for networking tools in the form of storefronts and business contacts. Product Map is sold by subscription. A single sector or product cluster costs a developed country subscriber US$400, a developing country subscriber US$200 and subscribers in least developed countries pay US$100. Product Map covering all 72 product clusters is sold for between US$1,500 and US$6,000 for an annual subscription according to the level of development of the subscribing country. Buyers of Trade Map had access to Product Map until the end of In 2003 revenue from sales of Product Map was US$6,500. Total revenue counting subscriptions to Market Briefs, the inquiry services and MNS was US$17,810. Inquiry/Reply service The Inquiry/Reply service is a traditional ITC service that originally involved several full time staff responding by telex or letter to trade inquiries from exporters in developing countries. The service with its one on one response to individual exporters was expensive and leverage was low. Product Map with its detailed information on 72 product clusters and access to contemporary reports including MNS and Market Briefs, trade data, trade fairs and contact points (store fronts) is intended to be a resource for guiding inquiries allowing a quicker and lower cost service out of ITC. This combination of relevant data and software systems to help people find this data means that the inquiry/reply service can be more efficient about empowering users to find their own answers. With this resource one staff member assisted by interns and irregular staff support can now handle the same number of about 2400 inquiries per year. Market News Service (MNS) The MNS at the time of amalgamation consisted of monthly price and market news covering cut flowers, leather, rice, wood, fruit and vegetables

14 8 2 ASSESS CONTINUING RELEVANCE OF THE PROGRAMME S OBJECTIVES fruit juices. A pharmaceuticals starting materials and essential drugs MNS has been established and produced from within MAS while those covering rice, wood and leather have been stopped or handed over to private providers. The production of fruit juices, medicinal plants and extracts, and a MNS for spices has been outsourced. The current MNS covers cut flowers, pharmaceuticals, fruit and vegetables, fruit juices, pharmaceutical starting material, spices and medicinal plants and extracts. Efficiencies in the preparation and dissemination of the MNS have been achieved by: curtailing some services which were well covered by other publications (for example, rice); sale or contracting out production of some MNS publications; and dissemination through the web based Product Map. Market Briefs Market Briefs is also a traditional ITC service involving preparation of concise market reports including trade contacts, market characteristics and access information for selected products and or markets. Market Briefs are produced by staff plus irregular support and in association with the Bordeaux Business Program. Market Briefs though sold by subscription for a short time are now distributed to users through Product Map. In 2002, MAS published 7 Market Briefs covering bananas, candles, dehydrated vegetables, avocado, frozen hake and shrimp. In 2003, three Market Briefs were published covering nutmeg, cassava and dehydrated beans. Single client studies Commissioned consultancies for individual clients were initially delivered as a part of RAU activities and now form a significant component of MAS activities. Clients include TSIs in developing countries, development agencies and international organisations. Single client studies are frequently but not always based on a particular application of a MAS tool. About 6 client studies are produced per year involving around three or four staff. Capacity building An important contemporary component of MAS services in more recent years, has been capacity building activities designed around MAS core products. Initially capacity building involved mainly launches of the products but in more recent years since 2002 capacity building has

15 2 ASSESS CONTINUING RELEVANCE OF THE PROGRAMME S OBJECTIVES 9 involved taking international trading principles and approaches to regional centres and other broadly based participants. Box 2.2 contains a timeline setting out key events in the evolution of MAS. 2.2 Evolution of MAS at a glance 1990 and before ITC preferred access to Comtrade raw data and the provision of market analysis and information on microfiche and before Traditional ITC services involving Market News Services, Market Briefs and inquiry services conducted out of various sections in DPMD Research Analysis Unit (RAU) established and initially produced client studies, data analysis and china studies RAU moved to DPMD MAS created by combining RAU, Market News Service, Market Brief and trade inquiry service with at total staff of 25 people, 4 of them budget financed Trade Map launched for sale at prices ranging from US$18,000 to US$36,000 initial subscription plus annual service fees Country Map released as a free service on ITC website Soft launch of Product Map Market Access Map launched MAS staff numbers reach 30 in total with about 2/3 financed from outside revenues 2004 Market Access Map to become web based. Context and next steps The evolution of MAS needs to be appreciated in the context of several external developments. First is the emergence of a capacity to make large quantities of information available to users through web-based systems and for an increasing number of users in developing countries to have access to these systems. Second, other agencies have cooperated to have ITC take the lead in providing user interfaces to their basic data. Third, ITC management has been flexible and supportive in accommodating MAS expansion. Fourth, donors have been generally supportive in funding the development and use of MAS products. The four databases contain a number of overlapping components and have a similar community of users. TradeMap, Country Map and Product Map all rest on Comtrade raw data. TradeMap and Market Access Map have overlapping treatments of market access data though TradeMap is less detailed and presents raw data. All are, or will be web based. Users also

16 10 2 ASSESS CONTINUING RELEVANCE OF THE PROGRAMME S OBJECTIVES overlap so that Product Map is often included in a TradeMap partnership agreement. The existence of the four different products at this point in the evolution of MAS is to a large extent an outcome of the allocation of work responsibilities and funding considerations which largely dictated they be built separately. A logical next step being tackled by MAS is to pull the four products into one so that users can enter at one point and then find their way to the information and analytical tools they need. While these plans are sensible they raise implications for the funding of development costs and pricing of the joint product upon completion. Compatibility with ITC objectives and goals The ITC has set out 5 goals in the pursuit of its technical cooperation mission (ITC 2002a). These are to: facilitate the integration of developing and transition economy enterprises into the world trading system; support national efforts to design and implement trade development strategies; strengthen key trade support services, both public and private; improve export performance in sectors of critical importance and opportunity; and foster international competitiveness within the business community as a whole and the small and medium size enterprise (SME) sector in particular. Broader user base and ITC goals Identifying agents to work through to achieve these goals is a continuing dilemma for ITC. Working through individual firms is expensive and provides little leverage. ITC s traditional multiplier organisations, the state sponsored export development or trade promotion organisations (TPOs) came under scrutiny through the 1980s and 1990s when it was found that many TPOs were not very effective. In recent years ITC has broadened its partnership base to cover trade support institutions (TSIs) generally. Thus it has variously enlisted chambers of commerce, think tanks, government departments, academic institutions and other grouping with a capacity for trade support. Web based systems that make data and analysis accessible to

17 2 ASSESS CONTINUING RELEVANCE OF THE PROGRAMME S OBJECTIVES 11 individual firms and agencies have the potential to make ITC and exporters less dependent on multiplier organisations. Three of the products described in the previous section Country Map, TradeMap and Market Access Map are designed to support priority setting by trade strategists and managers of TSIs. Product Map including market briefs, inquiry/reply service and MNS is an operational market research component designed to provide business intelligence support to business managers. According to the ITC Business Plan for these activities or products, and the Strategic and Operational Market Research Programme which supports them, relate to goals 2, 3, 4 and 5. But given that the unifying theme of these three products is transparency of trade and market access conditions, it could be argued that they relate mainly to goal 1 facilitating entry into the world trading system. The users of the first three mentioned products particularly Trade Map and Market Access Map represent a further broadening of the term, TSIs as many of the users are academics, policy think tanks, international agencies and mainstream economic policy departments. In a few limited cases possession of TradeMap appears to have enabled a traditional TSI to enter the public policy arena. Though these new organisations or users do not have the traditional TSI direct trade focus, their mainstream economic responsibilities cover key factors in enabling firms in developing and transition economies to enter the world trading system. Such policies include: reduction of domestic trade barriers; improved regulatory arrangements for all firms including firms which might engage in export; improved infrastructure for all firms; and improved communication. Impressions from users The following impressions of the kinds of people and agencies using MAS products is derived from field visits to Oman, Malaysia and South Africa from communications with users in agencies in person or by telephone and from data on website usage. Country field visits The application of Oman Trade Map through the Oman TSI OCIPED represents a stylised application of the product. There with the support of a

18 12 2 ASSESS CONTINUING RELEVANCE OF THE PROGRAMME S OBJECTIVES strong TSI, exporting firms worked through TradeMap to identify markets and competitors. These analyses were then followed up by more in depth research using information from other sources. In Malaysia the TradeMap partner MATRADE manages the use of TradeMap by requiring clients to utilise TradeMap through the MATRADE library. The general impression in that main users are firms exploring export opportunities. Use appears to be limited and there is little activity by academics or policy analysts. In South Africa TradeMap is managed by the Trade and Investment Policy Strategy (TIPS) centre. Passwords are freely issued and academics, government policy agencies and industry organisations are major users. Telephone discussions and meetings The IMF staff member contact reported that Market Access Map particularly had been helpful in preparing a general IMF/World Bank survey of trade barriers and in supporting trade policy, aspects of IMF mission work. At the World Bank, staff involved in similar work considered that MAS was meeting important policy related needs. An Australian consultant working on the World Trade Net project saw a major role for Market Access Map as a tool for trade negotiations. He had been arguing this case strongly but was unclear about how much take-up there was. The reviewer spoke by telephone with representatives of partner agencies in China and the Philippines. In the Philippines there seems to be a repeat of the Malaysian experience where the partner agency is keeping passwords restricted to use within the agency and the under-secretary s office. Whereas in China the agency appears to be pushing TradeMap out to the business sector with training courses and a high profile launch. Data on web usage Visits to Country Map have increased from 2,500 per month to around 8,000 per month in In October 2003 there were around 12,000 visits and 75,000 downloaded documents. The most visited applications are the National Export Performance and the Trade Performance Index. The indicators on Reliability of Trade Statistics is also frequently visited. The most visited countries are China, the US, Brazil, Peru and Argentina. Most visitors appear to come from the USA and the EU. But there are also regular visitors (at least 2 per day) from Egypt, India, Thailand, Australia and Canada.

19 2 ASSESS CONTINUING RELEVANCE OF THE PROGRAMME S OBJECTIVES 13 The visitor data does not lend itself to easily identifying the kind of people and agencies visiting Country Map. People from universities do seem to be frequent users including the University of Texas at El Paso, Northern Arizona University, the University of Missouri, Stanford University, the American University in Cairo, University of London Computer Centre and the City University of New York. TradeMap usage statistics indicate average daily visits of around 130. About one quarter of these visits are typically from the US but these figures should be interpreted with caution as some visitors from the US might actually originate elsewhere. While TradeMap usage figures are not very helpful about where visitors come from or how long they stay. USAID as a large block purchaser of TradeMap requires MAS to report on users of USAID TradeMap in more detail. For example in October 2003 the database showed 68 registered single users of USAID TradeMap. Of these about 15 users appeared to be Government Ministries of Economy, Finance or Industry. About 10 users came from different universities and another 10 users came from consulting firms. USAID officers were also well represented, as were Chambers and a few individuals. The total number of visits was 1,012 with nearly half visits of less than one minute. Forty per cent of visits were for more than 5 minutes. Product Map usage also appears to be increasing rapidly from around 80 visits per day in October 2002 to over 200 visits per day in October The most used parts of Product Map appear to be Market Studies (16 per cent) followed by MNS (11 per cent), business contact (10 per cent) and storefronts (40 per cent). The usage statistics do not provide detailed breakdown of where users of Product Map come from though the US, Switzerland, Guatemala and Brazil appear to be among the top country users in those cases where it is possible to resolve country of origin. Compatibility with goals To sum up, three core MAS products, Country Map, TradeMap and Market Access Map are compatible with ITC goal 1 facilitating integration of enterprises in developing and transition economies into the world trading system. The products each contribute to transparency. Transparency with regard to trade policy has been in short supply especially for developing and transition economies. While some of the users of these MAS products are not traditional ITC clients, they hold the key to policy and regulatory matters which are

20 14 2 ASSESS CONTINUING RELEVANCE OF THE PROGRAMME S OBJECTIVES central to entry into world trade, including infrastructure, regulatory reform, communications and stability of economic policy. Their use in analysis of international trade, including analysis of the costs of trade barriers and by trade negotiations offers hope for the reduction of barriers to trade. In some cases the trade performance index has been used to rebut arguments that globalisation has passed developing countries by. For example in the face of claims by African leaders that globalisation has backfired on the continent, ITC studies were used to demonstrate that growing number of African exporters are becoming competitive global suppliers (World Trade Agenda 2000). The provision of transparency and achievement of goal 1 are so closely linked that provision of transparency could be an appropriate goal for ITC. This reasoning differs from ITC s Business Plan for which envisages these three products as relating more closely to other goals: supporting national efforts to design and implement trade development strategies, strengthening key trade support services, improving export performance and fostering international competitiveness. While the activities are not inconsistent with these goals their special contribution appears to be more firmly in the first ITC corporate goal. Product Map appears to relate more closely to these latter goals. However while it can strengthen key trade support services it may weaken some agencies. This could happen as individual firms and their advisors get direct access to information through passwords to web sites. Concern about being by-passed might explain why some TPOs who are TradeMap partners with Product Map are reluctant to hand out passwords. Product Map, which has the potential to deliver traditional products such as MNS and Market Briefs directly to the door, could reduce reliance on traditional TSIs. Activities of MAS and some of its new partners that analyse the rationale for raising barriers to trade are also consistent with corporate goal 1. For example, in 2000, MAS staff cooperated with CEPII to assess environment related trade barriers. They found that there is a risk of a new set of protectionist tools justified on environmental grounds, which are becoming increasingly popular within industrialised countries. Such analyses though not directly related to enterprises or TSIs have the potential to substantially facilitate the transition of developing and transition countries into the world trading system (Fontagné et al 2001).

21 15 3 Assess quality of outputs The TOR requires that the review assess the quality of the outputs produced through various services of the section. The bases available for such an assessment include the following: usage or demand including existing levels and growth; opinions from users; the quality and reputation of partners and users; and opinions from technical specialists. The section examines these indicators of quality along with ITC s comparative advantage and the technical knowledge and resources needed to develop, service and use the emerging set of MAS products. Indicators of quality Demand growth One relevant guide to quality is the fact that people appear to be willing to pay for MAS products. This is particularly the case for TradeMap, which has generated revenues from the following sources: sales to partner agencies in some 36 countries involving the purchase by the agency or partner directly or through donor support; contracts with USAID involving the establishment of USAID TradeMap and its provision to users in USAID client countries; and purchase of market access profiles and access to TradeMap through World Trade Net Program. Country Map is provided free of charge so there are no market based indicators. But the steady increase in website use and the large number of visits involved indicates that a wide range of people are increasingly finding the service to be of relevant quality. The Trade Performance Index and the National Export Performance Index in Country Map are also used

22 16 3 ASSESS QUALITY OF OUTPUTS by the World Economic Forum in its Global Report series (World Economic Forum 2002). Product Map and the various products now integrated into it generate some revenue through subscriptions. Paid subscriptions are relatively small (typically between 30 and 50) and the total revenues utilised by the Product Map family is less than US$20,000 per annum. This relatively low revenue partly reflects the fact that Product Map is not difficult to access free of charge, as a trial user or when it has been included with a TradeMap partnership agreement. Product Map s work in progress status might also explain these relatively low direct revenues. But whatever the reason it is not easy to demonstrate quality by referring to Product Map sales or usage. Market Access Map is not yet available on-line but sales of 70 units at US$5,000 each is a strong indicator of quality in the market place. Market Briefs do not appear to be strongly supported by direct purchases. Client studies are reasonably well supported although growth is not strong. Capacity building on the other hand appears to be strongly supported by donors and is growing rapidly. The capacity to generate additional revenue is a limited indicator of quality. ITC clients are not commercial clients and do not make purchases on commercial considerations. As a vehicle for disseminating long standing ITC products that have traditionally been available free of charge to users, it is unrealistic to expect much in the way of fee generation on the grounds that traditional products are now web based. Opinions from users and cooperating agencies The reviewer spoke by telephone with main contacts in the World Bank and the IMF and met with colleagues in WTO and UNCTAD. All of these contacts were very positive about the quality of the achievements of MAS over recent years. From the IMF there was appreciation of the usefulness of the Market Access Map and TradeMap in supporting IMF teams in country missions. IMF people considered that the MAS staff was extremely productive though concerned that they may be fully stretched. At the World Bank the people contacted were also very supportive. They considered the work calculating bilateral ad valorem equivalents (AVEs) of specific duties to be a major and very important exercise. This work generally shows that specific duties lead to high AVE protection for imports from developing

23 17 3 ASSESS QUALITY OF OUTPUTS countries because the unit value of their exports tends to be lower. The AVEs calculated in this way address this problem but do not solve it completely as they do not reflect the trade that does not happen where AVEs are so high as to stop it. In Geneva officials from UNCTAD and WTO were extremely supportive. Indeed one of the striking aspects of the review is the goodwill, trust and cooperation of professionals across agencies. Matters of turf and ego do not seem to apply with all parties working cooperatively towards a common goal. This is a very special and powerful outcome. Some observations from people contacted follow. TradeMap is a convenient and effective system for working out where you might do a deal but you would never close a commercial contract on TradeMap information, the data is not current or detailed enough. Manager of an aluminium products firm in Oman. TradeMap is a good product and I used it when we had it on a trial basis but my institute can not afford it. Senior Researcher at Kiel University. Market Access Map is a remarkable achievement. It is inevitable that expert users will find mistakes (ways of improving it). Production and service capacity seems strained and there is a continuing tension between doing a lot with a little and making some mistakes along the way and doing less with fewer mistakes. Senior Staff at IMF. TradeMap is a wonderful teaching device, by exposing people to the rich diversity of trade in all manner of goods between all kinds of countries they come to appreciate the gains from trade. Professor at a South Africa University. Compared with World Trade Atlas, TradeMap has more countries and is cheaper but where you want detail about the big countries World Trade Atlas is better. MATRADE official. Back from Tbilisi, I wanted to thank you and your colleagues most sincerely for having given me access to the data files you maintain. It was extremely useful during the training sessions. I made a lot of advertisement for ITC s excellent work, but really the website did the job better than I could everyone found it very useful indeed. I strongly encouraged them to join, and I am hopeful that they will do. Letter from Professor to head of MAS. It is very helpful having these tools for us, since China as the leading developing country being a new member of WTO, transforming from planned economy to market economy, first from government organisation facing new challenges in providing up to date information for trade promotion purposes, and second, from business prospective, companies enjoy more freedom to trade with companies in foreign countries, but always lack of information. And we considered that the MAS tools really can help us to solve this problem. And it is something that only can be done by UN organisations since

24 18 3 ASSESS QUALITY OF OUTPUTS lots of information collected and updated is not easy thing to do on national basis. The only thing so far we are not happy with is the information update from the TradeMap, since for instance, now it is the year 2003 but the data in TradeMap is from So from our feedback, people really want to have updated data and count on it very much. And we are really looking forward to having more information in Product Map. There are people in China who have never done business with the outside world and they depend on these tools very much as starting point. Contact point in a Chinese partner agency. Quality or reputation of partners and users As already noted many activities of MAS are supported and used by an expanding range of high profile partners. UNCTAD and WTO have agreed that ITC will be responsible for coordinating the transparent provision of information on tariffs and other barriers to trade. The Global Competitiveness report produced annually for the World Economic Forum draws on MAS generated measures of trade performance. Under a grant of US$300,000 the World Bank Information Division has commissioned ITC/MAS to make available to selected countries TradeMap and Product Map. The IMF is a regular user of Market Access Map to support its general research programme and in applied country work. The French Economic Research Institute (CEPII) is a partner with MAS in development and applications of Market Access Map (along with Country Map and single client studies). The Global Trade Analysis Project at Purdue University hosts one of the largest and most detailed world trade models (GTAP) and is negotiating to use Market Access Map for the market access parameters in GTAP. ITC/MAS is a participant in the interagency working group on the Millennium Development Goals where it processes and calculates selected indicators. In terms of quality, scope and initiative Market Access map is a remarkable achievement. This achievement reflects effective cooperation between ITC and ITC s two sponsoring agencies, the WTO and UNCTAD. While WTO has several databases on market access (the Integrated Database IDB, the Consolidated Tariff Schedule CTS, and the Trade Policy Review Division

25 19 3 ASSESS QUALITY OF OUTPUTS Database) these have been generally only available for internal or restricted use. This restricted availability reflected: concerns by some WTO members about making tariff data widely available for various reasons including: a reluctance to make trade barriers transparent; in some cases the data was available but at a price from individual governments; in a few isolated cases there may have been concerns about confidentiality; the data in the various databases was not always consistent because the databases relied on members reporting there were frequently gaps in years; the databases that did exist lacked user interfaces. Thus for reasons largely beyond the control of the WTO Secretariat, there was a significant gap in the dissemination of public information. This gap had been addressed for many years by UNCTAD though the TRAINS database which contained tariff, non-tariff and trade information. The tariff information in TRAINS was compiled by UNCTAD based on national, international and private sources and is available at the tariff line level. ITC s Market Access map and UNCTAD s TRAINS have become cooperative efforts, strongly supported by the WTO Secretariat. Through Market Access Map and TRAINS the two agencies have effectively pooled resources. Thus Market Access Map was able to build on TRAINS, and by sharing information the managers of TRAINS have been able to enhance that product by access to additional market access information gathered at ITC. Compared with TRAINS, Market Access Map: has a more modern and friendlier user interface making it more accessible to non-technical users; provides a bilateral perspective on market access whereas TRAINS provides market access information by individual country; has more comprehensive information on specific duties, preferential trade agreements, antidumping measures and ad valorem equivalents of specific duties calculated at very detailed levels; and is available for a fee where as TRAINS is distributed in developing countries free of charge and in practice the only users who pay are large US universities.

26 20 3 ASSESS QUALITY OF OUTPUTS In letters written in September and October 2003, the WTO s Director of Research and Chief Statistician wrote expressing appreciation of ITC/MAS contributions. The letter proposed that there be a common database on market access barriers and trade flows for common use by the contributing agencies and consultations among the interested parties and that this database be hosted and managed by ITC with WTO and UNCTAD providing data from their respective databases in common format. The letter further proposed a financial contribution from WTO to support this activity. TradeMap s partners in applications are mainly TSIs with little established internationally recognised reputation. However major donors including USAID, the World Bank and the EU have all supported the development and dissemination of TradeMap. In the field, users in leading South African universities are extremely enthusiastic about TradeMap. Several leading academics conduct regular workshops built on TradeMap. There is widespread agreement among these South African users that it is a very powerful pedagogical device, which enables workshop participants to: appreciate the extent and nature of trade between countries and thereby come to an appreciation of the gains from trade based on practical observation; grasp the power of web based information and analytical support systems; and learn how to interrogate data. Product Map does not seem to be so distinguished in terms of having famous partners and supporters. This reflects its practical focus as a means of integrating MAS and other ITC products and perhaps its status as a work in progress. The same might be said for other traditional products such as MNS, the Inquiry Reply service, Market Briefs, capacity building and single client studies which all tend to meet the demands of traditional ITC clients. The reviewer has worked through the various products with CIE technical staff. They consider the user interfaces to be well designed and efficient. He has also interviewed by telephone technical specialists at the IMF, World Bank, UNCTAD, WTO and Kiel University who are all positive about the quality of TradeMap, Market Access Map and Country Map.

27 21 3 ASSESS QUALITY OF OUTPUTS Relevance of these assessments of quality This acceptance and use of MAS products by internationally recognised agencies is an impressive achievement but the question needs to be asked, how relevant is that acceptance and use to achieving ITC goals? In the reviewer s opinion it is very relevant even though the partners and users are in many ways new to ITC. Taken collectively the production of Market Access map, Country Map and TradeMap are all ventures in transparency. They all make available data and information which had hitherto been collected and held by governments and in UN agencies, but which for a range of reasons was not available to the public. The fact that leading policy agencies are bringing this newly available information to bear on policy is a major contribution to ITC goal 1: facilitating the integration of developing and transition economies into the international trading systems. In the reviewer s opinion increased transparency is a fundamental requirement for such integration. Transparency of publicly collected data on trade flows and of prevailing tariffs is essential for the fair and efficient conduct of policy. Transparency and the analysis made feasible by that transparency is a necessary ingredient for creating a climate for breaking down barriers to trade. When foreign trade regimes are complex and obscure everybody loses. Those people with limited access to information probably lose most. Making information widely available to all is good for everybody and especially good for those who previously had little or no access. It is both fair and efficient. Issues in improving quality Market Access Map which might be regarded as the highest profile product is a huge exercise currently being achieved by a small number of people. In these circumstances mistakes are bound to be made. There is a delicate balancing act between meeting deadlines and living with errors that could damage the confidence of users. The low-key style of the producers of Market Access Map appears to be creating a culture among users whereby those who identify mistakes or complexities become resources for feedback and product improvement. Whether charging user fees makes it more difficult to create such a culture is an important question that is addressed in the next section.

28 22 3 ASSESS QUALITY OF OUTPUTS While Country Map contains a nice treatment of issues arising in the reliability of trade data some of the applications might also be qualified. The Trade Performance Index for example was last posted in There could be more emphasis that the Trade Performance Index and the Trade Sim Model are ways of thinking and not black boxes. TradeMap could be strengthened by moving to more current and less aggregated data to better help commercial firms as they come to close on a deal. One option for overcoming this problem would be to collect directly from countries (at some cost). Another would be for countries to report earlier to UNSD. Product Map suffers from being a work in progress at this stage with many empty cells. In some cases material is dated and not well chosen. The assimilation of other MAS products into Product Map as their main outlet is incomplete and other ITC activities relating to trade information could be incorporated into Product Map. This would enhance the value of these products and Product Map. The MNS would benefit from having producers prepare to a standard well-designed template so that there was a common look or style. Market Briefs also suffer from the lack of a well-designed format. Market Briefs poses a dilemma for the section. They need to be market relevant and not driven by in-house expertise but it is difficult for staff to produce relevant Market Briefs on topics outside their specialty areas. For Market Briefs to be done economically they need to draw on in-house techniques or areas of specialisation or to be outsourced and draw on specialist knowledge. A study of the market for nutmeg in the EU prepared as a Market Brief at the end of 2003 contains little analysis or information that would be new to people in the market. Client studies are typically of adequate but not high quality. The MAS home page promises that client studies combine access to the world s leading trade-related databases with state-of-the-art analytical methodologies and models giving emphasis to the succinct presentation of results to non-technical users and to an effective dissemination of findings. However the following factors mitigate against client studies leading to high quality and influential reports or findings. Client studies usually address a narrow set of questions identified by the client. Notwithstanding the large costs imposed on developing countries by the trade barriers of developed countries (especially with respect to agriculture, textiles and clothing) the biggest barriers to trade in most countries are those that arise from poor domestic