Evaluation of the Commission s Market Access Database Debriefing - Market Access Advisory Committee Brussels, 17 November 2011
About the evaluation External evaluation of the Commission s Market Access Database (MADB): follow-up upon an earlier evaluation from 2006 Done by GHK in collaboration with Copernicus Purpose assessing the effectiveness of achieving MADB s main goals assessing the efficiency of operations providing recommendations for future orientations and eventual improvements of the service (content, user-friendliness, broader reach) Mostly primary research Web design and usability assessment Online survey (202 users) Interviews (users, data providers, EC) User observation Mystery shopping (14 trade promotion agencies) Timescale: February to October 2011
Targeting of the service Dual use: export support tool and monitoring tool 1 st function: Export support tool 2nd function: Monitoring tool Exporters Information intermediaries Public authorities Business associations Producers Traders Logistical service providers Freight forwarders Public trade promotion agencies Private export consultancies (Business representations) EU institutions National governments Agencies, customs offices Regional authorities Chambers of commerce Sector representations Target groups are well defined and appropriately targeted: Exporters: those with some export experience. All Incoterms. Services not explicitly targeted Intermediaries: consultancies, agencies Public authorities: EC, ministries, regional authorities, customs offices Business associations
Level of use 1,704 DUV in 2010 on weekdays (1,359 overall) number of natural persons higher! 26,018 unique visitors, 494,637 visits overall Usage relatively stable over time Level of usage clearly behind its potential Average number of Daily Unique Visitors 2500 2000 1500 1000 500 0 2008 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2009 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2010 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Level of use (2) Marked differences in usage between MSs Significance of exporting to third countries English language skills Awareness of the MADB Unique visitors to the MADB per 100,000 of registered enterprises (2010) Above 500 Between 100 and 500 Between 50 and 100 Between 20 and 50 Below 20 No data Source: traffic figures from DG Trade, Eurostat (number of registered enterprises refers to 2008)
Typology of MADB users Primary target groups are well represented More than half of the users are exporters (at least 56%) Trade advisors account for at least 12% Authorities 10 EU 1 Distribution of respondents by type Public adv 3 Business rep 18 Private adv 9 Other 1 Exporters 56
Typology of MADB users Exporters Mostly active in manufacturing Predominantly large; SMEs are under-represented Trade advisors Mostly not specialised to specific sectors Not all offer one-on-one consulting Public authorities Engaged mostly in policy-making, only rarely in trade promotion or information dissemination Business representations Many organisations working at local/regional level Predominantly cross-sectional representations Size distribution of respondents (companies only) in the sample 14 20 22 44 Large Medium Small Micro
Patterns of usage Very uneven popularity of sections ATD is by far the most often used section in terms of number of visits The TBD, the SPS database and the statistical database are less used The Complaint Register is barely used (<1%) Web traffic figures reflect different frequency of usage Users have already visited or used almost all sections Average visits per user: ADT: 13.5, Trade Barriers: 2.2 Individual sections visited during MADB sessions, 2010 Applied Tariffs db EGIF Statistical db Complaint Register Trade Barriers db SPS Export db Applied Tariffs db EGIF Statistical db Complaint Register Trade Barriers db 0 50 100 8 8 8 23 0 50 100 47 65 Proportion of respondents using individual sections 62 73 92 88 SPS Export db 52
Overall utility of the database ATD and EGIF are seen as highly useful Statistical database and the Complaint register less so Strengths Well adjusted to the needs of target groups Combines several pieces of useful information Easy navigation across databases, integration of data into the HS code system High reliability (accurateness, timeliness) of ATD And EGIF Appropriate level of detail Weaknesses Not sufficiently promoted (Limited search functions) Respondents view on the usefulness of individual sections Applied Tariffs db EGIF Statistical db Complaint Register Trade Barriers db SPS Export db 0 50 100 16 15 33 28 very useful rather useful neutral not so useful 46 18 66 31 26 29 not useful at all 39 29 30 26 18 22 12 16 6 2 8 16 9 9 7 5 7 13 8 10
Additional user needs Early warning information External links for countries not yet covered Removing restrictions on the quota for queries for each IP More links for contact points More information on investment barriers and financial regulations/requirements in order to help exporters
Reliability of the information ATD and EGIF information is very reliable Stakeholders see them as the most reliable tools available on the internet Data in the TBD and the SPS is often outdated and not detailed enough Verification exercise confirms reliability Results for 30 data clusters (10 countries, 3 sectors: textiles, chemicals, wine). 242 pieces of information 22 possible discrepancies (9%): mostly due to time lags in updating information, or more accurate explanation on requirement not given Data in the MADB sometimes more reliable than information of third-country customs offices
User-friendliness of the database Favourable views in general Good performance in comparison with alternative services Some scope for improvement Easier navigation, icons, more consistent use of controls Improving search facilities (better keyword search, auto-suggestions and validation, storing profiles) Multi-country or multi-tariff line searches Printer-friendly output format (PDF) and results in Excel More help for beginners (User Guide) Search engine optimisation (SEO) More complete logging of user behaviour RSS feeds Social media opportunities Multilingual service Respondents view on the userfriendliness of individual sections Applied Tariffs db Exporter s guide Statistical db Complaint register Trade barriers db SPS Export db 22 22 29 30 Very convenient Average convenience Very inconvenient 0 50 100 38 46 28 24 36 30 34 31 39 37 20 25 13 17 5 6 7 5 7 6 10 6 12 4 10 5 Convenient Rather inconvenient
Promotion and awareness The Commission s possibilities to reach businesses are limited Business representations, public authorities claim to inform companies about MADB but public awareness is low Few companies hear about MADB from a business representation or public authority Channels used by MS actors to inform exporters about the MADB Individually, one-to-one Link to MADB on our website Events (conferences) Info material (newsletter) Indirectly, by informing others We do not inform exporters 0 50 100 Perceived level of awareness of exporters of the MADB, by stakeholder group Total Business representation 13 15 9 31 33 30 29 42 53 67 0 50 100 47 47 9 9 Export consultancy 8 25 54 13 Public authority 15 40 40 5 Very well aware average awareness Not aware at all Well aware Not so much aware
Promotion and awareness (2) Only 251 backlinks found 41% business representations 18% public authorities DE, ES, CZ most prominent (but usage data not correlated) MADB is very difficult to find via Google (through trade-related search terms). Out of the top 10 hits for 5 different questions 4 MADB-related sites were found in Latvian 3 in Dutch 2 in Portuguese 1 in French, Hungarian, Polish and Romanian 0 in Finnish, German and Greek Mystery shopping revealed limited awareness/trust of information intermediaries Only 2 out of 14 trade promotion agencies/authorities recommended the MADB promptly In total, 6 recommended the MADB
Value of the service Experimental search to calculate time savings 3 products in 7 countries Time need: 153 minutes on average (MADB: 5 minutes) Information obtained is not well structured, less reliable and up-todate Time saving may add up to 20,833 working days a year, worth at least 3.75 million Additional value from the superior quality of the information (millions of euro) Total cost for MADB: about 2.35 million 2.8 euro per query, 14.8 euro for one average session Less than the estimates for its value Quasi monopoly a possible problem, but unit prices remained stable 10-20% discount is given on maximum budget unit cost lower than for the Export Helpdesk
Key recommendations 1. Extend coverage of the TBD and SPS sections by stakeholder data 2. Improve structure of the EGIF section 3. Improve feedback mechanisms and timeliness of information 4. Increase the frequency of updating the MADB 5. Complement the content with additional information or links 6. Enhance the user-friendliness of the database 7. Consider creating hybrid versions of the ATD and EGIF 8. Consider renaming the database and target information intermediaries in the Member States 9. Improve the visibility of the MADB through search engine optimisation (SEO) 10. Reinforce monitoring of user behaviour 11. Reduce data verification and feedback time
Thank you for your attention Máté Péter Vincze Senior consultant GHK Consulting