E-Government and its Measurements UN E-Government EGM Meeting Background Paper

Transcription

1 E-Government and its Measurements UN E-Government EGM Meeting Background Paper 4 May 2017

2 1. E-government and its Measurements Why Measure e-government? 1 To build effective public sector governance, ICT is used to make the functioning of government more efficient and in improving the delivery of government services for organizations and individuals. E-government development measurement over the years and across the nations is important to analyze the relative and absolute development of e-government world-wide and also to measure the development against the 2030 Agenda for Sustainable Development and other major international commitments and goals. E-government development is at varying stages across countries. Developed economies are relatively advanced in usage of ICT for improving functioning of the public sector and service delivery. However, most developing countries are less advanced and a comparable measurement framework is important for their e-government improvements. Examples: Access to information on available services to complete online processing of requests related to permits, certificates or payments. Effective use of e-government can also improve the efficiency and effectiveness of the public sector and linkages among government agencies. Computers and networks are used to improve the personal productivity of government workers and help in building a more efficient business process which is transitioned to offer government services electronically. In this context, an emerging imperative is to reformulate e-government policies and programmes to exploit these capacities Overview on Benchmarks 2 The digital government, e-government development and the uptake of ICT in the public sector are measured across the globe and benchmarked against each other by various methodologies provided by universities, companies (such as Mckinsey, Cisco, etc.), public-private organizations and multi-lateral international organizations (UN, World Bank, World Economic Forum, European Commission, OECD, etc.) These benchmarks mainly assess the progress made by an individual country over a period of time and compare its growth against other countries which have a political and potentially economic impact and which can influence the development of e-government services. The high-level macro indicators to measure the uptake of the ICT and e-government in the public sector of a country are mainly related to the political and regulatory environment, business and innovation environment, infrastructure, affordability, digital skills, people, business and government usage, economic and social impacts. These benchmarking surveys and reports serve as a tool for the countries and public institutions to learn from each other, identify areas of opportunities and challenges in e-government or digital government and accordingly develop strategies, policies, initiatives for those areas. It also helps in assessing the ICT readiness of the country and build capacities in the gap areas. It further enables discussions of intergovernmental bodies, including the United Nations General Assembly and the Economic and Social Council, on development and critical role of ICT and issues related to e-government. Benchmarking publications are mainly intended for Policy makers Government officials Academia Civil society Private sector 1 Framework for a set of e-government core indicators, Economic Commission for Africa, March E-government Benchmarking in European Union: A Multicriteria Extreme Ranking Approach, 17 February 2017 Page 2

3 Other practitioners or experts in the areas of public administration, e-government, and ICT for development United Nations E-government Benchmark Indicators: 3 United Nations e-government issued a survey which covers 193 countries and helps countries launch their implementation of the 2030 agenda for sustainable development and covers 17 sustainable development goals in support of the agenda. Progress of e-government development in United Nations (UN) is assessed by E-government Development Index (EGDI) and is the composite index based on weighted average of three normalized indices. 1/3rd: Telecommunications Infrastructure Index (TII) wherein data is provided by the International telecommunication union (ITU) 1/3rd: Human capital index (HCI) wherein data is provided by the United Nations Educational, Scientific and Cultural Organization (UNESCO) 1/3rd: Online service index (OSI) wherein data is collected from an independent survey questionnaire that assesses the national online presence of all 193 United Nations Member States E-participation index: quantitative sub-index of the OSI E-government Development Index (EGDI): It is the weighted average of normalized scores on the 3 dimensions: OSI, HCI and TII. These are composite measures and can be analysed and extracted independently. Before normalizing the 3 component indicators, the Z-score standardization procedure is implemented for each component indicator to ensure that the overall EGDI is equally decided by the three component indexes (comparable variance subsequent to the Z-score standardization). If the standardization treatment is not done then EGDI would mainly depend on the component index with the greatest dispersion. The arithmetic average sum becomes a good statistical indicator. The composite value of each composite index is further normalized to fall between the range of 0 to 1 and the overall EGDI is derived by taking the arithmetic average of the 3 component indexes. Indicator Index Description/Components 1/5: Individuals using the Internet (% population) e-government Development Index (EGDI) Telecommunications Infrastructure Index (TII) Human capital index (HCI) 1/5: Fixed (wired)-broadband subscriptions (per 100) 1/5: Wireless broadband subscriptions (per 100) 1/5: Mobile-cellular subscriptions (per 100) 1/5: Fixed-telephone subscriptions (per 100) 1/3: Adult literacy rate (%) 2/9: Mean years of schooling 2/9: Expected years of schooling 2/9: Gross enrolment ratio (%) 3 United nations e-government survey 2016 Page 3

4 Indicator Index Description/Components United Nations Volunteers (UNVs) assessed the country s national website in its native language including the national portal, e-services portal and e-participation portal as well as websites of education, labour, social services, health, finance and environment ministries The survey has a binary response and generated patterns for every positive answer Online service index (OSI) After initial assessment, the evaluations were done on each country and compared and discrepancies are reviewed again Final review is done by the data team co-ordinators by using multiple methods and sources and then sent for senior researcher approval After the 3 step evaluation process, the statistics team produces the OSI ranking and scores are created The total points scored are normalized in the range of 0 to 1 The online index value for a given country is equal to the actual total score less the lowest total score divided by the range of total score values for all countries Derived as a supplementary index to the UN e-government survey Focus on use of online services to provide information to citizens (einformation sharing), interaction with stakeholders (e-consultation) and engagement in decision-making processes (e-decision-making) Identify which countries use online tools to promote interaction between the citizens and government and also within the citizens of the country E-participation index It is a qualitative assessment and based on the participatory services available on the government websites and then carries out a comparative ranking of the countries After calculating the composite indexes, countries are ranked as It is not an absolute measurement of e-participation but a relative comparison between the performances of the countries at a particular time EPI is normalized by taking the total score value for a given country subtracting the lowest total score for any country in the survey and dividing by the range of total score values for all countries Page 4

5 Figure 1: EGDI Index Data for Very high EGDI: values greater than High EGDI: values between 0.50 and Middle EGDI: values between 0.25 and Low EGDI: Less than The Networked Readiness Index (NRI) by WEF 4 The Networked Readiness Index (NRI), as launched by World Economic Forum in 2001, measures the capacity of countries to use ICT for increased competitiveness and well-being. NRI also considers innovation trends of recent years. The framework for the NRI is made up of four main categories (sub-indexes), 10 subcategories (pillars), and 53 individual indicators against different pillars. The full list of indicators, grouped by pillars and sub-indexes are given below The Environment sub-index measures the extent to which a country s market conditions and regulatory framework support ICT development, entrepreneurship and innovation The Readiness sub-index assesses the country by the extent to which the infrastructure and other factors support the uptake of ICT The Usage sub-index considers the level of ICT adoption by a society s main stakeholders government, businesses, and individuals The Impact sub-index determines the broad economic and social impacts accruing from ICT 4 The Global Information Technology Report, WEF-Cornell University-INSEAD, 2016 Page 5

6 Figure 2: The Networked Readiness Index Framework by WEF Note: The indicators marked in asterisk (*) are derived from the WEF Executive Opinion Survey and the remaining indicators are taken from the external sources or sourced from international organizations such as International Telecommunication Union, UNESCO and other UN agencies, and the World Bank As per the NRI 2016 results, the composition of the group of top 10 performers is unchanged from the previous year and mainly consists of the high income countries such as the US, European countries (Finland, Sweden, Norway, the Netherlands, Switzerland, the United Kingdom, and Luxembourg), Singapore and Japan. Thus, the networked readiness of a country is highly correlated with per capita income EU E-government Benchmark Indicators 5 European Union e-government benchmarks implementation has been improving every year and covers 28 countries in its category. The benchmarking framework found on key EU e-government priorities is built on a very rich source of research data using different methods with strong collaboration from member States to build a robust and coherent insight into the current state of e-government plays in the EU-28+ countries. EU e-government benchmark is evaluated through the priority areas of the government action plan and analyzed through the progress of those priority areas which are measured by one or more indicators. 5 e-government Benchmark 2016,European Commission Page 6

7 Mystery shopping method was used to assess these indicators wherein, shoppers are trained and experienced to observe and measure a public service process. They act as prospective users and follow a detailed, objective evaluation checklist. At the start of the evaluation, member states validate the sample of the services under assessment and then validate the results with the responsible organizations in the country. Seven life events are selected as measurements from the common domains of public services and covered in this evaluation process. Each life event is measured once in 2 years wherein countries arrange follow-ups for improvements and compare the progress in Europe as well as other countries. Figure 3: Overview of Life Events under Assessment European e-government landscape has undergone a visible transformation to achieve the policy targets of the e- government action plan and captured the progress made by the EU28+ countries. After three levels of biennial rounds of assessment, the e-government benchmarking technique measures the progress as Page 7

8 Figure 4: Biennial Averages of the Top Level Benchmarks for Eu28+ Countries 6 Key enablers Business mobility Citizen mobility Transparent government User centric government User centricity: It assesses user expectation fulfillment by the government s online services including the availability and experience. It is designed around user s needs and supports flexible and pro-active interactions between citizens, businesses and public organizations. Governments have advanced in making public services digital but the focus is less on the quality of the delivery from the user s perspective. This growth majorly focuses on the availability and usability but citizens have not gained major benefits in terms of ease and speed. The absolute score was 77 in from a score of 70 in (online availability: 81% with an increase of +9 points since 2012 and online usability: 83% with an increase of +4 points since 2012). Transparency: It is a part of the user empowerment priority area and focuses on building trust with the citizens and improve the policy maker s accountability through better use of personal data in the decision making process. It measures the extent of public organization information about their processes which is made accessible to the users. There has been a progress over the years but it has not been consistent and variation has been observed between the countries. Score is 56% with +8 points rise over the years. Transparency is covered as accessibility of personal data to users with a measure of 55% and implementation of service delivery procedures with a measure of 47%. Cross-border services: It measures the extent to which e-government services support citizens and business mobility across the EU28+ borders. It uses the user centric indicators to assess whether the services are available online (quantity) and are also usable (quality). As highlighted in the national assessment, the supply-side performed better than the demand side, thus being the concern area. They are divided as business mobility with a score of 64% (rise of +13 points over the years) and citizen mobility with a score of 52% (rise of +11 points). Key enablers: Biennial averages for Biennial averages for Biennial averages for It investigates the availability of the five key enablers; Electronic Identification (eid), Electronic Documents (edocuments), Authentic Sources, Electronic Safe (esafe), Single Sign On (SSO) in the online services. Score is 54% but some of the individual enablers have scored marginally high in the period. Country Clusters Using Benchlearning Approach: 6 The graph is built based on the guesstimates from the original source Page 8

9 This approach aims to support the definition of e-government policies and strategies that a country should implement by considering the following points 1. Impact of specific context on e-government maturity performances 2. Context-specific differences of countries with similar performances 3. Country differences with similar context and different performances There is a two step process followed which calculates the absolute and relative indicators to cluster the countries. Step 1: Two absolute indicators are used to measure e-government maturity namely Penetration (usage of online e- government services) and Digitisation (Public administration s efficiency and effectiveness in internal procedures). Table 1: Indicators representation Indicator Composed variables Data source Penetration Internet usage to interact with public administration Eurostat Submitting completed forms (from last 12 months) Percentage of individuals who used the internet within the last year Digitization Authentic Sources: personal data pre-filled, documentation required Automated Service: percentage of automated services per country (across all life events Mystery shopping) E-government benchmark Mystery Shopping These indicators have been calculated as a biennial average of 7 life events similar to the mystery shopping method with 3 time series ( , and ). Step 2: Evaluate the factors which shape the specific context of individual countries with three relative indicators Government supply: Spread of e-government services e-government demand: citizens willingness to use online services Environment: Readiness of the background like socio-demographic data, ICT Readiness and Governance structure A cluster analysis is conducted using the absolute and relative indicators to identify the countries with similar e- government maturity performances and context. Five groups of countries with similar context are identified with respect to the context variables (e-government Supply, e-government Demand and Environment). Table 2: Country grouping Group Countries Group 1 Latvia Slovenia Luxembourg Iceland Cyprus Estonia Lithuania Malta Group 2 Poland Germany Italy France UK Spain Group 3 Netherlands Belgium Austria Group 4 Romania Czech Republic Greece Hungary Portugal Bulgaria Croatia Slovakia Turkey Group 5 Sweden Ireland Denmark Finland Norway Page 9

10 Group1: Smaller population (relatively young), highly educated and of medium income (measured by GDP per capita) Group2: Largest population (relatively old), high level of education, maturity of infrastructure in line with the EU average Group3: Large population, high income countries, skilled in ICT and ICT infrastructure is developed, more inclined in using e-commerce and banking services Group4: Less urbanized population with lower income, relatively low education level, few digital skills, infrastructure not highly developed, higher levels of public sector corruption Group5: Small population with high income highly educated and inclined to e-commerce and banking services infrastructures, well developed centralization of services and low levels of corruption The countries are then divided into clusters based on e-government performance factors Figure 5: Performance Measurement of E-government Clusters Neophytes cluster: Scores low on penetration and digitization resulting in low use of ICT opportunities and depends on significant efforts to move towards e-government maturity. High potential cluster: Low level of digitization and medium-high level of penetration which shows that though citizens use online services, increase in public administration processes efficiency and cost savings can be realized only if necessary actions are initiated. Progressive cluster: Medium level of digitization and penetration wherein most of the services are online but lack in full distribution of satisfactory e-government services. They should focus on removing those barriers and opt for a citizen-centric approach. Builders cluster: High level of digitization and medium-low level of penetration wherein public administration is into innovation but there is less usage of online interactions due to low penetration. Mature cluster: High level of penetration and digitization Considering the group of countries with respect to the e-government performance factors, the cross country analysis shows that context-specific variables impact the performance of countries with relevance in degree of Page 10

11 penetration and digitization. Over the time, the countries move from one cluster to another and display different performance paths The Digital Economy and Society Index (DESI) by European Commission 7 The Digital Economy and Society Index (DESI) also measure Europe s digital performance and EU member states in digital competitiveness through certain indicators under the five main dimensions. Figure 6: Digital Economy and Society Index 2017 Ranking DX FI SE NL LU BE UK IE EE AT DE MT LT ES PT EU FR SI CZ LV SK HU CY PL HR IT EL BG RO 1 Connectivity 2 Human Capital 3 use of Internet 4 Integration of Digital Technology 5 Digital Public Service As per the DESI 2017 ranking, countries such as Denmark, Finland, Sweden and the Netherlands have the most advanced digital economies in the EU followed by Luxembourg, Belgium, the UK and Ireland. Romania, Bulgaria, Greece and Italy have the lowest scores on the DESI. However, all the Member States have improved in Slovakia and Slovenia improved the most (more than 0.04 as opposed to an EU average of 0.028) and Portugal, Latvia and Germany (below 0.02) improved a little. The DESI was developed following the guidelines and recommendations in the OECD s Handbook on constructing composite indicators: methodology and user guide. The data included in the index were mostly collected by the European Commission services (DG CNECT, Eurostat) and by ad-hoc studies launched by the Commission services. The DESI is composed of 5 principal dimensions, each divided in a set of sub-dimensions, which are in turn composed by individual indicators The Connectivity dimension measures the deployment of broadband infrastructure and its quality. The Human Capital dimension measures the skills required to leverage the opportunities offered by a digital society. The use of internet dimension measures the variety of activities performed by online citizens. The activities mainly range from consumption of online content (videos, music, games, etc.) to modern communication activities or online shopping and banking. 7 The Digital Economy and Society Index (DESI), European Commission, The graph is built based on the guesstimates from the original source Page 11

12 Integration of digital technology dimension assesses the digitisation of businesses and their exploitation of the online sales channel. Digital public services dimension measures the digitisation of public services, focusing on e-government. Dimension Sub-dimension Indicator Connectivity Digital Skills Use of Internet Integration of Digital Technology Digital Public Services Fixed Broadband Mobile Broadband Speed Affordability Basic Skills and Usage Advanced skills and Development Content Communication Transactions Business Digitization E-commerce e-government Fixed Broadband Coverage Fixed Broadband Take-up Mobile Broadband Take-up 4G Coverage Spectrum NGA Coverage Subscriptions to Fast Broadband Fixed Broadband Price Internet Users At Least Basic Digital Skills ICT Specialists STEM Graduates News Music, Videos and Games Video on demand Video Calls Social Networks Banking Shopping Electronic Information Sharing RFID Social Media einvoices Cloud SMEs Selling Online ecommerce Turnover Selling Online Cross-border e-government Users Pre-filled Forms Online Service Completion Open Data Middle East Digitisation Index Benchmark by Mckinsey 9 The Middle East digitisation index assesses the impact and position of digitisation across nine Middle Eastern countries namely Bahrain, Egypt, Jordan, Kuwait, Lebanon, Oman, Qatar, Saudi Arabia, and the United Arab Emirates. 9 Digital Middle East: Transforming the region into a leading digital economy Page 12

13 It was observed that only 6 percent of the Middle Eastern public fall under the purview of a digitised smart government. The study reveals that there is a strong correlation between a country s GDP per capita score with the McKinsey Digitisation Index wherein a healthy GDP growth value allows countries to spend more on the digital adoption increasing the countries performance on the digitisation index which in turn contributes to an increase in the GDP and the economic growth. After considering the sources like International Monetary Fund, World Economic Forum, World Industry Service Database, World Bank, World Market Monitor, Wearesocial.org, Euromonitor, ITU, UN data, Countries are assessed on the level of digital adoption across four areas Consumer Business Government sectors ICT supply and innovation Further, the McKinsey Middle East Digitisation Index assesses the country s level of digitisation based on 24 variables of supply (digital creation) and demand (digital adoption) as provided in the table below Sub-index Area Metric Description Consumer Internet usage Internet penetration Active Internet users over total population Mobile broadband usage Mobile broadband users per 100 people Smartphone usage Smartphone penetration Active smartphone users over population Social networks usage Active social network accounts (% internet users) Active social media users divided by active internet users Time spent on social media Hours spent per day Internet retail Internet retail value as % of total retail Internet retail divided by total retail Business Technology usage B2B internet usage Survey (ranking of min 1 to max 7) response: (what extent do businesses use ICTs for transactions with other businesses?) Firm level-technology absorption Survey (ranking of min 1 to max 7) response: (what extent do businesses adopt new technology?) Advertising Online ad spend per capita Internet advertisement spending per capita in USD Online ad spend as % of total Internet spending as a percentage of total advertisement spending Government Promotion of ICT Government s success in ICT promotion Survey (ranking of min 1 to max 7) response: How successful is the government in promoting the use of ICT Page 13

14 Sub-index Area Metric Description Usage of ICT Government Online Service Index ICT use and government efficiency Core administration system digitization Digital identification E-government index Survey response: The Government Online Service Index assesses the quality of government s delivery of online services on a 0-to-1(best) scale Survey (ranking of min 1 to max 7) response: what extent does the use of ICT by the government improve the quality of government services to the population? Based on multiple indicators/surveys such as financial management information system, human resources information system, e-tax, e-customs, e-procurement Based on multiple indicators/surveys such as access to services, digital signature, card features United Nations E-government development index ICT supply and innovation Coverage 3G network coverage Mobile network coverage as percentage of population Connectivity International Internet bandwidth International internet bandwidth in kbps per user Secure Internet servers per million population Average download speed Secure internet server per capita Average download speed in kbps Affordability Broadband tariffs Fixed-broadband internet tariffs in PPP USD per month PCT patents ICT companies Mobile broadband pricing PCT patents application per million population Share of country s ICT companies revenue in global Top 1000 Mobile-broadband pricing PPP adjusted PCT patents, applications per capita Share of 2015 revenues per country as % of overall top 1000 revenues E-Government Rankings by Waseda University - IAC International 10 Waseda University, the institute of e-government conducted a survey through numerous workshops, forums, professional meetings and discussions. This data was taken from Organization for Economic Co-operation and Development (OECD), Asia Pacific Economic Cooperation (APEC), the International Telecommunications Union (ITU), the World Bank (WB), and many government agencies, Think tanks and NGO/NPOs in charge of e-government 10 The 12 th Waseda IAC International e-government Rankings Survey 2016 Report Page 14

15 activities in their respective countries. It includes economies of 65 countries and evaluates the application of new ICT administration trends in those countries. The Waseda-IAC International e-government ranking considers the benchmarking indicators to assess the latest ICT developments of e-government. Researchers found that IT is increasing and is widely used in government activities and IT applications are used in different fields of various sectors. Due to the trending information and communication technologies role in development of e-government across the countries, 10 indicators are classified along with its 35 sub-indicators Indicators Network preparedness/infrastructure Management Optimization/ Efficiency Online Services / Functioning Applications National Portal/Homepage Government CIO e-government Promotion E-Participation/Digital Inclusion Open Government Cyber Security Sub-indicators Internet Users Broadband subscribers Mobile Cellular subscribers Optimization Awareness Integrated Enterprise Architecture Administrative and Budgetary Systems E-procurement E-tax Systems E-custom Systems E-health Systems One-stop Services Navigation Interactivity Interface Technical Aspects GCIO Presence GCIO Mandate CIO Organizations CIO Development Programs Legal Mechanism Enabling Mechanism Support Mechanism Assessment Mechanism E-Information Mechanisms Consultation Decision-Making Legal Framework Society Organization Legal Framework Cyber Crime Countermeasure Page 15

16 Internet Security Organization The use of Emerging ICT The use of Cloud Computing The use of Internet of Things The use of Big Data Network Preparedness/Digital Infrastructure: It is an essential requirement for conducting data communication between two access points. Communication from government s premises to the citizens becomes a necessity for effective e-government implementation. This indicator is supported by three sub-indicators. Management optimization: This indicator reflects the ICT utilization for improving government business processes and optimizes the effort to integrate the silo of business processes using ICT. This indicator is supported by five subindicators. Online Services/ Applications: Integrates business processes, policies, procedures, tools, technologies and human efforts to support both assisted and unassisted customer services using networks. Government provides communication services at all levels and plans to establish e-service infrastructure and organizational capacity for constituents to transact official business online. This indicator is supported by five sub-indicators. National Portal/Homepage: It is the foundation of e-government s basic interface for stakeholders to access government and also making all services available via one portal. It also helps reduce costs, improve communication experience and improves perceptions about the government. This indicator is supported by four sub-indicators. Government Chief Information Officer (GCIO): The CIO has the rights to align management strategy with ICT investment so that the balance between the applied business strategy, organizational reform and management reform is achieved. This indicator is supported by four sub-indicators. E-government Promotion: It is evaluated by using a list of parameters to judge the degree of development in each sector and current status of e-government promotion. This indicator is supported by four sub-indicators. E-Participation/ Digital Inclusion: It refers to ICT supported participation in government and governance processes like service delivery, policy making. It takes the demand side of e-government into the account and checks as to see to what degree people are using e-government platforms. This indicator is supported by three sub-indicators. Open Government Data: Data which can be freely used, reused and redistributed by anyone with respect to the requirement to attribute and share sources. The released data is available to allow the public to access it without having to pay fees and avoid unfair restrictions over its use. This indicator is supported by three sub-indicators. Cyber Security: It strives to attain and maintain security properties of the organization and user assets against relevant security risks in the cyber environment. This indicator is supported by three sub-indicators. The Use of Emerging ICT: Aims to accommodate developed countries that examine the emerging technologies for improving e-government quality and developing countries which expect to obtain an important score by implementing ICT technologies for their e-government. This indicator is supported by three sub-indicators HAL E-government Evaluation 11 HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents. The documents come from teaching and research institutions in France or abroad, or from public or private research centres. As per a research document by HAL, the global e-government evaluation is done through putting quantitative criteria and dimensions to each of the four major qualitative points of view such as infrastructure, investments, e-processes and user attitude. Further the criteria are grouped and ranked for assessment of the European Union countries in e- government development. 11 E-government Benchmarking in European Union: A Multicriteria Extreme Ranking Approach Page 16

17 In this paper, 21 EU countries are evaluated and ranked over their e-government progress. Their ranking is obtained through an additive value model which is assessed by an ordinal regression method and the use of the decision support system, MIIDAS. Figure 7: The Evaluation System of Global e-government Page 17

18 1.2. Methods and Instruments Used to Measure E-government Progress E-government development ranking studies conducted by various institutions, use a mix of instruments (Benchmarking, bench-learning, self-assessment) and methodologies (web research, public data analysis, user testing, interviews) to measure the progress of e-government in various countries. Some of the methods and instruments used by the benchmarking studies illustrated in our study are as follows EU egovernment Benchmark Indicators 12 Indicator User Centricity Transparency Cross-border services Key technological enablers Direct Source of Data/ Method used Mystery shopping method: Involves shoppers (prospective users) who are trained and experienced to observe a public service process. It is method of choice for assessing all the top level benchmarks Benchlearning: This approach divides the countries into clusters with similar e-government maturity performance as Neophytes, High Potential, Progressive, Builders, and Matured clusters United Nations e-government Benchmark Indicators 13 Index EGDI Telecommunications Infrastructure Index (TII) Human capital index (HCI) Online service index (OSI) Direct Source of Data/ Method used Survey Method: Aims to give an indicative assessment of e-government through a performance rating of national governments relative to one another. EGDI is a weighted average of normalised scores on the three most important dimensions of e-government: scope and quality of online services (Online Service Index, OSI), status of the development of telecommunication infrastructure (Telecommunication Infrastructure Index, TII) and inherent human capital (Human Capital Index, HCI). Data is collected from the International Telecommunications Union (ITU) Data is collected from United Nations Educational, Scientific and Cultural organization (UNESCO) Data is collected from an independent questionnaire, assessing the national online presence of all 193 United Nations member states It assesses a number of features related to online service delivery, which includes government approaches, open government data, e- participation, multi-channel service delivery, mobile services, usage uptake, digital divide and innovative partnerships using ICT This data is collected by researchers under UNDESA supervision through 12 egovernment Benchmark 2016,European Commission 13 United nations egovernment survey 2016 Page 18

19 primary research and collection endeavour E-participation index (supplementary index) UN egovernment Survey includes additional questions to address data publishing and sharing by government agencies The Networked Readiness Index (NRI) by WEF 14 Sub-index The NRI index is divided into four main subindexes which are further classified as 10 subcategories (pillars) with 53 individual indicators 1. The Environment 2. The Readiness 3. The Usage 4. The Impact Direct Source of Data/ Method used The 53 indicators which are a part of the four sub-indexes are sourced in two ways: The individual indicators partly are sourced from international organizations like International telecommunication union (ITU), UNESCO, other UN agencies and World Bank In order to align them with the Survey results, a min-max transformation is applied to transform them into a 1-to-7 scale World Economic Forum s Executive Opinion Survey (the Survey) The survey measures concepts which are qualitative in nature for those countries wherein the International comparable statistics are not available It is administered annually to over 14,000 business executives of the economies covered in the NRI It is a unique source of insight into critical aspects related to a country s enabling environment, preparedness of population, ICT usage, ICT impacts The survey scores are measured on a 1-to-7 scale and do not require transformation prior to aggregation process The Digital Economy and Society Index (DESI) by European Commission 15 Dimensions The index covers five dimensions with sub dimensions and indicators are described further. Direct Source of Data/ Method used Data is collected from the European Commission services (DG CNECT, Eurostat) and also by ad-hoc studies launched by the Commission services The Dimensions cover: 1. Connectivity 2. Digital skills 3. Use of Internet 4. Integration of Digital technology 5. Digital public services Middle East Digitisation Index Benchmark by Mckinsey The Global Information Technology Report, WEF-Cornell University-INSEAD, The Digital Economy and Society Index (DESI), European Commission, Digital Middle East: Transforming the region into a leading digital economy Page 19

20 Subindex The index measures the country s level of digitization based on 24 variables on supply and demand under 4 major subindexes 1. Consumer 2. Business 3. Government sectors 4. ICT Supply and innovation Direct Source of Data/ Method used Data is collected in two ways according to the nature of the variables: The data is collected from various sources like International Monetary Fund (IMF), World Economic Forum (WEF), World Industry Service Database, World Bank, World Market Monitor, Wearesocial.org, Euromonitor, ITU, UN data Some of the data for variables is obtained through surveys with the scale of 1-to-7 (1= not successful at all; 7= extremely successful E-Government Rankings by Waseda University - IAC International 17 Indicator Waseda-IAC International e- Government ranking includes 10 main indicators which are classified along 35 sub-indicators: 1. Network preparedness/infrastructure 2. Management Optimization/ Efficiency 3. Online Services/ Functioning Applications 4. National Portal/Homepage 5. Government CIO 6. E-Government Promotion 7. E-Participation/Digital Inclusion 8. Open Government 9. Cyber Security 10. The use of Emerging ICT Direct Source of Data/ Method used E-Government development is evaluated by this ranking survey based on group of sub-indicators in a country ranging from policy development and e-services implementation to management optimization and egovernment promotion For the Survey, the data is researched through workshops and forums with a variety of organizations like Organization for Economic Cooperation and Development (OECD), Asia Pacific Economic Cooperation (APEC), the International Telecommunications Union (ITU), the World Bank (WB), government agencies, think tanks and NGO/NPO s of respective countries HAL E-Government Evaluation 18 Criteria The global e-government evaluation is done through putting quantitative criteria and dimensions to each of the four major qualitative points of view such as 1. Infrastructure 2. Investments 3. E-processes 4. User attitude Direct Source of Data/ Method used The primary data is collected from reliable and recognized sources such as the Statistical Administration of the European Commission (Eurostat) and the International Monetary Fund (IMF) Data is also collected from the e-government surveys (e.g. UN survey) 17 The 12 th Waseda IAC International e-government Rankings Survey 2016 Report 18 E-government Benchmarking in European Union: A Multicriteria Extreme Ranking Approach Page 20

21 Indicators used in benchmarks are usually quantitative and collectively form a framework for assessment and ranking. The multicriteria evaluation system uses 4 major points which are further classified into 8 criterias Open Data Maturity by European Data Portal, 2016 Criteria Open data (OD) maturity is assessed for EU28+ (31 countries) based on following indicators and subindicators: 1. Open data readiness policy: a. Presence of OD policy b. Licensing norms c. National coordination d. Use of data 2. Portal maturity: a. Usability of the portal b. Re-usability of data c. Spread of data across domains 3. Open data readiness impact: a. Social b. Economic c. Political Direct Source of Data/ Method used The indicators and the sub-indicators are assessed through a list of 72 questions that describe the ideal situation for the open data maturity The questionnaire was finalised with the individual country representatives from the Public Sector Information (PSI) expert group, chaired by the European Commission The questions are then categorised into four main categories: presence of an OD policy, use of OD, impact of OD, and portal features Out of 72 questions, 54 questions were multiple-choice or open quantifiable questions that are scored The sources used for answering these questions or assessing indicators were: Insights from the existing open data monitoring activities such as the Open Data Monitor and the Open Data Barometer, Open Data Index and the epsi scoreboard Survey method to collect additional information from each European country for the list of questions Desk research on different national portals for validating data availability and portals usability Consider various monitoring activities and studies that assess the benefits of Open Data 1.3. Summary of Benchmarks E-government development benchmarking or digitization assessment for the countries across the globe is done by various international organizations, companies and universities. Some of the key insights from the study of the benchmarking practices are: More governments are using ICT to deliver services to engage people in decision-making processes in all regions of the world. There is a positive global trend towards higher levels of e-government as countries in all regions are increasingly embracing innovation and utilizing ICT to deliver services and engage people in decision-making processes. An uneven e-government progress has been witnessed within and among countries due to lack of access to technology, poverty and inequality that hinder citizens uptake of ICT and e-government for sustainable development. Page 21

22 E-government has the potential to help support the implementation of the 2030 UN agenda and its 17 sustainable development goals (SDGs). Efforts are made by more countries to ensure that public institutions are more inclusive, effective, accountable and transparent through opening up their data for public information and scrutiny. The methods used for collecting data for benchmark indicators were mystery shopping, user survey, statistical analysis, interviews, etc. and some of the instruments used to conduct the benchmarking studies were benchlearning approach, self-assessment and benchmarking. Page 22

23 Summary of Benchmarks EU e-government benchmarks UN e-government benchmarks Networked Readiness Index by WEF Digital Economy and Society Index by EU McKinsey Middle East Digitisation Index e-government rakings by Waseda University and IAC International HAL Global Evaluation System Description EU e-government benchmarks measure progress over the years with four indicators and ranks the countries into clusters basis penetration and digitization UN e-government benchmarks the countries according to four quantitative/ qualitative indexes with components like digital infrastructure, education/literacy, population, participation, transparency and finally ranks them according to the EDGI index The Network readiness index benchmarks the countries according to four sub-indexes with 10 pillars as subcategories and 53 individual indicators Indicators linked to Education, Population, digital Infrastructure, Government transparency, participation and economy growth are covered DESI tracks the evolution of EU member states in digital competitiveness through 5 principal dimensions and 13 sub dimensions Measures the digital economy in nine countries in the Middle East Considers the benchmarking indicators to assess the latest ICT developments of e- government Researchers found that IT is increasing and is widely used in government activities and IT applications are applied in fields of various sectors As per a research document by HAL, the global e-government evaluation is done through putting quantitative criteria and dimensions to each of the four major qualitative points of view such as infrastructure, investments, e- processes and user attitude Type of Agencies European Commission (EU institution) UN (Intergovernmental organization) United Nations Department of Economic and Social Affairs (DESA) Division for Public Administration and Development Management (DPADM) World Economic forum (Non-profit foundation) INSEAD (Business school, France) European Commission (EU institution) McKinsey & Company (Private global management consulting company) The Institute of e- government-waseda University in Japan IAC International (academic institution) National Technical University of Athens (University) No. of countries assessed 28 European Union countries 193 UN member states 139 countries across the globe 29 EU Member States 9 countries in the Middle East 65 countries across the globe 21 European countries Page 23

24 Top 10 countries rankings High-level Dimensions/Ben chmarks Strengths Countries with higher e-government maturity - Norway, Sweden, Estonia, Netherlands, Finland, Ireland, Iceland, Germany, France and Denmark User centricity Transparency Cross-border services Key enablers Detailed in-depth indicators Constant evolving of indicators Mix of indicators (from backoffice to user centricity) Validation of figures by EU Member States Strong political support Primary data collection Top 10 countries - United Kingdom, Australia, Republic of Korea, Singapore, Finland, Sweden, Netherlands, New Zealand, Denmark and France Telecommunication infrastructure index (TII) Human capital index (HCI) Online service index (OSI) E-participation index Extensive Country Coverage (193) Strong brand/reputation, especially the ranking Combination of different relevant dimensions (TII, HCI and OSI) Primary data collection for OSI Thematic topics for different editions emphasizes US, Finland, Sweden, Norway, the Netherlands, Switzerland, the United Kingdom, Luxembourg, Singapore and Japan Environment subindexes Readiness sub-indexes Usage sub-indexes Impact sub-indexes Focus on enabling environment and readiness Primary data collection among many enterprises Denmark, Finland, Sweden, Netherlands, Luxembourg, Belgium, the UK, Ireland, Estonia and Austria Connectivity Human capital Use of internet Integration of digital technology Digital public services Condensed view of politically relevant indicators Leverages existing EU data Focus on Digital Society as a whole and not just E-Government Bahrain, Egypt, Jordan, Kuwait, Lebanon, Oman, Qatar, Saudi Arabia, and the United Arab Emirates Consumer Business Government sectors ICT supply and innovation Very specific regional focus Condensed summary of key findings Top 10 countries - Singapore, USA, Denmark, Korea, Japan, Estonia, Canada, Australia, New Zealand, UK and Taiwan Network preparedness Management optimization Online services National portal Government CIO e-government promotion E-participation/Digital inclusion Open government Cyber security Broad set of indicators Sweden, Denmark, Finland, Netherlands, Norway, Germany, United Kingdom, France, Estonia and Austria Infrastructure Investments E-processes User's attitude Mix of Qualitative and Quantitative indicators based on existing sources Page 24

25 Weaknesses Limited country coverage globally Resource intensive to do the mystery shopping Validation of results by MS costs a lot of time and can lead to negotiation behavior Focus on national portals Limited number of services/indicators that can be assessed Supply-side focused Country coverage not the entire world Limited primary data collection Composite indicator of numerous existing benchmarks E-Government Dimension uses a small subset of figures of the E-Government Benchmark Limited country coverage globally Very limited country coverage Limited data collection Commercial focus Very high level Not all countries in the world covered No new data collected compared to other benchmarks High level focus of indicators Limited country coverage Limited set of indicators Page 25

26 Page 26