Reference Model for E-Government Monitoring, Evaluation and Benchmarking

Transcription

1 Reference Mode for E-Government Monitoring, Evauation and Benchmarking Egidijus Ostasius, Agirdas Laukaitis Vinius Gediminas Technica University Sauetekio ave.11, LT-10223, Vinius, Lithuania e-mai: The e-government as a resut of innovations, emerging technoogy and internet expansion is a reaity today and became a necessity for any country wishing to enter the 21st century as a competitive nation in the word arena. The processes are constanty changing and continue to drive and provide opportunities and open new possibiities for e-government deveopment. The evoution of society requires pubic administrations to tacke many new chaenges and changes. A these evoutionary changes demand the further deveopment of government soutions to meet the chaenges and new adequate requirements. After the soution is deveoped, it must be assessed whether those requirements have been met. As the e-government is constanty evoving a around the word this evoution is reguary measured and tracked by pubic institutions, private agencies and researchers. There are a number of pubished research papers on modes that aim to evauate e-government but most of them are focused on some particuar purposes and objectives, types of evauation perspectives. However, there is no evauation mode that coud be directy adapted to e-government deveopment in each country or each organization or each service in the country. In this paper, we aim to combine the anayzed modes into a conceptua mode that coud provide a common framework for future research in the area of e-government assessment and a common point of reference in this area. For this purpose, a reference mode for e-government monitoring, evauation and benchmarking is proposed. The mode is not associated with any particuar e-government definition and/or mode, evauation purpose, dimension or criterion. It coud be adapted to many specific cases in the process of design of specific modes for monitoring, evauation or benchmarking of e-government or e-government service. Keywords: e-government, e-government services, assessment, evauation, monitoring, benchmarking, reference mode. Introduction The rapid transformation of our society and the digita revoution, aong with budgetary pressures pose chaenges for governments and the future of pubic services (European Commission, 2013a). This promotes not ony to invest in e- government projects in the hope that these chaenges wi be met, but aso to measure and assess the government's progress in the improvement and change. There are proposed various modeing techniques and approaches for e-government benchmarking, monitoring and evauation. As a benchmarking in our context, we refer to a process of comparing one s e- government characteristics (such as services) to best practices from other countries or different e-government administrations in the same country, as a monitoring - a process of comparing one s e-government characteristics in different periods for measuring improvements after deveopment of changes in e- government soution of the same administration, and as an evauation - a process of assessment one s e-government characteristics according to some purpose or perspective (e.g. evauating the maturity of e-government service). However, despite the importance of e-government evauation, the iterature anayzed in this paper suggests that this area is sti fragmented and imited by different e-government evauation modes. As a resut, it is not so easy to find and adapt the mode in case of the particuar country or the organization. This paper aims to review different methods and approaches of evauation and benchmarking and to present a generic mode that coud be appied for constructing the particuar e- government assessment modes. E-government because of the processes of innovations, emerging technoogy and Internet expansion over the ast years became a reaity today. These processes are constanty and continue to drive e-government deveopment. E-government is said to be an efficient and effective way of deivering government services to its customers. Web information accessibiity and onine transactiona services increase transparency, openness of bureaucratic institutions and hence accountabiity and reduce cost of transactions (Kachwamba & Hussein, 2009). It made a great effect on transforming the way of interaction in society that opened new possibiities for the deveopment of governments. Initiay, e-government may seem ike another option for communication with citizens. Peope connected to the Web using their desktop and mobie devices or dedicated kiosks are abe to communicate, access information and services by means that severa decades ago were not possibe. But in the face of rising demands from demographic, economic, socia, and goba trends, e-government no onger appears to be a matter of choice, but a necessity for any country wishing to enter the 21st century as a competitive nation in the word arena (Kumar et a., 2007). Citizens and businesses are therefore expecting better and more individuaised pubic soutions and services, efficient and effective service deivery, burden reduction, transparency and participation. As a resut the e-government added new concepts in the science of pubic administration such as: transparency, accountabiity, citizen participation in the evauation of government performance, and changed the poitica practices which transitioned to e- democracy and e-governance (Mohammad et a., 2009). It may be, however, these e-government benefits wi depend on the stakehoder groups interests (Rowey, (2011); therefore this shoud be taken seriousy during the reaization of e-government services and soutions. The evoution of society requires pubic administrations to tacke many new chaenges incuding those around demographic change, empoyment, mobiity, security, environment and many other areas (European Commission, 2013a). Moreover, e-government deveopment can be seen as a compex strategy intended to change the beiefs, attitudes, vaues, and structure of governments so that they can better

2 adapt to information technoogies, markets, and chaenges (Zarei et a., 2008). A these evoutionary changes demand the further deveopment of government soutions to meet the chaenges. In order to respond to these changes new requirements for the government shoud be introduced. To deveop a successfu government soutions carefu requirements eicitation and anaysis has to be performed (Krenner, 2002). However, merey isting requirements is not enough and repeated consutation with potentia users is necessary if user requirements are to be successfuy impemented (van Vesen et a., 2008). The requirements wi direct the decision-makers and deveopers to deveop the government soutions on the desired way. When deveoping government soutions, one focus shoud be on pubic services as they are the centra eements of the system from the point of view of the pubic administration as we as from the point of view of the citizens and businesses (Krenner, 2002). The requirements may refect different aspects of the government to be deveoped and may be cassified according to different views or dimensions. There may be different categories of requirements - not ony for the pubic services but for the whoe government system. For exampe, they can be defined as process specific, technica, user, security reated, aw based, organizationa, socia and poitica as we as data and information specific requirements. After the soution is deveoped, it must be assessed whether those requirements have been met. Probaby, in view of the constanty changing environment and the emergence of new chaenges, the assessment shoud show how the requirements themseves shoud be changed - added, modified or even abandoned. As the e-government is constanty evoving a around the word this evoution is reguary measured and tracked by pubic institutions, private agencies and researchers (Assar et a., 2010). These measurements mean the evauation of e-government the existing or deveoping e-government soutions in ine with new features and chaenges to be met. Deveopment of an e-government mode is seen as a unique system of systems represented in a generic mode (Zarei et a., 2008). It is usuay caed the e-government deveopment or evauation mode. Evauation is vita to discovering the current state of e- government deveopment, working out the extent to which objectives within various strategies and action pans have been reached, ascertaining strengths and weaknesses, shaping new guideines, ooking for exampes of best practice and finay comparing different e-government organisations at the nationa and internationa eves (Kunstej & Vintar, 2004). E-government provides a number of benefits that can be seen from different points of view of different stakehoders perspective (Gomez-Reynoso & Sandova-Amazan, 2013). That eads to different purposes or perspectives of e-government assessment, monitoring and benchmarking. For exampe, poicymakers and researchers use e-government benchmarking studies to hep monitor impementation of e-government services, using the information to shape their e-government investments (Rorissa et a., 2011). For European countries, benchmarking is used to stimuate mutua earning, to perform mutiatera surveiance, and to contribute to further convergence of their poicies (Capgemini et a., 2014). E- government assessment for benchmarking might measure inputs, process, outputs, gain (outputs reative to inputs), demand, usage, effectiveness, impact, vaue for money (Bannister, 2007). They aso are used to assess the progress made by an individua country over a period of time, and to compare its growth against other nations (Rorissa et a., 2011) and/or e-readiness in different areas such as IT infrastructure, human resources, poicies and reguations, economic environment, e-government transformation (Azab, 2009). On the other hand there is an urgent need to study how to efficienty and effectivey deveop e-government systems and how to measure progress so as to estabish a road map to achieve the desired service eve (Siau & Long, 2005) or the maturity of the service (Vadés et a., 2011). Taking into consideration the amount of information and communications technoogy (ICT) investments made by the governments as we as their inherent potentia for transforming pubic services e-government evauation is imperative (Tsohou et a., 2013). In other cases the assessment may be focused on tangibe and intangibe benefits received from the government (Gupta & Jana, 2003), project vaue (Esteves & Joseph, 2008), user acceptance (Verdegem & Vereye, 2009), pubic vaue (Friendand & Gross, 2010; Karunasena & Deng, 2012), e-government poicies (Stanimirovic et a., 2010; Stanimirovic & Vintar, 2013), stages of growth, Internet service deivery, cost and benefits (Griffin & Hapin, 2005), functionaity, user-centricity, democracy (Mukabeta Maumbe et a., 2008), integration (Lam, 2005), interoperabiity (European Commission, 2013b), open data (Kaampokis et a., 2011) and others. Methods and Modes for E-government Assessment A number of methods and modes for e-government assessment and evauation have been proposed in the iterature. Their cassification depends on different aspects of the modes, such as the purpose and objectives of e-government assessment, types of evauation perspectives - dimensions, criteria, measurement rues and others. A we known representative of the evauation of modes is e-government readiness or ereadiness. E-government readiness primariy assesses the extent to which governments or economies are equipped to deiver various governmenta services onine and expoit ICT for interna functioning of the government (Omari & Omari, 2006). However, readiness maybe interchanged with deveopment : the term e-government deveopment describes how far governments have actuay advanced in this fied instead of how ready or abe they might be to do so, which was how egovernment readiness described nationa capacity (United Nations, 2010). Another important cass of evauation modes is composed of the e-government maturity modes as an essentia part of stage modes that means the evoution of e-government is often modeed by sequentia steps, in the stages of growth modes (Kievink & Janssen, 2009; Concha et a., 2012). Differences between these two casses are that e-readiness comprises of a prerequisite necessary to impement e-government whie egovernment maturity refers to the actua eve of e-government progress an e-government has attained based on the assessment (A-Khatib, 2009). These modes were anayzed and discussed by different authors (Irani et a., 2006; Coursey & Norris 2008; Lee, 2010; Ifinedo & Singh, 2011; Fath-Aah et a., 2014; Tripathi & Gupta, 2014) and some of them were adapted to the new modes (Moon, 2002; Siau & Long, 2005; Affisco & Soiman, 2006). Summing up these anaysis it can be concuded that most of the stage modes are focused on different purposes and/or evauation dimensions what is conditiona on appropriate definitions of various stages but the architecture of the evauation mode for a these modes is the same. However, most of these methods of e-government assessment have been too narrowy defined for them to propery promote the deveopment of soutions, i.e. integrated services or ife-events, that coud fuy utiise IT potentia and offer rea benefits to citizens and businesses as we as the administration itsef. Mosty they are just the refection of too narrow focus on e-government deveopment (Kunstej & Vintar, 2004). So some authors proposed an expanded way to assess e-government

3 deveopment based on reasonabe hoistic principes rather than a stage mode ony (Sandova-Amazan et a., 2013). Design of hoistic modes are based on we known Wimmer's Hoistic Framework which supports integrated modeing of e-government services and synchronization with the technica deveopments (Wimmer & Tambouris, 2002). Though this approach was initiay used to ensure that the e- government services deivered by a project wi meet a reevant requirements it can be successfuy appied to other evauation purposes. This is shown by the efforts of foowing works which appy a hoistic approach to create mutidimensiona evauation modes and use them in practice. As an exampe of such an activity has been created e-government Maturity Mode (egov- MM), integrating the assessment of technoogica, organizationa, operationa, and human capita capabiities, under a muti-dimensiona, hoistic, and evoutionary approach (Iribarren et a., 2008; Vadés et a., 2011). A simiar mode named e-government Procurement Observatory Maturity Mode (egpo-mm) is focuses on both ega and institutiona arrangements, and portas' technica aspects (Concha et a., 2012). Other authors propose a mutidimensiona mode based on a conceptua mode that is a necessary initia effort to buid more integrated and comprehensive methodoogies for measuring and evauating eectronic government (Luna-Reyes et a., 2012). Another work provide a hoistic evauation mode from a specific stakehoders perspective that is based on the most successfu measurement factors that impact the satisfaction of users with an e-government service (Osman et a., 2011). However, these modes can not be directy adapted to e- government deveopment in each country (Yidiz, 2007) or each organization or each service in the country, as there may be different oca poicies and requirements. On the other hand the necessary pre-conditions for e-government depend upon the most important needs of a society. For exampe, the eve of technica infrastructure, ega framework and professiona skis needed for egovernment, vary with the objectives being pursued (Omari & Omari, 2006). Different infrastructures (incuding technica, cutura, socia, poitica and economic) of different countries create different requirements that shoud be adequate to their governments and their citizens (Zarei et a., 2008). So there is a need for more customized modes that coud be adapted to different and changing requirements in order to monitor, evauate and benchmark readiness of pubic administrations to compy with the new chaenges (Vadés et a., 2011). According to the above anaysis there was not found a mode that coud be universay appied for e-government assessment of any country or any service. The presence of a number of these modes poses another probem the ack of a common framework of reference for the assessment (Siau & Long, 2005). In addition, anaysis of the iterature and practice suggest that a universa assessment mode has not been created yet for the cases where there are different goas and different evauation criteria, or if they are changing. There were made some attempts to deveop e-government assessment frameworks or modes, but most of these modes were focused ony on a certain perspective. For exampe, stage modes that are focused on evauation of maturity, sophistication and some other indicators (Mukabeta Maumbe et a., 2008; Kachwamba & Hussein, 2009; Lee, 2010; E-Qawasmeh, 2011; Ifinedo & Singh, 2011), a marketing mode - on marketing indicators (Steyaert, 2004), a comprehensive framework - for the assessment the vaue of project post-impementation (Esteves & Joseph, 2008), a conceptua mode - for measuring user satisfaction (Verdegem & Vereye, 2009), conceptua framework - for evauation of pubic vaue (Karunasena & Deng, 2012), an integrated indicator mode - for evauation of e- government poicies (Stanimirovic & Vintar, 2013). Athough efforts are being made to create different e-government evauation modes at present there is no methodoogy that aows for fexibe and comparative measurement of the phenomenon of e-government in a comprehensive and integra way (Luna- Reyes et a., 2012). In this research, we aim to combine the anayzed modes into a conceptua mode that coud provide a common framework for future research in the area of e-government assessment and a common point of reference in this area. The aim of this paper is to summarize the efforts and offer a reference mode for e- government monitoring, evauation and benchmarking, which is not associated with a particuar e-government definition, and/or mode, evauation purposes, dimensions or criteria, but which coud be used universay and be fexibe enough to adapt to each particuar case. In other words, the proposed mode is not depended on the specific objectives and evauation perspectives. The research method is based on inductive approach that consist of the empirica study of the iterature on individua e- government evauation methods and modes, observation and practica experience. Reference Mode The proposed reference mode for e-government evauation is buit using entity-reationship modeing technique and is presented in Figure 1. Figure 1. Reference mode for e-government assessment. The mode is constructed around the key eements - entities: E-government (service) soution: soution (technica, organizationa, etc.), which impements e-government or e- government services. Requirement: any requirement to be met by the e-government (service) soution. Objective: any objective to which the evauation is directed. Dimension: any intrinsic aspect that is used for e-government evauation. Criterion: any criterion (indicator, variabe) which is used to form the dimension.

4 Metric: a rea-vaued function that is used to measure the evauation. Type: any type of reated entity Requirement or Criterion. E-government (service) soution in our context represents the e-government modes of a very genera form or represents ony a separate part that concentrates on some determinants such as e-government service issues ony. Though there are many concepts mentioned in different definitions of e-government the service is a key concept which can be found in most cases. That's why most of evauation methods of e-government are imited to the assessment of e-government services ony. Before the construction of e-government soution, we have to define what does e-government mean and the key characteristics that wi be reaized in this soution. As e-government is a argey amorphous concept with different meanings for different peope (Azab, 2009) there are mutipe definitions of e-government among authors (Jansen, 2005; Yidiz, 2007; Mukabeta Maumbe et a., 2008). Therefore, it coud be a reason that a common mode that coud fit for every e-government case does not exist though there were some attempts to fi in this gap by creating hoistic reference framework for integrated modeing of e- government services (Wimmer, 2002), or a generic mode based on paradigm that the pubic administration is composed of an unstructured network of entities that exchange eectronic requests in order to deiver services (Dias & Rafae, 2007). An important number of authors describe e-government service modes of the revoutionary approach to the dynamic phenomenon, which describes the stages of e-government in terms of their degree of technoogica and organizationa sophistication, from deveoping a web page to integrating government systems behind the web interface (Hassan et a., 2011). Whie there is increasing adoption of e-governments the objectives and specific requirements of impementations differs. Therefore, the soutions shoud be designed individuay for every e-government case. Like in the cases of the definitions of e-government the soution modes may vary from the very generic that focus just to the use of ICT by the government, to the more specific ike enhancing onine access and deivery of information and services through the Internet (Rorissa et a., 2011) by using different eectronic media or other digita means. And though most impementations activities focus on service deivery concerns with itte emphasis on rea transformation of the services themseves or the processes associated with their deivery these other issues of transformation initiative ike effective constituent reationship management (West, 2004), improvement of reated interna processes (Mukabeta Maumbe et a., 2008), its administration, rues, reguations and frameworks set out to carry out service deivery and to coordinate, communicate and integrate processes within itsef (Amarabeh & AbuAi, 2010), as we as transparency, accountabiity (Frissen et a., 2007), openness and coaboration (European Commission, 2013a) and other issues coud aso be incorporated into the e-government modes. Each e-government (service) soution wi meet the reevant requirements (see reation R1 in fig. 1). Most often they are expressed by the contract-stye requirement ists (Wimmer, 2001), but aso may be and other, more compex forms of requirements. Various e-government service requirements studies have come up with simiar requirements that are abe to form a generic set of requirements but for each e-government service domain they shoud be expanded with the particuar specific requirements (van Vesen et a., 2008). Therefore the requirements may be cassified by one or more types (see R2 in fig. 1) that wi separate the genera requirements from the specific. An exampe of requirement separation by type is presented next. There are four major areas of e-government deveopment: government-to-customer (G2C), government-tobusiness (G2B), government-to-government (G2G), and government-to-empoyee (G2E) (Siau & Long, 2009). G2C and G2E invove interaction and cooperation between government and individuas, whie G2B and G2G dea with the reationship between government and organizations. Moreover, G2C and G2B invove externa interaction and coaboration between government and outside institutes, such as individua citizens and businesses; whie G2E and G2G invove the interna interaction and cooperation between governments and their empoyees, as we as between governments at different eves and distributed ocations. As these areas have different objectives and activities, therefore different requirements are specific to each of them. One of the drawbacks of the evauation is that they do not differentiate between different eves of e-government services. The impementation of e-government services can take various forms ranging from a singe website that provide static information (e.g. contact information: address, teephone and fax numbers, emai address, etc.) to an interactive, consoidated gateway to integrated services at a eves of government, from oca to federa/nationa (Rorissa et a., 2011) that are fuservice portas (e.g. highy interactive). For this purpose severa cassification for e-government deveopment have been proposed, for exampe, according to e-service types: informationa, interactive/transactiona and personaized (Osman et a., 2011; Rorissa et a., 2011; Tsohou et a., 2013). Here we have another exampe, when type categorizes e- government services and requirements according to the phases of e-government deveopment: pubishing, interacting, and transacting (Kumar et a., 2007). On the other hand, the requirements themseves may create different eves of hierarchica structures (see R3 in fig. 1). For instance, they may be grouped into Functiona Requirements and Technica Requirements or other form of more compicated structure (Lenk, 2002). There may have any number of eves of the hierarchy, or may incude any number of hierarchies. Types themseves may be aso grouped together to form any number of eves in the hierarchy (see R4 in fig. 1) (e.g. Type of the service coud be a super-type for the Service interaction mode type). It is important to know if the requirements are met in a particuar soution, therefore they shoud be vaued or measured. Requirements may be measured by one or more metrics (see R5 in fig. 1). For instance, metrics coud be responses to questions that woud be incuded in some questionnaire (Luna-Reyes et a., 2012). A set of metrics combining a questionnaire may be used as a too to gather information for the evauation if the requirement is met for the particuar case. For this purpose organization must provide the appropriate responses to the questionnaire. The answers to the questions may be booean (Yes-1 or No-0) (e.g. Is this service provided on-ine? (Y/N) ), incrementa measurement scae (e.g. a score may range from 1 to 10) (Vadés et a., 2011), or any other actua (numerica) information (e.g. How many service transactions were provided on-ine? ). As it was mentioned above the e-government evauation may be focused to different objectives. Depending on the evauation objective it may be distinguished one or more dimensions (areas, determinants) of evauation and each dimension may express one or more evauation objective (see R6 in fig. 1). For instance, for e-government assessment Jansen (2005) identified three major dimensions: E-democracy, E-service and Eadministration ; Layne and Lee (2001) used two dimensions: Integration and Technoogy & Organiztiona Compexity ; Luna-Reyes et a. (2012) introduced three dimensions that are based on genera areas of measurement framework of e- government: Characteristics, Determinants and Resuts ;

5 Pina et a. (2009) assessed the eve of e-government deveopments using four dimensions: Transparency, Interactivity, Usabiity and Web site maturity. Dimensions themseves may create different eves of hierarchica structures (see R7 in fig. 1). For instance, the structure of assessment mode consists of two hierarchica eves of the dimension: Leverage Domains and Key Domain Areas (Vadés et a., 2011). Each dimension is evauated by using one or more criterion (critica variabe, indicator) (e.g. Existence of Information, Quaity of Information (Luna-Reyes et a., 2012)) and each criterion may be used to evauate one or more different dimensions (see R8 in fig. 1). For instance, dimension IT Architecture consists of 5 criteria: Security, Appication, Data, Technoogy and Network (Vadés et a., 2011). Criteria themseves may be grouped and form any number of eves of hierarchica structures (see R9 in fig. 1). For instance, criteria may be cassified into three groups: the technica issue group, the economic issue group, and the socia issue group (Ashawi & Aawany, 2009). Or it may be any other form of more compicated structure (Lenk, 2002). Furthermore, ike in case of requirements, the criteria may be cassified by one or more types (see R10 in fig. 1) and their seection may depend on the e-government (service) soution to be evauated (see R11 in fig. 1). E-government (service) soution assessment is essentiay carried out by the evauation of each criterion by one or more metrics (see R12 in fig. 1). Like in case of requirements, most cases each criterion conform to one metric ony but it is possibe to measure the criteria using more than one metric; or the same metric may be used to measure different criteria. For instance, Osman et a. (2011) used 49 questions as metrics to assess 4 criteria where were used the same questions for the measurement of different criteria. Furthermore, metrics themseves may form any number of eves of hierarchica structures (see R13 in fig. 1). For instance, Gupta & Jana (2003) used 6 eve metrics hierarchy for measuring performance of e- government. Criteria in this case must be examined according to the eve of measure that is appicabe in a specific context. Metrics associated with each criterion as we as criteria and dimensions may be seected on the basis of the iterature review or any other resource ike evauation reports or experts opinions on this area, for exampe (Steyaert, 2004; Jansen, 2005; Osman et a., 2011; Luna-Reyes et a., 2012; Concha et a., 2012; Stanimirovic & Vintar, 2013; Osman et a., 2013; Tsohou et a., 2013). Each criterion or dimension may be cacuated according to some rues by using metrics and other variabes. There may be different forms of rues expressed by simpe formuas or structures that are more compicated. Next formuas sampes iustrate the widespread cases for cacuation of criteria and dimensions. They bear a sufficienty broad cass of evauation modes, starting with the simpest aready mentioned above e-government stage modes and ending with the mutidimensiona hierarchica evauation modes (for instance, Vadés et a., 2011; Concha et a., 2012; Luna-Reyes et a., 2012). However, for such the modes it does often occur when the evauation formuas are not presented at a (for instance, Iribarren et a., 2008; Stanimirovic & Vintar, 2013; Osman et a., 2013). For computing of criterion c i (i=1, M, M number of evauation criteria) a weighted average evauation formua (1) is often used: m i where c i = Σ γ ij (v j / v j max ) (1) j=1 m i the number of metrics that are used for the evauation of criterion c i; γ ij weight for the metric j of criterion i on condition - m i Σ γ ij = 1, for every i; j=1 v j evauated vaue for the metric j; max v j the maximum potentia vaue for the metric j according to the chosen measurement scae. Each metric vaue v j is assessed by the chosen measurement scae, which depends on the question structure in the evauation questionnaire and the answer vaue or experts opinion: At issue in need of ogica answer "Yes" or "No" v j can obtain the appropriate vaue of "1" or "0". In this case, the maximum possibe vaue of the metric v max j =1. In case of evauation of incrementa measurement scae the answers can acquire numerica vaues from the chosen measurement scae according to pre-defined assessment condition for each scae. For instance, for the assessment of service maturity may be subject to a 4-stage mode, in which v j can gain vaue from the set v j {1, max 2, 3, 4} (v j =4), depending what conditions of the maturity eve description meet the vaued features of the service: "1" - "Informative", "2" - "Simpe Transactions", "3" - "Compex Transactions", "4" - "Integrate (Concha et a., 2012). In case of actua quantitive evauation the measure answers may obtain any of the numerica vaues from the interva [0, v j max ], where v j max the maximum possibe vaue of the metrics. In the simpest case of formua (1) the weights are equa for a metrics - γ ij = 1/m i, j=1,, m i, but can be used different weighting vaues that can be determined, for exampe, in accordance with internationa practice, or expert opinion (Vadés et a., 2011). However, in some cases for the evauation of one criterion it can be appied more compex formuas, for instance, such as the simutaneous combination of different properties - maturity and usage of the service operations (Ostasius, 2012). For evauation of any eve of the hierarchy of dimensions may be used aggregation formua (2): n k -1 D k = Σ ik D i (2) i=1 where hierarchy eve of dimensions: = 1,2,...; D k the aggregate evauation of the dimension k for the hierarchy eve ; -1 D i evauation of the dimension i for the hierarchy eve -1; 0 D i = c i, for each i, i = 1,..., M, M - number of evauation criteria; n k number of dimensions that are used in the cacuation of the aggregated dimension D k1 ; ik evauation weight of dimension i of eve on condition n k Σ ik = 1, for every k and. i=1 Weights ik can be chosen in the same way as in the case of the criteria. Formua (2) can aso be used in the cacuation of aggregated estimates of criteria and/or metrics. Concusions This study contributes to the area reated with e-government assessment. Despite the increased interest in e-government evauation, it appears that iterature does not offer much studies

6 of reusabe mutidimensiona integrated modes for e- government evauation. Furthermore, they are mosty focused on some particuar evauation purposes or are imited to some specific area of the assessment because it depends on the specifics of each country, their differences in economic, cutura, educationa issues, or the specific of poicies and requirements concerning e-government and services in the organization itsef. In the paper we have examined and summarized the efforts and modes proposed in the iterature and offered a reference mode as a reusabe conceptua mode for e-government assessment. This mode is not associated with any particuar e- government definition and/or mode, evauation purposes, dimensions or criteria. We identified and presented the main eements of the mode entities (e-government soution, objective, dimension, criterion, metric, requirement and type), and estabished reationships between them. Moreover, to the best of our knowedge, this is the first appication of entity reation modeing technique in the e-government fied, athough it has been widey appied in other fieds such as system or data modeing. We aso presented the main generaized formuas of most commony occuring cacuations of the evauation criteria and dimensions. The practica impications that derive from the proposed reference mode are that there is no need to create a new evauation mode for every specific case from scratch. We beieve that the proposed mode wi cover a wide range of evauation modes, so it can be used in designing specific modes for e-goverment monitoring, evauation or benchmarking. An evauation reference mode corresponds to an existing off-theshef-soution, therefore, what you need is, according to the evauation object and the objectives, to seect the parameters dimensions, criteria and metrics for your specific evauation mode. Additionay, this mode can be used for evauation of any service or soution that is not necessariy concerned with e- government. Future work wi be focused on the vaidation of the reference mode by constructing in practise the specific monitoring, evauation and benchmarking modes for various e- government cases and services. References Affisco, J. F., & Soiman, K. S. (2006). E-government: a strategic operations management framework for service deivery. Business Process Management Journa, 12(1), A-Khatib, H. (2009). A citizen oriented e-government maturity mode. Brune University, Londres. Amarabeh, T., & AbuAi, A. (2010). A genera framework for e-government: definition maturity chaenges, opportunities, and success. European Journa of Scientific Research, 39(1), Ashawi, S., & Aawany, H. (2009). E government evauation: Citizen's perspective in deveoping countries. Information Technoogy for Deveopment,15(3), Assar, S., Boughzaa, I., & Boydens, I. (2010). Practica studies in e-government: Best practices from around the word. Springer. Azab, N. A. (2009). Assessing Eectronic Government Readiness of Pubic Organizations. Communications of the IBIMA, 8, Bannister, F. (2007). The curse of the benchmark: an assessment of the vaidity and vaue of e-government comparisons. Internationa Review of Administrative Sciences, 73(2), Capgemini, IDC, Sogeti, IS-practice and Indigov, RAND Europe, & Danish Technoogica Institute (2014). egovernment Benchmark. Deivering on the European Advantage? How European governments can and shoud benefit from innovative pubic services. FINAL INSIGHT REPORT. Avaiabe from internet: Concha, G., Astudio, H., Porrua, M., & Pimenta, C. (2012). E-Government procurement observatory, maturity mode and eary measurements. Government Information Quartery, 29, S43-S50. Coursey, D., & Norris, D. F. (2008). Modes of e government: Are they correct? An empirica assessment. Pubic Administration Review, 68(3), Dias, G. P., & Rafae, J. A. (2007). A simpe mode and a distributed architecture for reaizing one-stop e-government. Eectronic Commerce Research and Appications, 6(1), E-Qawasmeh, E. (2011). Assessment of the Jordanian E-Government: An Empirica Study. Journa of Emerging Trends in Engineering and Appied Sciences (JETEAS), 2(4), Esteves, J., & Joseph, R. C. (2008). A comprehensive framework for the assessment of egovernment projects. Government information quartery, 25(1), European Commission. (2013a). A vision for pubic services. Digita Agenda for Europe. Avaiabe from internet: European Commission. (2013b). Interoperabiity Maturity Mode. ISA. Avaiabe from internet: Fath-Aah, A., Cheikhi, L., A-Qutaish, R. E., & Idri, A. (2014, May). E-GOVERNMENT MATURITY MODELS: A COMPARATIVE STUDY. In Internationa Journa of Software Engineering & Appications (IJSEA), Vo.5, No.3. Friendand, C., & Gross, T. (2010). Measuring the Pubic Vaue of e-government: Methodoogy of a South African Case Study. In Proceedings of the 1 st Africa 2010 Conference, IIMC Internationa Information Management Corporation Frissen, V., Miard, J., Huijboom, N., Iversen, J. S., Koo, L., Kotterink, B.,... & van der Duin, P. (2007). The future of egovernment: An exporation of ICT-driven modes of egovernment for the EU in D. Osimo, D. Zinnbauer and A. Bianchi, Joint Research Centre. Gomez-Reynoso, J. M., & Sandova-Amazan, R. (2013). The Adoption of e-government Services in Mexico: A Citizens Perception Anaysis. American Journa of Industria and Business Management, 3(06), 12.

7 Griffin, D., & Hapin, E. (2005). An exporatory evauation of UK oca e-government from an accountabiity perspective. The Eectronic Journa of e-government, 3(1), Gupta, M. P., & Jana, D. (2003). E-government evauation: A framework and case study. Government Information Quartery, 20(4), Hassan, H. S., Shehab, E., & Peppard, J. (2011). Recent advances in e-service in the pubic sector: State-of-the-art and future trends. Business Process Management Journa, 17(3), Ifinedo, P., & Singh, M. (2011). Determinants of egovernment maturity in the transition economies of Centra and Eastern Europe. Eectronic Journa of e-government, 9(2), Irani, Z., A-Sebie, M., & Eiman, T. (2006). Transaction stage of e-government systems: identification of its ocation and importance. In System Sciences, HICSS'06. Proceedings of the 39th Annua Hawaii Internationa Conference on (Vo. 4, pp. 82c-82c). IEEE. Iribarren, M., Concha, G., Vades, G., Soar, M., Viarroe, M. T., Gutiérrez, P., & Vásquez, Á. (2008). Capabiity maturity framework for egovernment: A muti-dimensiona mode and assessing too. In Eectronic Government (pp ). Springer Berin Heideberg. Yidiz, M. (2007). E-government research: Reviewing the iterature, imitations, and ways forward. Government Information Quartery, 24(3), Jansen, A. (2005). Assessing E-government progress why and what. NOKOBIT, Kachwamba, M., & Hussein, A. (2009). Determinants of e-government Maturity: Do Organizationa Specific Factors Matter?". Journa of US-China Pubic Administration, 6(7), 1-8. Kaampokis, E., Tambouris, E., & Tarabanis, K. (2011). Open government data: A stage mode. In Eectronic Government (pp ). Springer Berin Heideberg. Karunasena, K., & Deng, H. (2012). Critica factors for evauating the pubic vaue of e-government in Sri Lanka. Government Information Quartery, 29(1), Kievink, B., & Janssen, M. (2009). Reaizing joined-up government Dynamic capabiities and stage modes for transformation. Government Information Quartery, 26(2), Krenner, J. (2002). Refections on the requirements gathering in an one-stop government project. In Eectronic Government (pp ). Springer Berin Heideberg. Kumar, V., Mukerji, B., Butt, I., & Persaud, A. (2007). Factors for successfu e-government adoption: a conceptua framework. The eectronic journa of e-government, 5(1), Kunstej, M., & Vintar, M. (2004). Evauating the progress of e-government deveopment: A critica anaysis. Information Poity, 9(3), Layne, K., & Lee, J. (2001). Deveoping fuy functiona E-government: A four stage mode. Government information quartery, 18(2), Lam, W. (2005). Barriers to e-government integration. Journa of Enterprise Information Management, 18(5), Lee, J. (2010). 10 year retrospect on stage modes of e-government: A quaitative meta-synthesis. Government Information Quartery, 27(3), Lenk, K. (2002). Eectronic Service Deivery-A driver of pubic sector modernisation. Information Poity, 7(2), Luna-Reyes, L. F., Gi-Garcia, J. R., & Romero, G. (2012). Towards a mutidimensiona mode for evauating eectronic government: Proposing a more comprehensive and integrative perspective. Government Information Quartery,29(3), Mohammad, H., Amarabeh, T., & Ai, A. A. (2009). E-government in Jordan. European Journa of Scientific Research, 35(2), Moon, M. J. (2002). The Evoution of review, 62(4), E Government among Municipaities: Rhetoric or Reaity? Pubic administration Mukabeta Maumbe, B., Owei, V., & Aexander, H. (2008). Questioning the pace and pathway of e-government deveopment in Africa: A case study of South Africa's Cape Gateway project. Government Information Quartery, 25(4), Omari, A. A., & Omari, H. A. (2006). E-government readiness assessment mode. Journa of Computer Science, 2(11), 841. Osman, I. H., Anouze, A., Irani, Z., Lee, H., Bacı, A., Medeni, T., & Weerakkody, V. (2011). A new COBRAS framework to evauate e-government services: a citizen centric perspective. In Proceedings of tgov Workshop (Vo. 11, pp ). Osman, I. H., Anouze, A. L., Azad, B., Daouk, L., Zabith, F., Hindi, N. M., Irani, Z., Lee, H., & Weerakkody, V. (2013). The eicitation of key performance indicators of e-government providers: A bottom-up approach. European, Mediterranean & Midde Eastern Conference on Information Systems. Ostasius, E. (2012). Assessing Maturity for e-government Services. In Coaborative Networks in the Internet of Services (pp ). Springer Berin Heideberg. Pina, V., Torres, L., & Royo, S. (2009). E-government evoution in EU oca governments: a comparative perspective. Onine Information Review, 33(6), Rorissa, A., Demissie, D., & Pardo, T. (2011). Benchmarking e-government: A comparison of frameworks for computing e- government index and ranking. Government Information Quartery, 28(3),

8 Rowey, J., (2011). e-government stakehoders Who are they and what do they want? Internationa journa of information management [ ] Rowey, J yr:2011 vo:31 iss:1 pg:53 - Esevier Sandova-Amazan, R., Leyva, N. K. S., & Gi-Garcia, J. R. (2013). Maturity and evoution of e-government portas in centra America: a three-year assessment In Proceedings of the 7th Internationa Conference on Theory and Practice of Eectronic Governance (pp ). ACM. Siau, K., & Long, Y. (2005). Synthesizing e-government stage modes a meta-synthesis based on meta-ethnography approach. Industria Management & Data Systems, 105(4), Siau, K., & Long, Y. (2009). Factors Impacting E-government Deveopment. Journa of Computer Information Systems, 50(1). Stanimirovic, D., Leben, A., Jukic, T., & Vintar, M. (2010) Conceptuaization of an integrated indicator mode for evauation of e- government poicies. University of Ljubjana, Facuty of Administration, Institute for informatization of administration, Sovenia. Avaiabe from internet: Stanimirovic, D., & Vintar, M. (2013). Conceptuaization of an Integrated Indicator Mode for the Evauation of e-government Poicies. Eectronic Journa of e-government, 11(2). Steyaert, J. C. (2004). Measuring the performance of eectronic government services. Information & Management, 41(3), Tripathi, R., & Gupta, M. P. (2014). Evoution of government portas in India: mapping over stage modes. Journa of Enterprise Information Management,27(4), 7-7. Tsohou, A., Lee, H., Irani, Z., Weerakkody, V., Osman, I. H., Anouze, A. L., & Medeni, T. (2013). Proposing a reference process mode for the citizen-centric evauation of e-government services. Transforming Government: Peope, Process and Poicy, 7(2), United Nations. Department of Economic. (2010). United Nations e-government survey 2010: Leveraging e-government at a time of financia and economic crisis (Vo. 2). United Nations Pubications. Vadés, G., Soar, M., Astudio, H., Iribarren, M., Concha, G., & Visconti, M. (2011). Conception, deveopment and impementation of an e-government maturity mode in pubic agencies. Government Information Quartery, 28(2), van Vesen, L., van der Geest, T., ter Hedde, M., & Derks, W. (2008). Engineering user requirements for e-government services: A Dutch case study. In Eectronic Government (pp ). Springer Berin Heideberg. Verdegem, P., & Vereye, G. (2009). User-centered E-Government in practice: A comprehensive mode for measuring user satisfaction. Government Information Quartery, 26(3), West, D. M. (2004). E Government and the Transformation of Service Deivery and Citizen Attitudes. Pubic administration review, 64(1), Wimmer, M.A. (2001). European Deveopment towards Onine One-stop Government: The egov Project. In Proceedings of the ICE2001 Conference, Vienna. Wimmer, M. A. (2002). A European perspective toward onine one-stop government: the egov project. Eectronic commerce research and appications,1(1), Wimmer, M. A., & Tambouris, E. (2002). Onine one-stop government. In Information Systems (pp ). Springer US. Zarei, B., Ghapanchi, A., & Sattary, B. (2008). Toward nationa e-government deveopment modes for deveoping countries: A nine-stage mode. The internationa information & ibrary review, 40(3), The artice has been reviewed. Received in September, 2014; accepted in June, 2015.