User Satisfaction with E-government Websites: An Australian Experience

User Satisfaction with E-government Websites: An Australian Experience Abstract Many nations worldwide are continuing to invest in electronic government (e-government) initiatives that now demand an investigation into how well users perceive the success of these initiatives. Currently, there exists a dearth of empirical evaluation of e-government success in particular for the Australian context. Our research project, reported in this paper, is concerned with developing a conceptual model on user satisfaction with e-government websites and reports and initial empirical evaluation of the model with regard to the e-government website developed by the Australian Department of Immigration and Citizenship. The findings suggest that user satisfaction incorporates four factors and is positively related to users overall trust in e-government and negatively related to their anxiety towards using e-government website. These findings have implications for theory and practice alike. Keywords: E-government, user satisfaction, e-government success, e-government website, Australia 1.0 Introduction The term e-government generally refers to the application of an e-commerce model, in which a public institution buys or provides goods, services, or information to businesses, individual citizens, their employees, and other government bodies (Turban et al, 2010). A rich body of studies is emerging that report initial e-government adoption decisions, implementation strategy and hurdles, and citizens awareness of e-government initiatives (Welch, et al. 2005; Rao, et al. 2008; Gauld, et al. 2010; Jinhua et al. 2010; Byun and Finnie 2011, Rana et al., 2014, Sanyal et al. 2014). In contrast, relatively little has so far been reported on post-implementation aspects of e-government initiatives. Success, which is often measured in terms of user satisfaction, represents an important post-implementation outcome of e- government initiatives. Due to enormous investments already made in e-government initiatives, it is now time to measure and scrutinise their success. In response to this knowledge gap, the research project (described in this paper) was initiated for the Australian context. The specific aim of our research is to find out what constitutes user satisfaction with Australian e-government websites and how such satisfaction is related to users trust and anxiety towards e-government websites. Australia was chosen as an appropriate context because it has an active immigration program and has many renowned universities that attract a large number of foreign students each year. To facilitate prompt delivery of services for Australian citizens, newly arrived immigrants, and foreign students, the Australian government has invested heavily in delivering immigration services (i.e., visas and passport renewals) 1 P a g e

online via the website of Immigration and Citizenship Department. Hence, a very large user community of e-government services exists in Australia. 2.0 Background Literature In the broader IS/IT discipline, success of an IT applications is generally evaluated in terms of user satisfaction (Doll and Torkzadeh, 1988). The term user satisfaction generally refers to the positive perceptions expressed by computer application users (Al-Ghtani and King, 1999). Although the notion of user satisfaction has originated from end-user computing (EUC) context, it has been extended for situations beyond intra-organisational context (e.g. e-business, e-government). A rigorous literature review on e-government satisfaction was conducted. A number of studies were selected based on Google Scholar search engines and multiple online databases using e-government, satisfaction and success as the main search keywords. The list of studies is by no means exhaustive but is adequate to provide a general picture of user satisfaction concept discussed in the e-government literature. Several important observations deduced from the review are now highlighted. First, some authors (e.g. Jinhua et al. 2010) have offered only theoretical models with regard to measuring satisfaction with e-government. However, these models in general lack an empirical evaluation. Second, most studies on e-government satisfaction heavily draw on the End-user Computing Satisfaction (EUCS) model that was developed for intra-organisational IT applications (Doll and Torkzadeh, 1988). However, adjustments and adaptations were made by some scholars to suit the model for e-government context (e.g. Mohamed, Hussain, and Hussein 2009). Third, a range of factors were considered by various e-government scholars to represent user satisfaction with e-government; however there is no agreement among them on the number of factors. Fourth, a few scholars (e.g. Welch et al. 2005) have attempted to relate e- government user satisfaction with other variables of research interest (e.g. anxiety, trust), but the findings are inconclusive. Fifth, survey appears to be the most dominant approach for empirical evaluation of e-government satisfaction model. Survey samples vary widely ranging from 176 (Abdinnour et al. 2005) to even 5590 (Verdegem and Verleye, 2009). Finally, some scholars have evaluated user satisfaction with a particular e-government website. For example, Horan and Abhichandani (2006) measured satisfaction with an online advanced transportation information systems (ATIS) developed by a government agency for public to use. In contrast, others were silent about the type of e-government website for which user satisfaction was measured. In summary, despite the growth in the e-government literature, the association between user satisfaction with e-government websites and users anxiety and trust in e-government is still not adequately understood. In particular, 2 P a g e

relatively less in known on how such relationship manifests itself for the Australian e-government websites context. Our research reported in this paper seeks to address this gap. 3.0 Research Model Drawing on a rigorous review of the identified studies (mentioned earlier), a total of 67 factors were identified that could potentially constitute user satisfaction. It is not however practical to include all these factors because of the difficulty of operationalising such a large model. The following criteria were thus used to guide the short-listing process: a) factors that have similar meanings were removed, thus reducing the number of factors from 67 to 48, and b) those factors that were more frequently mentioned and, at the same time, received empirical support were retained. As a result, a set of nine factors were considered relevant and were thus included in our research model (Figure 1). Accuracy of information Ease of use User trust in e-government Customisation H2 Interactivity Security & Privacy Sufficiency of contents H1 User satisfaction with e-government website Timeliness Transparency Utility H3 User anxiety towards using e-government website Figure 1: The research model The model suggests that user satisfaction with e-government consists of nine factors. It further proposes that increased user satisfaction with e-government websites should lead to increased user trust in e-government and reduced anxiety toward using e-government websites. Due to page limitations, it is not possible for us to include a detailed discussion explaining how each of the nine factors influence user satisfaction. However, the literature sources and the number of items used to measure these factors are included in Table 1. 3 P a g e

Table 1: A list of items used to operationalise the factors included in the research model Research variables No. of Literature sources items Accuracy 4 Mohamed et al., 2009, Doll and Torkzadeh, 1988 Content 4 Mohamed et al., 2009, Doll and Torkzadeh, 1988 Customisation 3 Horan and Abhichandani, 2006, Abhichandani and Horan, 2006) Ease of use 4 Doll and Torkzadeh, 1988, Horan and Abhichandani, 2006 Interactivity 3 Welch et al., 2005 Timeliness 3 Doll and Torkzadeh, 1988, Mohamed et al., 2009 Security and privacy 3 Welch et al., 2005 Transparency 2 Welch et al., 2005 Utility 4 Horan and Abhichandani, 2006, Abhichandani et al., 2006 Trust 3 Alsaghier et al., 2011 Anxiety 4 Venkatesh et al. ( 2003) As indicated in Figure 1, three hypotheses are proposed in our model. Hypothesis H1 is concerned with nine factors contributing to user satisfaction with e-government websites. User satisfaction is represented as a latent variable that is measured in terms of the nine factors shown in the left-hand side of the model. Scholars address satisfaction with online systems (Fisher and Kingma (2001; Mohamed et al. (2009), and intra-organisational IT applications (Doll and Torkzaddch (1988) alike reported accuracy to be a major factor contributing to user satisfaction. Likewise, there is a rich body of EUC, e- business (Pikkarainen, et. al., 2006; Yang and Fang 2004, Chen 2010), e-government (Mohamed et al. 2009) literatures which supports the view that ease of use is a major contributor towards the formation of user satisfaction with online systems. In several other studies, customisation has been found to be a key factor that influences user satisfaction (Wang, 2003; Horan & Abhichandan, 2006). In another study, Welch et al. (2005) examined citizens satisfaction with e-government and trust in government and concluded that interactivity was one of the important factors affecting user satisfaction. For e- government context, privacy, security, and transparency were further reported to affect user satisfaction. For example, according to Verdegem and Hauttekeete (2007), privacy is significant for measuring user satisfaction. Likewise, security was considered an important factor to determine the satisfaction of users for e-government utlisation (Verdegem & Hauttekeete, 2007). Transparency too was found to be a significant factor that directly affects satisfaction of e-government users (Welch et al., 2005). In a study of e-government for the Malaysian context, Mohamed et al. (2009) argued that timeliness was a significant factor for measuring user satisfaction. Finally, according to Horan et al. (2006), utility is one of the important factors affecting user satisfaction with e-government services. Based on these observations drawn from various literature sources, we now propose the first hypothesis: H1: The nine factors together constitute user satisfaction with e-government systems. 4 P a g e

The research model also hypothesises the existence of a relationship between user satisfaction and their trust in e-government. This is because trust in e-government is known to lead citizens to engage in e- government. The relationship between trust and satisfaction has been further examined by Christensen and Laegreid (2005), who reported that increased service satisfaction should led to increased trust. In a recent study, Alsaghier et al. (2011) examined the factors affecting the citizens trust in e-government and found that trust and satisfaction with e-government among citizens are related. Hence, the following hypothesis is proposed: H2: User satisfaction with e-government is positively related to their trust in government. Lin and Yu (2006) defined anxiety as an affective feeling associated with computer use. According to them, there is a negative relationship between Internet usage and people's anxiety. In another study, Acar (2008) examined users behaviors in online social networking based on Facebook usage and found a negative relationship between anxiety and stranger users of the network. Existing e-government literature however does not report much how user satisfaction with e-government relates to users' anxiety toward using e-government. It is argued that when using e-government services satisfied users should experience less anxiety toward using e-services delivered through e-government websites. This leads to the following hypothesis: H3: User satisfaction with e-government is negatively related to their anxiety of using e- government websites. 4 RESEARCH DESIGN Our research is exploratory because the relationship between user satisfaction with e-government websites and their trust and anxiety towards e-government websites has not been adequately investigated. Moreover, we are concerned with theory building as we propose development of a preliminary model linking user satisfaction with their trust and anxiety. Hence, according to Yin (2003), an exploratory survey method is considered appropriate. A survey instrument consisting of 37 items (Table 1) was developed based on the critical literature analysis. This was improved using a qualitative evaluation involving three domain experts (two academics and an IT manager from a council), and a pilot test (involving five international Phd students who interact with the Australian immigration website for visa purpose). Their comments were analused using the guidelines given by Eklim and Rahim (2008) and consequently several items were dropped and a few items were reworded to improve readability; this resulted in 33 items. A survey questionnaire was then distributed to a convenience sample of 400 students and staff at a leading Australian university. A total of 190 completed 5 P a g e

questionnaires were received. This represents 34% response rate which according to Hikmet and Chen (2003) is considered acceptable. Survey data were analysed using SPSS software. 5 FINDINGS AND DISCUSSION Demographic characteristics: Out of 190 completed survey responses, 120 respondents acknowledged using e-government services offered through the website of the Department of Immigration and Citizenship, and thus they form the net sample size used in our study for data analysis. We observed that: a) most (77.5%) of the users are male respondents; b) except for one particular age group (18-23 years), all the remaining age groups are well represented; and c) most of the participants (72.5%) have a postgraduate degree. We further find that a majority of the participants (64.3%) visited the website between one and five times, and they have done so (66.7%) within the last six months. Statistical analysis: The Kaiser-Meyer-Olkin Measure of Sampling Adequacy was found to be 0.794, and Bartlett s Test of Sphericity results (Chi-Square 1099.025, df = 91, p =.000) were significant. Thus, there is evidence that we have an adequate minimum sample size and that the data we collected can be considered reliable for factor analysis (Pallant, 2005). The survey responses were then subjected to a principal component factor analysis. Varimax was chosen as the rotation method because survey data demonstrated high correlations among the extracted factors. Items were removed when (a) they did not load on any factors, and (b) they were cross loaded on more than one factor. Two commonly used decision rules were adopted to identify the factors underlying the construct: (a) an Eigenvalue of one as the cut-off value for extraction, and (b) factor loadings of less than 0.4 on all factors (Tojib and Sugianto, 2007). The iterative sequence of factor analysis and item deletion was applied, resulting in a final set of 14 items representing four distinct factors associated with the user satisfaction with e- government (Table 2). Items Table 2: Factor loadings of the retained items contributing to user satisfaction Utility (F1) Ease of use (F2) Customisation (F3) Website understandability (F4) Corrected Item-Total Correlation EaseofUse1.865.535 EaseofUse2.777.546 Accuracy3.715.569 Accuracy4.878.486 Customisation1.781.609 Customisation2.844.511 Customisation3.842.520 EaseofUse3.720.406 Interactivity2.634.511 Timelenss3.683.398 Utility1.903.696 6 P a g e

Utility2.897.689 Utility3.883.609 Utility4.869.688 Out of four factors, three include utility (F1), ease of use (F2) and customisation (F3). This is consistent with our research model. However, a new factor (F4) has emerged that we named as "website understandability" because there are three different items found from different factor, but they seem to have a somewhat similar meaning and can fall under the rubric of the e-government website understandability factor. We describe this new factor as a property of an e-government website that encourages users to better understand the operations of the website due to the presence of online support, expected screen changes, and automatic logoff function. Moreover, two items (e.g. Accuracy3, Accuracy4) were renamed because they in fact indicate characteristics of ease of use factor. Taken together, these four factors accounted for around 74.2% of the variance (Table 3). The significant loading of all 14 items (Table 2) on the single factors indicates unidimensionality. The fact that no item had multiple cross loadings provides support for the discriminate validity of the user satisfaction with e- government scale. As, only four of the nine factors included in our research model contributed to user satisfaction; hence hypothesis H1 is partially supported. Table 3: Results of factor analysis Factors Utility (F1) Ease of use (F2) Customization (F3) Website understandability (F4) Eigen values 5.674 2.243 1.453 1.029 Variance by individual 40.528 16.020 10.378 7.352 factor Cumulative variance 40.528 56.548 66.926 74.268 A correlation analysis was performed to evaluate the relationship between user satisfaction and their trust in e-government. The results (Pearson Correlation =.595*, p-value:.000) indicate the presence of a significantly positive correlation indicating a support for hypothesis H2. This finding is in a broad sense consistent with the views expressed in the literature. For example, Warkentin et al. (2001) and Alsaghier et al. (2011) suggest that trust in e-government and engagements in e-government are positively related. However, these findings are different from the observations of Goldfinch et al. (2009) who find the level of e-government use to have negative association with trust. Another round of Pearson correlation analysis was performed and the results (Pearson Correlation = -.2405*, p-value:.008) render support to the assertion that user satisfaction is significantly negatively correlated with their anxiety. As a result, hypothesis H3 too is supported. This observation is in line with the views 7 P a g e

expressed by some scholars with regard to online systems usage context. For instance, Acer (2008) reports a negative relationship between anxiety and using online services to communicate with strangers. Meuter et al. (2003) find that a negative relationship between individuals Internet use and their anxiety levels. 6. CONCLUSION Even though nine factors were identified from literature analysis that could potentially contribute to the formation of user satisfaction with e-government websites, our study empirically confirms that only four factors make up user satisfaction: utility, ease of use, customisation and website understandability. Out of these four factors, only one is new (Website understandability). User satisfaction with e- government websites was found to have a significant positive relationship with their trust in e- government websites and is significantly negatively correlated with their anxiety toward using e- government websites. Therefore, the lesson from our research is that to enhance the satisfaction of users with e-government websites, government agencies should concentrate on those four factors (i.e. utility, ease of use, customisation, and website understandability) that are perceived to have a significant effect on satisfaction. Furthermore, improved user satisfaction would help increase user trust and reduce their anxiety levels toward using e-government websites. Overall, the key contribution of our research is the identification of a new factor (i.e. website understandability) that has not been reported previously. For practice, our findings may guide the government agencies to improve their e- government websites by focusing on the four key factors. However, despite this contribution, there are a few limitations that could constrain our findings. First, although, the sample size (n=120) used in our study is satisfactory but we still advocate future studies to consider a larger sample size, which would help increase the generalisability of our results. Second, the participants in our survey are dominated by the staff and students of a single university. Further research is needed to replicate this study using corporate users (i.e., business managers, doctors, engineers) who may have different expectations from e-government services. This study has used a cross-sectional survey undertaken at a particular point in time. Hence, longitudinal studies are needed to find out if the four factor satisfaction construct is still valid over time and whether the direction of relationship between user satisfaction, trust, and anxiety remains the same over time. References Abhichandani, T., & Horan, T. A. (2006). Toward a new evaluation model of e-government satisfaction: results of structural equation modeling. Abhichandani, T., Horan, T. A., & Rayalu, R. (2005). EGOVSAT: Toward a Robust Measure of E- Government Service Satisfaction in Transportation. 8 P a g e

Acar, A. (2008). Antecedents and consequences of online social networking behavior: The case of Facebook. Journal of website promotion, 3(1), 62-83. Al-Gahtani, S. S., & King, M. (1999). Attitudes, satisfaction and usage: factors contributing to each in the acceptance of information technology. Behaviour & Information Technology, 18(4), 277-297. Alsaghier, H., Ford, M., Nguyen, A., & Hexel, R. (2011). Factors Affecting the Citizens Trust in E- Government. Byun, D. H., & Finnie, G. (2011). Evaluating usability, user satisfaction and intention to revisit for successful e-government websites. Electronic Government, an International Journal, 8(1), 1-19. Chen, C. W. (2010). Impact of quality antecedents on taxpayer satisfaction with online tax-filing systems-an empirical study. Information & Management. Christensen, T., & Lægreid, P. (2005). Trust in government: The relative importance of service satisfaction, political factors, and demography. Public Performance & Management Review, 28(4), 487-511. Doll, W. J., & Torkzadeh, G. (1988). The measurement of end-user computing satisfaction. Mis Quarterly, 259-274. Doll, W. J., Xia, W., & Torkzadeh, G. (1994). A confirmatory factor analysis of the end-user computing satisfaction index. Mis Quarterly, 18(4), 453-461. Eklim, S., & Rahim, M. M. (2008). A Qualitative Evaluation of an Instrument to Measure Organisational Motivations for Inter-Organisational Systems Adoption. International Journal of Business and Information, 3(1). Fisher, C. W., & Kingma, B. R. (2001). Criticality of data quality as exemplified in two disasters. Information & Management, 39(2), 109-116. Gauld, R., Goldfinch, S., & Horsburgh, S. (2010). Do they want it? Do they use it? The 'Demand- Side'of e-government in Australia and New Zealand. Government Information Quarterly, 27(2), 177-186. Goldfinch, S., Gauld, R., & Herbison, P. (2009). The Participation Divide? Political Participation, Trust in Government, and E government in Australia and New Zealand. Australian Journal of Public Administration, 68(3), 333-350. Hikmet, N., & Chen, S. K. (2003). An investigation into low mail survey response rates of information technology users in health care organizations. International journal of medical informatics, 72(1-3), 29-34. Horan, T. A., & Abhichandani, T. (2006). Evaluating user satisfaction in an e-government initiative: results of structural equation modeling and focus group discussions. Journal of Information Technology Management, 17(4), 33. Horan, T. A., Abhichandani, T., & Rayalu, R. (2006). Assessing user satisfaction of e-government services: development and testing of quality-in-use satisfaction with advanced traveler information systems (ATIS). Jinhua, Y., Yong, L., & Peng, Z. (2010). E-government Evaluation based on Citizen Satisfaction and its Implementation. Lin, C. H., & Yu, S. F. (2006). Consumer Adoption of the Internet as a Channel: The influence of driving and inhibiting factors. Journal of American Academy of Business, 9(2), 112-117. Meuter, M. L., Ostrom, A. L., Bitner, M. J., & Roundtree, R. (2003). The influence of technology anxiety on consumer use and experiences with self-service technologies. Journal of Business Research, 56(11), 899-906. Mohamed, N., Hussin, H., & Hussein, R. (2009). Measuring Users Satisfaction with Malaysia s Electronic Government Systems. Electronic Journal of e-government, 7(3), 283-294. Pallant, J. (2007). SPSS survival manual, 3rd: Sydney, Australia: Allen and Unwin. Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1988). SERVQUAL: A Multiple-Items Scale for Measuring Consumer Perceptions of Service Quality. Journal of Retailing, 64(1), 12-40. Pikkarainen, K., Pikkarainen, T., Karjaluoto, H., & Pahnila, S. (2006). The measurement of end-user computing satisfaction of online banking services: empirical evidence from Finland. International Journal of Bank Marketing, 24(3), 158-172. Rana, Nripendra P., et al. "Investigating success of an e-government initiative: Validation of an integrated IS success model." Information Systems Frontiers (2014): 1-16. Rao, H., Chai, S., Herath, T., & Park, I. (2008). Repeated Use of E-Gov Websites: A Satisfaction and Confidentiality Perspective. Sanyal, Manas Kumar, Sudhangsu Das, and Sajal Kanti Bhadra. "E-District Portal for District Administration in West Bengal, India: A Survey to Identify Important Factors towards Citizen's Satisfaction." Procedia Economics and Finance 11 (2014): 510-521. 9 P a g e

Tojib, D. R., & Sugianto, L. F. (2007). The development and empirical validation of the B2E portal user satisfaction (B2EPUS) scale. Journal of Organizational and End User Computing (JOEUC), 19(3), 43-63. Turban, E., King, D., & Lang, J. (2010). Introduction to Electronic Commerce: International Version: Pearson Education, Limited. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS quarterly, 425-478. Verdegem, P., & Hauttekeete, L. (2007). User centered e-government: measuring user satisfaction of online public services. Verdegem, P., & Verleye, G. (2009). User-centered E-Government in practice: A comprehensive model for measuring user satisfaction. Government Information Quarterly, 26(3), 487-497. Warkentin, M., Gefen, D., Pavlou, P. A., & Rose, G. M. (2001). Encouraging citizen adoption of e- government by building trust. Electronic Markets, 12(3), 157-162. Welch, E. W., Hinnant, C. C., & Moon, M. J. (2005). Linking citizen satisfaction with e-government and trust in government. Journal of Public Administration Research and Theory, 15(3), 371. Yang, Z., & Fang, X. (2004). Online service quality dimensions and their relationships with satisfaction: A content analysis of customer reviews of securities brokerage services. International Journal of Service Industry Management, 15(3), 302-326. Yin, R. K. (2003). Case Study Research: Design and Methods, volume 5 of Applied Social Research Methods Series: Sage Publications, Thousand Oaks, USA. 10 P a g e