ADOPTION OF TECHNOLOGY IN HIGHER EDUCATION: EXPANDING THE TECHNOLOGY ACCEPTANCE MODEL

Transcription

1 ADOPTION OF TECHNOLOGY IN HIGHER EDUCATION: EXPANDING THE TECHNOLOGY ACCEPTANCE MODEL Bøe, Tove, Buskerud Vestfold University College, School of Business Administration and Social Sciences, Postboks 164 Sentrum, NO-3502 Hønefoss, Abstract This study examines the teachers usage intention to use digital tools and services in higher education. Based on the Technology Acceptance Model and Agency Theory, teachers usage intention is theorized as a function of their perceived usefulness, perceived ease of use, and incentives. By combining these two theories, both personal and managerial perspectives are addressed, and prior TAM-research is extended by adding the concept of managerial incentives from Principal Agency Theory. Data from an online survey from teaching academics were collected, and the research model was validated by a structural equation modeling (SEM) approach. The results indicate a good fit with the data, and three out of four hypotheses are supported. Finally, theoretical and managerial implications are presented and discussed. 1 INTRODUCTION Universities and colleges are working hard to provide high quality education. Wide use of e- learning/information Communication Technology (ICT) in education is considered to be an important element in bringing these institutions toward that goal and facilitating improvements with respect to the quality of teaching and learning outcomes. Use of information technology-based tools in education have existed for years, but education today is still dominated by traditional teaching methods, such as face-toface lectures, tutorials, mentoring, and project work (Ørnes, Wilhelmsen, Breivik, & Solstad, 2011) even though the technology itself has become more sophisticated and has improved dramatically. One of the main questions will be whether teachers have the competence, motivation, and incentives needed to use the technology. Students today have more or less grown up with this new technology often they are described as digital natives. Digital natives are those who grew up with the internet and are completely comfortable using online sources to meet their information needs (Prensky, 2001). On the other hand, this might not be the situation for university teachers; the availability of ICT does not automatically secure implementation and usage intention. Identifying ways to motivate teachers to utilize ICT has turned out to be a challenge for university management (Ørnes, Wilhelmsen, Breivik, & Solstad, 2011). The technology provides teachers and students with new possibilities, not a ready to use resource (Sørebø, Halvari, Gulli, & Kristiansen, 2009, p. 1177). This study aims to understand how university management can stimulate increased technology usage intention among teachers. Such acceptance and increased use is asked for by the students and also by the Norwegian government. Within the Information Systems (IS) literature, there are several models or theories focused on explaining IS-use from a psychological as well as behavioral viewpoint. However, these models do not safeguard management s active influence over employees in securing organizational goals (Bhattacherjee, 1998). Hence, to address the issues related the implementation of IS, there is a need to incorporate perspectives that can explain the role of managerial influence, which is complementary to the traditional IS-literature. Especially, there is a need for theoretical frameworks that explain the impact of managerial tools from an economical viewpoint. Principal Agency Theory (PAT) is one the dominating approaches used when organizing relationships in organizations (Eisenhardt, 1989), and the concept of incentives is described as the core element of economics (Milgrom & Roberts, 1992). Incentives may be regarded as the most dominant motivators of human behavior within organizations (Eisenhardt, 1989) and can be used as an instrument to achieve goal congruency between the principal and the agent (Milgrom & Roberts, 1992).

2 The Technology Acceptance Model is widely used when explaining personal acceptance of information technology. The Technology Acceptance Model has proven to be a model that explains variances in users behavioral intentions related to IT adoption and usage across different contexts (Taylor & Todd, 1995). Combining the TAM with PAT might explain use both from a personal use perspective as well as from a managerial perspective. The study addresses this issue by expanding TAM with incentives from PAT in a common research model to explain personal IS usage in a university setting. The Technology Acceptance Model and PAT focus on different factors determining the adoption of ICT. The Technology Acceptance Model states that perceived usefulness and perceived ease of use are the main determinants for personal use, while PAT focuses on incentives as one of the main explanatory factors. There have been extensive empirical works within the TAM framework, while there are limited empirical works connecting PAT with the phenomenon in question. My review shows that only a few numbers of empirical studies have integrated TAM and PAT (Tao, Cheng, & Sun, 2009), and, to my knowledge, no studies have extended TAM with incentives in a university setting with a teacher perspective. Combining TAM with PAT will give a theoretical synthesis that explains personal use both from a personal-use perspective as well as from a managerial perspective. The study addresses this issue by integrate TAM and PAT derived variables in a common research model in order to explain personal IS usage in a university setting. This lead to the following research question: To what degree will a combination of PAT and TAM explain teaching academics usage intention of ICT? 2 THEORETICAL FRAMEWORK The Technology Acceptance Model is one of the dominating theories within the IS research (Venkatesh, 2008) and is described as one of the most mature approaches (Taylor & Todd, 1995). The Technology Acceptance Model has been adapted to higher educational settings in several studies (Mahdizadeh, Biemans, & Mulder, 2008; Saade & Bahli, 2005; Tao, Cheng, & Sun, 2009). The model itself has been the subject of several amendments and adjustments; the most dominant may be the Unified Theory of Acceptance and the Use of Technology (Venkatesh, Morris, Davis, & Davis, 2003). Information Systems continuance theory, developed by Bhattacherjee (2001a), is another relevant theory when studying technology acceptance. This theory is based upon Expectation Confirmation Theory (Oliver, A cognitive Model on the Antecedents and Consequences of Satifaction Decisions, 1980) and TAM (Davis, 1989). However, based on the dominance and the parsimony of the theory, TAM is selected in combination with PAT. Principal Agency Theory has been widely used and applied in different settings.in this study, PAT is used to explain how incentives can influence teaching academics acceptance of digital tools and services. A short presentation of each of the theories is given in sections 2.1 and Technology Acceptance Model The Technology acceptance model was proposed by Davis (1989) and is one of the dominating theories in explaining the individual adoption of IS in organizations. The theory is influenced by both social cognitive theory and decision-making theories, and its theoretical basis is adopted from the Theory of Reasoned Action (Ajzen & Fishbein, 1980). The Technology Acceptance Model introduces two important characteristics about an information system and the predicted outcome. The first characteristic, perceived usefulness, is defined as the degree to which a person believes that using a particular system would enhance his or her job performance (Davis, 1989). The second, perceived ease of use, is the degree to which a person believes that using a particular system would be free of effort (Davis, 1989). Effort is a finite resource that a person may allocate to the various activities to which he or she is responsible (Radner & Rotschild, 1975). The users attitude is expected to influence their behavioral intention, which in turn influences actual use. Empirical support for TAM has been proven through several studies (Venkatesh, Morris, Davis, & Davis, 2003; Karahanna, Straub, & Chervany, 1999). Perceived usefulness has especially been regarded as the

3 dominant predictor of IS use, while the results of perceived ease of use have been mixed (Premkumar & Bhattacherjee, 2008). The Technology Acceptance Model has, in educational settings, been used to predict both students and teachers adoption of new technology (Tao, Cheng, & Sun, 2009; Mahdizadeh, Biemans, & Mulder, 2008). 2.2 Agency Theory The principal-agent problem was originally formulated to study the separation of ownership and control that arose with the rise of professional managers controlling assets that they did not own (Berle & Means, 1932). The principal-agent perspective builds upon the original formulation of agency theory and addresses the agency relationship in which one entity (the principal) delegates work to another (the agent) who performs the work according to a mutually agreed contract (Eisenhardt, 1989). In this relationship, uncertainty will occur since the principal is not able to fully monitor the agent. This uncertainty leads to two information problems: hidden information and hidden action. Hidden information arises precontractually because the agent possesses private (hidden) information about its true quality (Akerlof, 1970). This fact creates difficulties for the principals in selecting the good agents from the bad. Hidden action takes place post-contractually after the principal hires an agent who may not exert the promised effort or engage in hidden actions that profit an agent at the principal s expense (Jensen & Meckling, 1972). Hidden action might become relevant because the principal cannot perfectly monitor the agent s behavior or performance. Hence, the agent may act in a manner that is inconsistent with the principal s interest, resulting in an agency problem. Principal Agency Theory assumes that human beings are risk averse and motivated by self-interest as well as that organizations are characterized by goal incongruence, information asymmetry, and efficiency norms (i.e., more effort leads to better outcomes) (Eisenhardt, 1989). Given these assumptions, PAT attempts to resolve the agency problem by proposing appropriate incentive schemes and control mechanisms. Agency theory has been extended to virtually all types of transactional exchanges that occur in a socio-economic system where information asymmetry, fears of opportunism, and bounded rationality exist (Milgrom & Roberts, 1992). Thus, PAT is considered to be a ubiquitous theory and has been applied to many types of relationships. Principal-agent perspective explains transaction arrangements between self-interested parties with incongruent goals in the presence of uncertainty. In this study, the university management is regarded as the principal and the teachers as the agents. 3 RESEARCH MODEL AND HYPOTHESES The dependent variable Usage Intention is defined as the users intention to use ICT. Intention is derived from attitude theories such as TAM and the Theory of Planned Behavior. Intention is considered to be a strong predictor of individual behavior (Ajzen & Fishbein, 1980). This association is based upon cognitive dissonance theory, which claims that perceived gaps between intentions and behavior create a psychological tension (cognitive dissonance) that individuals attempt to relieve by harmonizing their behavior and their intentions. Various studies can document this strong correlation between intentions and behaviors (Davis, 1989; Taylor & Todd, 1995). Consequently, and adopted in this study, this association between intention and behavior is taken for granted and the focus is on understanding the predictors of user intention. Below, the research model and the corresponding hypotheses are presented. 3.1 Research Model This study explains acceptance of IS by expanding TAM with the construct of incentives from PAT. The expectation is that an integration of these two models may provide a more comprehensive understanding of the phenomenon than each perspective may explain separately. Basically, PAT addresses the

4 relationship in which one entity (the principal) delegates work to another (the agent), who performs the work according to a mutually agreed contract. In this study, PAT is used to explain how incentives can influence the teachers use of ICT. While the principal (i.e., the university management) wants to increase the quality and flexibility related to teaching and learning through the use of digital tools and services, the teachers are expected to implement the managerial strategy. It is, however, not certain that the teachers share the management s ambition. Consequently, it is necessary to include PAT explanations related to IS acceptance to explain why and how agents should use the technology as expected. The model applied in this study is a simplified version of TAM where attitude is omitted. Davis et al. (1989) proposed a direct relationship between perceived usefulness/perceived ease of use to usage intention based on the assumption that users practical evaluation will override users attitude. The variables from TAM are perceived usefulness, perceived ease of use, and usage intention; the expected relationship between the variables is shown below (Figure 1): Perceived usefulness H1 Usage Intention H3 H4 Perceived Ease of H2 Incentives Figure1 - Research Model 3.2 Hypotheses The term perceived usefulness has its origins in TAM and is hypothesized as a direct predictor of acceptance intention (Bhattacherjee, 2001a). Perceived usefulness refers to the users own evaluation of whether the system will improve their performance or not and represents the rational element in the usage decision. Attitude theories hold that human behaviors are influenced by their subjective perceptions, even if such perceptions are biased or inaccurate (Ajzen, 1991). Hence, perceived rather than objective assessments are relevant. Several studies support the positive association between perceived usefulness and ICT usage intention (Bhattacherjee, 2001a; Limayem & Cheung, 2008; Hung, Chang, & Hwang, 2011). The following hypothesis is stated: H1: There is a positive relationship between the users perceived usefulness and the usage intention. Perceived ease of use means to what degree a person believes that using an IS will be relatively free of effort. If the teacher perceives the system as easy to use, it is expected to positively influence their future use. Hence, H2: There is a positive relationship between the users perceived ease of use and the usage intention. If a system is regarded as easy to use, it is expected that the users perceived usefulness will increase. Davis et al. (1989) identified such a relationship between perceived ease of use and perceived usefulness. Hence,

5 H3: There is a positive relationship between the users perceived ease of use and the users perceived usefulness. Research on ICT implementations documents that the management of an organization can motivate the users to utilize ICT appropriately by harmonizing the goals of the management and the goals of the users (Bhattacherjee, 1998). According to PAT, managerial incentives can be used to secure goal harmonization between the principal and the agent. People may act based on self-interested motives or moral hazards, such as a lack of effort on the part of the teachers. Such lack of effort may be difficult or too costly to monitor. Without incentives, it might be convenient for the teachers to not put forth the agreed upon effort. Hence, incentives are needed to secure usage intention, and the university management can, by implementing appropriate incentive mechanisms, secure the use of ICT/digital tools in education. Incentives can, for instance, be project funding related to ICT usage, additional courses, or financial compensation. Hence, the more incentives offered by management, the more the users will continue to use ICT. This lead to the following hypothesis: H4. The level of incentives will be positively related to the usage intention. 4 ANALYSIS 4.1 The measurement process The study includes four constructs, and all have been used and validated in previous studies. Nevertheless, several interviews with experts in the setting were conducted to establish content validity for the constructs. Thus, all measures were pretested on a small group of university teachers and further refined to ensure that the measures covered the domain of the construct. After an additional feedback round of the refined measures from three teachers, the questionnaire was finally revised and administered to the full sample. Reflective scales were used to measure the latent variables, and all the observed variables were measured by perceptual data (Table 1). All the measures used a seven-point Likert scale ranging from strongly disagree to strongly agree. Construct/Items Perceived Usefulness, source: Bhattacherjee (2001a): PerU1 - Use of digital tools improves the quality of my teaching. PerU2 - Use of digital tools increases my productivity as a teacher. PerU3 - Overall, use of digital tools is useful in my educational work. Perceived Ease of Use, source: Davis (1989): PerE1 - Using digital tools in my teaching is easy. PerE2 - I do find digital tools easy to use. PerE3 - I do find digital tools difficult to use. Intention to Continue, source: Bhattacherjee (2001a): Int1 - I intend to continue and even increase my use of digital tools in education. Int2 - My intentions are to increase my use of digital tools at the expense of traditional teaching methods. Int3 - If possible, I would like to discontinue the use of digital tools in education next year. Incentives, source Bhattacherjee (2001b): Inc1 - Allocation of project funds related to the use of digital tools in education will stimulate its increased use. Inc2 - Increased number of hours for teaching preparation will stimulate the increased use of digital tools. Inc3 - University management that clearly focuses on increased use of digital tools will stimulate its increased use. Table 1 Constructs and items

6 4.2 Data collection The sample of the study includes teaching academics at a Norwegian university college with approximately 5000 students and 650 teachers. The data were collected through an electronic questionnaire sent by to all teaching academics. To stimulate participation, an ipad was offered as a lucky draw prize. The respondents had the following split among the faculties: Faculty of Arts and Sciences (30.5%); Faculty of Art, Folk Culture, and Teacher Education (39.5%); Faculty of Health and Social Studies (14.1%); and the Faculty of Technology (15.9%). Altogether, 49.2% of the respondents were female, and 177 respondents completed the questionnaire, giving a response rate of 38.5%. 5 RESULTS The research model is validated by a structural equation modeling (SEM) approach using LISREL 8.80 (Jöreskog & Sörbom, 1993). The estimation of the models is performed using the maximum likelihood approach. The recommended two stage analysis proposed by Anderson and Gerbing (1998) was followed. The first stage uses confirmatory factor analysis to estimate the parameters of the measurement model. Second, path analysis was used to test the structural model, and the research hypotheses were then tested. 5.1 Measurement model results The evaluation of the constructs in the measurement model follows the recommended steps by Gerbing and Anderson(1998). First, the measures unidimensionality was evaluated by assessing various goodness-of-fit and modification indices (Table 2). Fit index Recommended value a Measurement Model Chi-square Chi-Square/df RMSEA NNFI IFI CFI Insignificant < 3 < 0.06 > 0.95 > 0.95 > (df=48; p=0.09) a (Hu & Bentler, 1999) Table 2 Fit Indices of the Measurement Model All values of the measurement model are acceptable according to the recommendations given by Hu and Bentler (1999). Next, the reliability and the validity of the constructs in the measurement model were assessed. Table 3 shows factor loadings for the four constructs and the composite reliability of each construct (Hu & Bentler, 1999). The composite reliabilities for each construct ranged from 0.71 to 0.90; these are higher than the recommended acceptance level of 0.60 (Bagozzi & Yi, 1988). The average variance extracted (AVE) are all above the acceptable 0.5 level (Bagozzi & Yi, 1988), meaning more than 50% of the variance observed in the items are accounted for by their hypothesized constructs.

7 Construct/Items Perceived Usefulness PerU1 PerU2 PerU3 Perceived Ease of Use PerE1 PerE2 PerE3 Intention to continue Int1 Int2 Int3 Incentives Inc1 Inc2 Inc3 Table 3 Internal consistencies Factor loading T-value Item reliability Composite AVE reliability Discriminant validity was assessed using the procedure suggested by Fornell and Lacker (1981). Composite reliability for each construct should be greater than the variance shared between the construct and other constructs in the model, namely the squared correlation between constructs (Duhan & Sandvik, 2009). As shown in Table 4 below, none of the squared correlations for each construct is higher than the composite reliability for each construct. Perceived Perceived Intention to Incentives usefulness ease of use continue Perceived usefulness.90 b.29 c Perceived ease of use.54 a Intention to continue Incentives a : The lower diagonal contains the estimated correlations among the latent variables. b : The diagonal contains the composite reliability of the constructs. c : The higher diagonal contains the squared correlations among the constructs. Table 4 Correlation matrix among the constructs 5.2 Structural model results As with the measurement model, SEM analyses using LISREL 8.80 was applied to the structural model. The same set of fit indices as analyzed in the measurement section was used to examine the fit of the structural model. All fit indices exceeded recommended levels for the structural model (Table 5). Fit index Recommended value a Measurement Model Structural Model Chi-square Chi-Square/df RMSEA NNFI IFI CFI Insignificant < 3 < 0.06 > 0.95 > 0.95 > (df=48; p=0.09) (df=49; p=0.02) Table 5 Comparison between the Measurement Model and the Structural Model

8 The hypotheses outlined in Section 3.1 are tested. The standardized path coefficients, path significances, and explained variances of the structural model are shown in Figure 2. Figure 2 Structural model Usage intention is directly influenced by perceived usefulness (β=0.51) and incentives (γ=0.58). Thus, hypotheses 1 and 4 are supported. Hypothesis 2, however, is not supported (γ=-0.09). Direct influences on perceived usefulness are observed by perceived ease of use (γ=0.54), and hypothesis 3 is supported. Explained variance (R 2 ) of intention to continue and perceived usefulness are 59% and 29%, respectively. 6 DISCUSSION 6.1 Theoretical implications Studies that address how managers can proactively influence users attitudes and actions toward organizational goals are asked for in the literature (Bhattacherjee, 1998, p. 140). My review shows that only a limited number of studies have combined TAM with PAT in higher education. Throughout the discussion of this study, I have argued for the relevance of combining the two perspectives. This study has shown the importance of managerial instruments for stimulating the acceptance of IS. Hence, one important contribution from the study is the incorporation of managerial influence concepts as complementary to the traditional ICT-centric concept. Incentives are documented to be a strong predictor of usage intention. Without incentives, it might be convenient for the teachers to continue as usual and not put forth the effort that is required. The Technology Acceptance Model has been criticized for primarily focusing on user acceptance and the first-time use of an IS (Bhattacherjee, 2001a). Other theories, such as Information Systems Continuance theory (Bhattacherjee, 2001a), focuses on continued use of an IS. It is a challenge for the management of institutions to secure continued use and especially secure the realization of the investment. Future research should evaluate the continuance intention of Information Systems Continuance Theory in combination with Agency Theory. This study did not find evidence for the expected positive relationship between perceived ease of use and usage intention. It is tempting to draw the conclusion that it will not be sufficient for a system to be easy to use, as the perceived usefulness must be sustained. According to Premkumar and Bhattacherjee (2008), the expected positive relationship between perceived ease of use and intention to continue in the model (TAM) is documented as somewhat inconsistent.

9 6.2 Practical implications Higher education will experience dramatic changes. E-learning in general is not well integrated in the higher level learning processes and teachers just use the superficial capabilities of the e-learning tools (Mahdizadeh, Biemans, & Mulder, 2008, p. 151). The universities and colleges will experience increased competition and, in some areas, decreased enrolments. Students want more flexibility and an increased number of massive open online courses (MOOC) or other online courses will be available. Digital tools and services are expected to change academia. Christensen (2008) describes these new possibilities as a disruptive innovation within higher education academia will be influenced by the new technology; the question is only how and at which speed. The Norwegian Ministry of Education supports a goal of increased use of ICT in education. The management of the universities and colleges must prepare for this change. The first step in such a process will be to develop a strategy that clearly states their academic goals. The management must support and promote this strategy and perform an anchoring process to secure goal congruence between the management and faculties. Based on the study s results, the motivation for teachers technology acceptance can be classified into two primary categories: i) Mechanisms to stimulate teachers characteristics of perceived usefulness and ii) Designing management systems of incentive structures. Below, and derived from the study s theoretical model, practical guidelines for an initially strategic formulation phase for universities is discussed. Secure perceived usefulness Perceived usefulness is defined as the degree to which a person believes that using a particular system would enhance his or her job performance (Davis, 1989). This enhancement can be increased flexibility for the students, increased efficiency for the teachers, increased throughput, or improved learning outcomes. Increased digital literacy will facilitate such enhancements, and a course program to secure this should be established. The faculty might be unaware of the various services and how it can be used in teaching, learning, and assessment. Increased use of digital tools and services will require a support organization. This does not only include technical support but also pedagogical support. Necessary infrastructure must be in place and a competent support organization should be established Establishing incentive structures The faculty might regard the use of digital tools and services as a threat to their academic freedom and autonomy. The university management will have to understand this possible technological change from the perspective of the faculty and establish incentive structures for encouraging the teaching academics to use the technology. As previously discussed, digital competence among teachers may vary, and a relevant incentive system is needed. Such a reward system is lacking in most universities. This could be not only seed money related to specific projects, but also sustained incentives for technical equipment and release time. 6.3 Limitations and future research The IS theory used in this study is TAM. As already discussed, TAM is a theory focusing on technology acceptance and is primarily related to first-time use. Other theories, like ISCT, are focusing on continued use. Developing and implementations of IS are associated with considerable cost and securing continued use is of great importance. Future research should consider implementing competing IS theories. There may also be additional determinants and associations with ICT-usage than the ones identified in this study. Further examination of the topic may include other and diverse theoretical perspectives, as internal motivation concepts from e.g. self-determination theory (see e.g. Sørebø et al. (2009) for a synthesis of IS

10 and SDT). Such kind of studies may further validate the study s model or alternatively propose a more comprehensive explanation of ICT-usage. This study employed only one possible PAT-theoretical lenses of causal relationships in the model. As such, there may be other theoretical alternatives of modeling the PAT-derived constructs. Thus, future studies should test different models with different paths in a synthesis. Especially, there are theoretical arguments for risk aversion and goal conflict as moderators between incentives and ICT-usage (Bhattacherjee, 1998; Eisenhardt, 1989). These alternative PAT-theorized models should be outlined and tested in future research. Moreover, the study uses a correlation design and conducted a cross-sectional survey. Such designs have methodological limitations when testing causal models. As all the hypotheses developed in the study rest on arguments about causal relations, the directionality arguments is based on and consistent with theory. Theory alone, however, may not be sufficient to confirm causality when the chosen design lacks the possibility to test directionality. The present study with its corresponding design can only examine if the pattern of relationships is consistent with the underlying established causal understanding. Thus, future studies should employ longitudinal studies in order to establish directionality and confirm causality. This study does not address the association between usage-intentions and -behaviors. Thus, future studies should go beyond the original TAM by incorporating continuance behavior as the ultimate dependent variable. Finally, the study surveyed the teachers in one University. In order to increase external validity, further studies should test the model in different organizational settings, including the Private sector. REFERENCES Ajzen, I. (1991). The Theory of Planned Behavior. Organizational Behavior and Human Decision Processes (50), Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice Hall. Akerlof, G. (1970). The Market for Lemons: Quality Uncertainty and the Market Mechanism. Quarterly Journal of Economics (84:3), Anderson, J. C., & Gerbing, D. W. (1998). Structural equation modeling in practice: a review and recommended two-step approach. Psychological Bulletin, 103, 3, Bagozzi, R. P., & Yi, T. (1988). On the evauation of structural equation models. Journal of the Academy of Marketing Science 16(1), Berle, A. A., & Means, G. C. (1932). The modern corporation and private property. New York: MacMillian Publishing. Bhattacherjee, A. (1998). Managerial Influences on Intraorganizational Information Technology Use: A principal-agent Model. Decision Sciences (29:1), Bhattacherjee, A. (2001a). Understanding Information Systems Continuance; An expectation- Confirmation Model. MIS Quarterly, 25, Bhattacherjee, A. (2001b). An empirical analysis of the antecedents of electronic commerce service continuance. Decision Support Systems 32,

11 Christensen, C. M. (2008). Disruptive Class: How Disruptive Innovation Will Change the Way the World Learns. New York: McGrawHill. Davis, F. (1989). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly, 13, Davis, F., Bagozzi, R., & Warshaw, P. (1989). User acceptance of computer technology: a comparison of two theoretical models. Management Science 35(8), ss Duhan, D., & Sandvik, K. (2009). Outcomes of advertiser-agency relationships: The form and the role of cooperation. International Journal of advertising, Eisenhardt, K. (1989). Agency Theory: An Assessment and Review. Academy of Management Review (14:1), Fornell, C. D., & Lacker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. J Mark Res;18 (Ferbruary), Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A multidisciplinary Journal 6:1, Hung, M. C., Chang, I. C., & Hwang, H. G. (2011). Exploring academic teachers' continuance toward the web-based learning system: The role of causal attributions. Computers & Education, Jensen, M., & Meckling, W. H. (1972). The Theory of the Firm: Managerial Behavior, Agency Costs and Ownership Structure. Journal of Financial Economics (3:4), Jöreskog, K. G., & Sörbom, D. (1993). LISREL8: Structural equation modelling with the SIMPLIS command language. Hillsdale, NJ: Lawrence Erlbaum Associates. Karahanna, E., Straub, D., & Chervany, N. (1999). Information technology acceptance across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs. MIS Quarterly 23(2), ss Limayem, M., & Cheung, C. M. (2008). Understanding information systems continuance: The case of Internet-based learning technologies. Information & Management 45, Mahdizadeh, H., Biemans, H., & Mulder, M. (2008). Determining factors of the use of e-learning environments by university teachers. Computers & Education 51, Milgrom, P., & Roberts, J. (1992). Economics, Organization and Management. Upper Saddler River, NJ: Prentice Hall. Oliver, R. L. (1980). A cognitive Model on the Antecedents and Consequences of Satifaction Decisions. Journal of Marketing Research, 17, Premkumar, G., & Bhattacherjee, A. (2008). Explaining information technology usage: A test of competing models. Omega 36, ss Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon, Vol. 9 No. 5, 1-6. Radner, R., & Rotschild, M. (1975). On the allocation of effort. Journal of Economic Theory (10),

12 Saade, R., & Bahli, B. (2005). The impact of cognitive absorption on perceived usefulness and perceived ease of use in on-line learning: an extension of the technology acceptance model. Information & Management 42, Sørebø, Ø., Halvari, H., Gulli, V. F., & Kristiansen, R. (2009). The role of self-determination theory in explaining teachers' motivation to continue to use e-learning technology. Computers & Education 53, Sørebø, Ø., Halvari, H., Gulli, V. F., & Kristiansen, R. (2009). The role of self-determination theory in explaining teachers' motivation to continue to use e-learning technology. Computers & Education 53, pp Tao, Y., Cheng, C., & Sun, S. (2009). What influences college students to continue using business simulation games? A Taiwan experience. Computers & Education 53, Taylor, S., & Todd, P. A. (1995). Understanding Information Technology Usage: A Test of Competing Models. Information Systems Research 6:2, Venkatesh, V. (2008). Technology Acceptance Model 3 and a research agenda on interventions. Decision Sciences, 39(2), Venkatesh, V., Morris, M., Davis, G., & Davis, F. (2003). User acceptance of information technology: toward a unified view. MIS Quarterly 26(4), ss Ørnes, H., Wilhelmsen, J., Breivik, J., & Solstad, K. J. (2011). Digital tilstand Norgesuniversitetets monitor. Tromsø: Norgesuniversitetet.