The Impact of Non-Normality and Estimation Methods in SEM on

Transcription

1 The Impact of Non-Normality and Estimation Methods in SEM on Satisfaction Research in Marketing. Tor W. Andreassen*, Associate Professor, Department of Marketing, Norwegian School of Management BI Bengt G. Lorentzen, PhD-candidate, Department of Marketing, Norwegian School of Management BI Ulf H. Olsson, Professor, Department of Economics, Norwegian School of Management BI *) Corresponding author Norwegian School of Management BI Box 580 N-1301 Sandvika, Norway Phone FAX

2 Abstract This paper discusses consequences of violating the normal distribution assumption imbedded in Structural Equation Modeling (SEM). Based on real data from a large sample customer satisfaction survey we follow the procedures as suggested in leading textbooks. We document consequences of this practice and discuss its impact on decision making in marketing. Key words: Non-normality, SEM, estimators, survey data, satisfaction studies The authors recognize valuable help offered by Kenneth Wathne on an earlier draft of this paper. 2

3 1. Introduction Being able to estimate the financial benefits from investments in revenue expansion activities, cost cutting activities, or both simultaneously (Rust, Moorman et al. 2002) are critical to any business, particularly in stagnating markets. According to Marketing Science Institute s research priorities for 2002 and 2004, assessing marketing productivity (return on marketing) and marketing metrics are top tier priority topics. In an effort to move from data to prioritized decisions managers today draw heavily on importance-performance analysis (Martilla and James 1977) derived from customer satisfaction models. According to these analyses, managers focusing on return on quality (Rust, Zahorik et al. 1994) should allocate resources into areas that are important to customers and in which the firm is performing poorly. But what would happen if these recommendations were erroneous due to technical issues embedded in the statistical method or software package used to analyze the data? Today it is fair to claim that most analyses linking brand equity, value equity or retention equity (Rust, Zeithaml et al. 2000) to customer equity (Blattberg and Deighton 1996) usually are founded on structural equation modeling (SEM). Over the years SEM has helped researchers develop and test new theories by studying correlations between several complex constructs simultaneously. In this respect seminal work linking, for example, service quality to economic performance at the macro level (see for example Johnson and Fornell 1991; Fornell 1992; Fornell, Johnson et al. 1996; Johnson, Gustafsson et al. 2001) and micro level (see for example Rust and Zahorik 1993; Rust, Zahorik et al. 1994; Rust, Zeithaml et al. 2000) have been documented and communicated. Despite extensive use of SEM in marketing research, surprisingly little work has been reported in the literature related to estimation methods. Reviewing SEM in Journal of Marketing, Journal of Marketing Research, Journal of Consumer Research and Journal of the Academy of Marketing Science for the years , Olsson, Troye and Foss (2001) 3

4 found that of 73 applications only 16 reported to have used ML, five had used GLS, three had used WLS, four had used ERLS (Elliptical Reweighted Least Squares), while the majority (46) did not specify which method was used. In a similar vein, Breckler (1990) identified 72 articles in personality and social psychology journals that had used structural equation modeling and found that only 19 percent acknowledged the normal theory assumptions, and fewer than 10 percent explicitly considered whether these assumptions had been violated. The fact that GL and ML estimators assume that the observed variables are multivariate normally distributed is fundamental to estimation methods in SEM. Violation of this assumption may distort the standard error of the path coefficient between latent variables and the test statistics. Accordingly it is imperative to investigate the observed variables with regard to excessive kurtosis and skewness as this usually eliminates asymptotic efficiency and makes the estimated asymptotic covariance matrix, the chi-square and t-tests potentially inaccurate. Interestingly, whereas most satisfaction measures are negatively skewed (Westbrook 1980; Oliver 1981), little if anything, is reported with regard to kurtosis. Large deviations from the assumption that observed variables are multivariate normal distributed may cause problems with regard to statistical conclusion validity (Bearden, Teel et al. 1980). The fact that both theory development and management decisions may be impacted by the findings suggested by SEM studies raises one question of major concern: If correlations between constructs are impacted by technical issues related to and imbedded in SEM, could this cause researchers and managers to draw wrong conclusions? Research studying the effect of the different estimators under varying degrees of non-normality and misspecification (Chou, Bentler et al. 1991; Chou and Bentler 1995; Yuan and Bentler 1997; Olsson, Foss et al. 1998; Olsson, Troye et al. 1999) has found that technical issues embedded in SEM do indeed impact parameters. These studies have, however, to a large extent been based on simulated data thus leaving the question unanswered in a real world setting. 4

5 The purpose of this paper is to investigate the consequences of using Maximum Likelihood (ML), Generalized Least Squares (GLS) and Weighted Least Squares (WLS) with survey data on a typical marketing model. Based on real data from a large customer satisfaction survey, a simple but typical marketing model linking service quality to behavioral intent is proposed. Second, the same model is estimated by using different estimators and procedures. Third, parameter-estimates and fit-statistics from the different estimations are compared and conclusions drawn. Finally, the implications of the findings are discussed. 2. The conceptual model Most quality > satisfaction > loyalty models are founded on the disconfirmation of expectation paradigm (Oliver 1980), which argues that customer loyalty (for example repurchase intentions, willingness to provide positive word-of-mouth) is a function of customer satisfaction. Customer satisfaction is a summary cognitive and emotional reaction to a service incident (or sometimes to a long-term service relationship), which occurs after the point where the customer makes a comparison of expectations prior to consumption with actual experience. In keeping with recent thinking and business practice, our simple model reflects actionable quality drivers of customer satisfaction, which is an antecedent to customer loyalty (Johnson, Gustafsson et al. 2001). In short, our model reflects a banking context and consists of three exogenous variables which are in keeping with three of Rust & Olivers (1994) four components of service quality: physical product (branch), service product (savings, loan), service environment (branch), and service delivery. The exogenous variables are all positively linked to customer satisfaction. Finally, satisfaction is positively linked to customer loyalty. Figure 1. The conceptual model. INSERT FIGURE 1 ABOUT HERE 5

6 From the conceptual model (see figure 1), which is supported by contemporary research in marketing, we can learn that a positive change in any of the quality variables will have a positive filter-through effect on customer loyalty. According to the return on quality and customer equity management literature (see for example Rust, Zahorik et al. 1994; Rust, Zeithaml et al. 2000), this will materialize in the form of improved company performance. Managers of private service companies, who want to maximize profits in the long run, will have to allocate resources to quality areas, which have a stronger impact on customer satisfaction and thus customer loyalty. 3. The theoretical model Quality can be used to operationalize utility (Perreault and Russ 1976), i.e. customer satisfaction. Customer satisfaction cannot be measured directly by using an objective measure (Simon 1974). If, however, satisfaction is treated as an abstract and theoretical phenomenon it can be measured as a weighted average of multiple indicators (Johnson and Fornell 1991). Measurement errors in the index are taken care of through the quality and quantity of the measures being used (Fornell 1989). In our model, customer satisfaction (CSAT) is a function of branch (+BRA), loan (+LOA), and saving (+SAV). Customer loyalty (+CLOY) is a function of +CSAT. Each construct is treated as a latent variable with multiple indicator measures (Bolton and Drew 1991; Oliver 1992). The theoretical framework can be summarized as: Customer loyalty = (customer satisfaction, ζ 2 ) Customer satisfaction = (branch, loan, savings, ζ 1 ) 6

7 ζ i is a vector of other factors not captured by the equation (e.g. environmental, situational, error). The theoretical model can be expressed formally as: η 1 = γ 11 ξ 1 + γ 12 ξ 2 + γ 13 ξ 3 + ζ 1 η 2 = β 21 η 1 + ζ 2 The measurement models are illustrated in Table 1. Table 1. The measurement models. INSERT TABLE 1 ABOUT HERE The structural model (and the measurement model) is shown graphically in Figure 2: η 1 = Satisfaction, η 2 = Loyalty, ξ 1 = Branch, ξ 2 = Loan, and ξ 3 = Saving. All items relating to each construct are documented in appendix A, whereas appendix B documents descriptive statistics of data used (i.e. mean, standard deviation, skewness and kurtosis). The theoretical model is illustrated in figure 2. Figure 2. The theoretical model. INSERT FIGURE 2 ABOUT HERE 4. Why estimation methods may produce discrepant empirical fit and parameter estimates In structural equation modeling (SEM), we want to estimate the parameter vector θ and a model implied covariance matrix Σ (θ ) where vector θ contains all the unknown parameters in a theoretical model. The estimate is based on a sample covariance matrix S that is assumed 7

8 to converge to the population covariance matrix Σ as N grows large. An estimate θ of θ is obtained by minimizing a discrepancy function F ( S, Σ( θ )). A precise definition of a discrepancy function is given in Browne (1984). In the same paper, three typical discrepancy functions are presented: the first is the quadratic form discrepancy function of the type 1 (1) F ( S, Σ( θ ) U ) = ( s σ ( θ ))' U ( s σ ( θ )), 1 where the weight matrix U is a k * k * positive definite matrix ( k * = k( k + 1) and k is the 2 number of manifest variables). This is what we understand as the WLS 1 estimator in the LISREL language. The second example is the GLS estimator given by { (2) G( θ W ) = tr ( S Σ( θ )) W }, 2 where W is a k k weight matrix, often chosen to be the sample covariance matrix S. The third example is the maximum likelihood (ML) discrepancy function 1 (3) F( S, Σ( θ )) = log Σ( θ ) + tr{ SΣ( θ ) } log S k. However, 1), 2) and 3) can be written on the same form: (4) F = ( s σ ( θ ))' W ( s σ ( θ )), where the weight matrix W is defined differently for the different functions (Jøreskog, Sørbom et al. 1999): 1 1 ML: W ML = D'( Σ ( θ ) Σ ( θ )) D ; where D is the duplication matrix (Magnus and Neudecker 1988) and θ is the ML estimator. 1 1 GLS: W GLS = D'( S S ) D ; where S is the sample covariance matrix. WLS: W WLS 1 = U WLS 8

9 is a k * k * matrix, the asymptotic covariance matrix 2 of the k * 1 vector N 1( s σ ). It is well known that 1), 2) and 3) will produce estimates for a structural equation model that will converge to the same optimum and have the same asymptotic properties (Browne 1974) when models are correctly specified and the observed variables come from a multivariate normal distribution 3. However, the above listed ideal conditions are almost never satisfied. When models are mis-specified, but the data are multivariate normal, the fit function and parameter estimates for ML should be expected to differ from those provided by GLS and WLS. On the other hand, when models are not mis-specified (or only mildly misspecified), but data are not normally distributed, the WLS solutions should differ from those obtained with GLS and ML (Olsson, Foss et al. 2002). The reason why the various fitfunctions do not converge when models are mis-specified or data are not multivariate normal is that the nature of the weight element differs. For the GLS fit-function, these elements consist of second order moments s s + s s. When the model is correct under normality ig jh ih jg the elements of σ σ W WLS and W GLS will converge in probability to the population values + σ σ although their variability is different (Hu, Bentler et al. 1992). ig jh ih jg The weight matrix for ML, 1 1 D '( Σ ( θ ) Σ ( θ )) D, is a function of the model, but for GLS and WLS the elements in the weight matrix are only functions of second and fourth order moments of the observed variables. The elements of W WLS for GLS do not depend on the model, only on S, and since S will converge to Σ even if the model is poor, the result can be quite different from the ML-situation where Σ( θ ), ( where θ is a ML estimate), approximates Σ only when the model fits very well or exactly. For the WLS method, the elements of W WLS also depend on S but include the 4 th order moments, which are estimated 9

10 from the sample. Without any specification error we know that both S and Σ( θ ) will converge to Σ. In this situation the differences between the ML and GLS are due only to sampling error. Whether WLS will be close to ML and GLS will depend on the distributional properties of the observed variables. For small deviations from normality and little or no specification error, the weight matrix for WLS should not differ systematically from the weight matrix of GLS or ML. For negligible kurtosis is it reasonable that the three are close. The literature recommends WLS when data are non-normal. The assumed superiority of WLS when data are peaked or skewed may not hold when models are misspecified. Olsson, Foss and Troye (2002) shows analytically, for WLS, that the fit-function value in the minimum point can be a decreasing function of the 4 th order moment when the models are misspecified. This can of course lead to inflated values of fit ( too good fit ) when the data are peaked (see appendix D for illustrations). A low chi-square may not only point to good fit, but also to bad fit with low power. Another aspect concerning WLS is that the method has been shown reliable only for large sample sizes (N > 1000 for relatively simple models (Curran, West et al. 1991); N > 5000 for more complex models (Hu, Bentler et al. 1992)). To summarize: the reason why ML provides results that differ from those obtained with GLS and WLS is that mis-specification causes the model-based estimated covariances in the weight matrix for ML to be different from the observed covariances that are entered in the weight matrices of GLS and WLS. The discrepancy between WLS and the two other estimators when data are non-normal is that only the weight matrix of the former contains elements (the 4 th order moments) that reflect this violation. When the models are incorrectly specified and the data are not multivariate normal (see cell 4 in table 2), the methods should, 10

11 according to Olsson et al (2000), give different results. When data are multivariate normal and the model is mis-specified WLS and GLS are equivalent (cell 2 in table 2). Table 2. When are ML, GLS and WLS equivalent? INSERT TABLE 2 ABOUT HERE 5. Hypothesis Given that Σ( θ ML ) Σ( θ GLS ) Σ( θ WLS ) => θ ML θ GLS θ WLS in general, the three estimators will produce different parameter vectors. If this difference is significant, the choice of estimators may have implications for theory development in marketing and subsequently decision making. 6. Research strategies Often the observed variables are not continuous, but ordinal. In other words, the variables represent responses to a set of ordered categories. Even though it is common practice to treat scores assigned to categories as if they have metric properties, this is wrong. The researcher therefore faces several challenges in choosing the optimal estimator. First, the scaling problem, and second the question of normality. There are two assumable main options available to the researcher: 1) Assume that the observed variables are continuous and then: a) Ignore non normality and estimate the model with ML (or GLS). If the observed variables are non-normal this can lead to incorrect standard errors and chi-square statistics. b) Transform the observed variables and estimate the model with ML (or GLS) c) Estimate the model with ADF/WLS. This method should, in theory, be the correct procedure, given non-normal observations. However, the method requires a very 11

12 large sample (Hu, Bentler et al. 1992), and it has also been shown that the ADF/WLS chi-square test lacks power if the kurtosis is severe (Olsson, Foss et al. 2002). d) Estimate the model with ML and correct for bias by using the Satorra-Bentler correction (Satorra and Bentler 1988). This correction also requires a large sample. 2) Treat the observed variables as ordinal and estimate polychoric correlations and then estimate the model with ADF/WLS. In order to assess the consequences of the different procedures put forth in the abovementioned strategies, an identical model with a corresponding identical covariance matrix was subjected to the different estimation procedures. 7. The dataset The data set was extracted from a database that consisted of customer satisfaction data, which had been collected over a period of two years, from private customers of a large Scandinavian bank. The data had been gathered through completed postal surveys from active customers of the bank. The design of the questionnaire, as well as the data gathering method, was identical for the two surveys, which had been undertaken to build the database content. The questionnaire was based on a satisfaction-model, which covered the entire scope of the bank s performance related to products and service. For this study, a sub-section of the original model was selected in order to achieve a satisfactory level of number of observations with no missing data embedded. To impose a high degree of non-normality on the data with respect to kurtosis, selections were made from the database based on various background variables, such as gender, age, number of years as a customer with the bank and so forth. By 12

13 screening the observed variables after each selection procedure with regards to skewness and kurtosis, the degree of non-normality was assessed. This provided a suitable set of data for the study. A total of 1042 valid observations were included in the study. (See appendix A and B for an overview of the wording of the items and descriptive statistics. Appendix C gives an overview of descriptive statistics of the normal scores.) 8. Results The results of applying the above options are listed in table 3 and table 4. Table 3. Parameter estimates, chi-square statistics and RMSEA values for the original model. INSERT TABLE 3 ABOUT HERE Table 4. Parameter estimates, chi-square statistics and RMSEA values for the modified model. INSERT TABLE 4 ABOUT HERE By observing the results from ML (table3, model 1), we find support for the underlying theory, i.e. ξ1, ξ2, and ξ3 have positive effects on η1, which has a positive effect on η2. However, the model fit is not good as the chi-square statistic is large. It is more than nine times as large as degrees of freedom. Ideally it should be close to 1. However, a large chi-square is to be expected for a sample size as large as this. In addition, RMSEA (0.092) at the same time, indicates a rather bad fit because it is out of the range of and thus deemed unacceptable (Hair, Anderson et al. 1995, p. 685). To most researchers, this is a familiar situation, which triggers a search for a better fit by modifying the original model. In keeping with Jöreskog (1993), model generating is the most common situation, i.e. if the initial model does not fit the given data, the model should be modified and tested again using the same data. Following this advice the researcher will employ the modification indices 13

14 provided by LISREL. In our case LISREL advice respecifying MI -> Fr TD 3 4, TD 4 5, TD 8 9, TD and TE 1 2. As illustrated in table 4 (model 1) this will provide us with a better fit (RMSEA =0.058). On the other hand, two of the path coefficients, γ11 and γ13, are nonsignificant in addition to the fact that the sign of γ11 (Branch) is not as anticipated. Still, there is a consistent pattern, i.e. ξ2 (Loan) is the stronger driver of η1 (Satisfaction), and η1 drives η2 (loyalty) in all models. Based on this, a practitioner could conclude that improving Loans would improve customer loyalty due to largest increase in satisfaction. A more trained researcher would, on the other hand, identify the non-normality of the data as a possible problem and would try to compensate for kurtosis and skewness. One option is to transform the data. In this scenario we use the normal scores provided for each variable by LISREL The results are illustrated in table 3, model 3. Since RMSEA (0.090) is somewhat outside the acceptable limit of 0.08 we would conclude that the model does not fit the data well. Interestingly, although ξ2 (Loan) is the stronger driver of satisfaction, ξ3 is now identified as the second stronger driver of η1. Despite the fact that the model findings are in keeping with extant literature, the fairly poor fit statistics will make the researcher feel somewhat uneasy. In search for a better model fit, the researcher would again employ Jöreskog s (1993) advice and look for modification suggestions and try to respecify the model. Although the fit is improved (see table 4, model 3), the RMSEA drops from to 0.055, the change of sign in γ11 represents a problem. The second option to handle non-normality is to use the WLS estimator. In this scenario we keep the same basic model but change the estimation technique to WLS and assume that the observed variables are continuous. From Table 3 (model 6) we can see that the model fits the data very well (i.e. RMSEA = 0.042). Interestingly γ11 is now the stronger driver of satisfaction. Sticking to WLS but treating the variables as ordinal (which they really are) and using polychoric correlations as input (table 3, model 7), we observe almost identical 14

15 fit values, but there is a significant change with respect to the parameters: now γ12 is the stronger driver of satisfaction. With WLS for continuous data γ11 = 0.46 and γ12 = 0.33 while for WLS with ordinal variables the estimates become 0.29 and 0.44 respectively. To summarize the results the following pattern emerges: estimating the initial model with ML results in poor fit, but expected directions of parameter estimates. The lack of fit might be caused by non- normality in the data. Transforming the data to normal scores does not improve the fit. Interestingly, however, the parameter estimates change significantly. Using the WLS estimator also results in a mixed situation depending on whether we are treating the variables as continuous or ordinal. The fit values are almost identical, but the parameter estimates are significantly different. 9. Theoretical implications We do not know to what extent researchers make comparisons of fit as a basis for choosing estimation method. Our results suggest that such a decision rule, whether it is common or not, is not warranted. The choice of an estimation method and its favourite model based on fit alone, may easily lead to the rejection of a method and a model that on other grounds are more desirable. It is difficult to assess the degree to which our findings can be generalized beyond this particular study. What the findings above do suggest, however, is that existing measures of fit are not necessarily comparable across estimation methods (see also Olsson et al. 2000). Rules of thumb as to what are acceptable values of fit should, therefore, not be invariant over estimation methods. What do the findings suggest with respect to choice of estimation method and choice of empirical fit measures? In the following we will first summarize what we believe to be the implications for the use of fit measures and then proceed to an evaluation of estimation methods. 15

16 For highly peaked data, RMSEA appears particularly permissive to misspecification when WLS is used (Olsson, Foss et al. 2000). Model selection based on RMSEA may thus easily result in Type II error. WLS would only be the natural choice when the tested model is close to the population model and the data are highly peaked. The problem is that the fit measures for WLS may suggest a good model, when in fact the model is not close to the generating model (Olsson, Foss et al. 2000). As will be discussed below, a diagnostic procedure that could be used to assess whether this is the case, is to compare the results across methods. Large parameter discrepancies would suggest that the model is misspecified even in situations where isolated assessments of empirical fit for WLS suggest that the model is acceptable. We assume that the majority of the marketing studies that did not report which estimation method was employed, in fact used ML since this estimation method is the default procedure and ML is the one most often reported to have been chosen. Our results suggest that marketing researchers are well advised to continue using ML. On the other hand, researchers should be required to report which estimation method is used. Other estimation methods may be employed as complementary procedures to ML to assess misspecification. Thus a test procedure that could be employed, is estimation triangulation. When different estimation methods, e.g ML, GLS, MLns, GLSns (see table 3, models 1 through 4) provide similar (but not identical) solutions, choose WLS if data are peaked. When data are not peaked and solutions differ substantially across methods in terms of empirical fit and parameter estimates, choose ML and the model structure and parameter values suggested by this method. Based on the current results, we do not see any justification for using GLS. More research is, however, needed to justify this conclusion fully. 16

17 10. Managerial implications The general question we addressed in this paper was Can Estimation Methods in SEM Influence Decision Making in Marketing? From the previous discussion it is obvious that this is the case. If not dealt with, it may cause resource allocation based on the importanceperformance analysis to have non-desired consequences on for example customer equity. There are three managerial implications to be drawn from this study. 1) Allocating resources to non-key drivers of customer satisfaction may distort customer equity either through changes in the firm s overall value equity or through changes in the underlying segments perception of value equity (Zeithaml, Rust et al. 2001). 2) Negative changes in perceived value equity may update customers perception of and attitude towards the firm in general. In keeping with this, the allocation of resources to non-key drivers of customer satisfaction may distort brand equity (Bolton and Drew 1991; Johnson, Gustafsson et al. 2001). Both changes have the potential to distort the firm s retention equity through increased exit and thus reduce the firm s customer equity. 3) For service firms, the allocation of resources to the wrong drivers of customer satisfaction may have organizational implications through structural change. Both loss of productivity due to organizational changes and negative feed back due to reduced customer satisfaction (alternatively: increased customer dissatisfaction) may have a distorting impact on the firm s service climate and thus perceived service quality and customer satisfaction in the next period (Schneider, White et al. 1998). As we have moved through the results from table 3 and table 4 in our study the model quality and results changed. Managers or analysts may, based on the data sampled and results from SEM as shown in table 3, models 1-5 and 7, conclude that improving Loan will have the highest impact on satisfaction and thus loyalty. In table 3, model 6, the picture changes. Whereas employing all other estimators and procedures than WLS (continuous variables) defines Loan as the stronger driver of satisfaction, WLS (continuous variables) identifies 17

18 Branch as the stronger driver. With a strong theory two decision makers will draw different conclusions as to where they should invest in order to improve customer satisfaction. In summary, from the same theoretical model and the same data set we identify two dominant drivers of customer satisfaction (i.e. Loan and Branch) and three different number two drivers (i.e. ξ1, ξ2, and ξ3). Focusing on WLS (continuous variables) as the preferred estimation technique (table 3, model 6) investments in Branch which may improve the score by 10 points on a point scale will increase customer satisfaction by 4,6 points (10 x 0.46) and customer loyalty by 3,8 points (10 x 0.46 x 0.83). By breaking down the data on various customer segments similar models can be calculated. With this in mind, SEM forms the foundation for Return on quality and customer lifetime value analyses that may have a major impact on managers` resource allocation in order to maximize shareholder wealth. Our concern is based on the set of assumptions and defaults imbedded in SEM. 11. Limitations and future research This study represents a systematic approach to investigate the consequences of using different estimators and textbook procedures on non-normal data. It has a number of features, including a large sample size, an actual customer satisfaction model that is used as a basis for making a specific company s strategic marketing decisions and data that is based on responses from active customers of that company, thus creating a realistic setting that lends strength to the study. However, it is not without its limitations. First, the context of this study is in private banking. On the one hand, because we are mainly interested in testing a given model on non-normal data and exploring the differences in the results between the various procedures, the context should be of minor concern. At the same time some of the differences we observe may partly be a result of the chosen context in this study, and not a result of the chosen procedure alone. For future research the same 18

19 research design therefore might be employed on other contexts thus controlling for contexteffects. Second, our data-file is drawn from a comprised data set of two separate studies. The questionnaire and the design of the studies were identical, but they where undertaken at different times. External conditions may have been different in the two points of time resulting in incomparable data-files. This has not been controlled for in this study, which also may affect the results of our estimations. Thirdly, our theoretical model s degree of misspecification has not been considered in this study. A model s level of misspecification does have important impacts on choice of estimator and ultimately parameter results. The model in this study is likely to be under-specified due to the fact that we use a sub-section of the original model in the survey. Keeping in mind that this was necessary to achieve a satisfactory number of observations in order to test the ADF-estimator as well as the Satorra- Bentler procedure under conditions of no missing data, we have probably introduced a certain level of misspecification in the study in addition to non-normality in the data. Future studies should aim to reduce the level of misspecification in order to investigate better the effects of non-normality in the data only. Finally, the different strategies that decision-makers in SEM use in situations of non-normality put forth in this study may be challenged. Although all strategies put forth are derived from textbook knowledge, users of SEM (whether it be practitioners or researchers) may still follow other strategies. An investigation into what strategies that are actually employed by SEM-users might be fruitful for future research within the field. 19

20 Appendix A: Observed variables The following observed variables correspond to the model in figure 2. Branch Loan Savings Satisfaction Loyalty How satisfied or dissatisfied are you with your regular branch with regards to; x 1 - the employees readiness to help? x 2 - the employees ability to recognize you? x 3 - the employees ability to be flexible? x 4 - the employees mode of treatment? x 5 - the employees personal conduct? x 6 - the employees ability to counsel you? x 7 - atmosphere? Consider your latest loan in the bank. How satisfied or dissatisfied are you with regards to; x 8 - the swiftness of the reply on your application for the loan? - how your historical relationship with the bank were x 9 taken into account in the loan application assessment? x 10 - how the terms of your loan has been adjusted in accordance with changes in the market? x 11 - how the terms of your loan has been adjusted in accordance with your personal relationship with the bank? Consider your relationship with the bank with regards to savings. How satisfied or dissatisfied are you with regards to; x 12 - the banks ability to promote products? x 13 - the banks ability to advice on savings? x 14 - return on your savings? x 15 - the frequent information you get on savings? y 1 Overall, how satisfied or dissatisfied are you with your bank? y 2 Compared to an ideal bank, how satisfied or dissatisfied are you with your bank? y 3 Compared to other banks, how satisfied or dissatisfied are you with your bank? y 4 Compared to your expectations, how satisfied or dissatisfied are you with your bank? y 5 How likely is it that you still will remain a customer with this bank one year from today? y 6 How likely is it that you will recommend this bank to a friend? 20

21 Appendix B: Descriptive statistics Descriptive Statistics (mean, st.deviation, skewness and kurtosis) Variable Mean St. Dev. T-Value Skewness Kurtosis x x x x x x x x x x x x x x x y y y y y y

22 Transformed variables (Normal scores) Appendix C: Normal scores Variable Mean St. Dev. T-Value Skewness Kurtosis x x x x x x x x x x x x x x x y y y y y y

23 Appendix D Aish and Jöreskog (1990) analysed a data set on political attitudes. The data set consisted of six measured variables and the total sample size was Permitted responses to the six statements were agree strongly, agree, disagree, disagree strongly, don t know and no answer. The responses were coded 1, 2, 3, 4, 8 and 9. If we ignore that 8 and 9 represent don t know and no answer and define the measured variables to be continuous the six variables show excessive (univariate) kurtosis ranging from 8 to This data set was used to estimate the two factor model in du Toit and du Toit (2000, p. 208) where we got the following results: The ML chi-square was and the ADF chisquare was We then estimated the model using normal scores (univariate kurtosis was close to 3.0 for all measured variables). The ML chi-square was and the ADF (and GLS) chi-square was Olsen et. al (2002) tested a nine items measurement model for the constructs ambivalence, satisfaction and loyalty. The kurtosis of the measured variables ranged from 2.0 to The sample size was The model was estimated both with ML and ADF. The ML chi-square was (df = 24) and the ADF chi-square was (df = 24). The model was also estimated using normal scores 5 (univariate kurtosis ranging from 2.4 to 3.0), and the ML chi-square became while the ADF chi-square was

24 Table 1 THE MEASUREMENT MODELS Exogenous variables: Endogenous variables: x 1 = λx 1 ξ 1 + δ 1 y 1 = λy 1 η 1 + ε 1 x 2 = λx 2 ξ 1 + δ 2 y 2 = λy 2 η 1 + ε 2 x 3 = λx 3 ξ 1 + δ 3 y 3 = λy 3 η 1 + ε 3 x 4 = λx 4 ξ 1 + δ 4 y 4 = λy 4 η 1 + ε 4 x 5 = λx 5 ξ 1 + δ 5 y 5 = λy 5 η 2 + ε 5 x 6 = λx 6 ξ 1 + δ 6 y 6 = λy 6 η 2 + ε 6 x 7 = λx 7 ξ 1 + δ 7 x 8 = λx 8 ξ 2 + δ 8 x 9 = λx 9 ξ 2 + δ 9 x 10 = λx 10 ξ 2 + δ 10 x 11 = λx 11 ξ 2 + δ 11 x 12 = λx 12 ξ 3 + δ 12 x 13 = λx 13 ξ 3 + δ 13 x 14 = λx 14 ξ 3 + δ 14 x 15 = λx 15 ξ 3 + δ 15 24

25 Table 2 WHEN ARE ML, GLS AND WLS EQUIVALENT? Model/Distribution Normal Non-normal Correct model ML GLS WLS asymptotically ML GLS asymptotically. But 2 ( N 1) F is not χ distributed Misspecified model GLS WLS asymptotically No equivalence 25

26 Table 3 PARAMETER ESTIMATES, CHI-SQUARE STATISTICS AND RMSEA VALUES FOR THE ORIGINAL MODEL Original model ML ML GLS MLns* GLSns* Satorra- WLS WLS Bentler cont. ordinal Model Chi-sq RMSEA 0, , ,074 0,042 0,046 Df γ 1 1 0,33 0,33 0,17 0,35 0,33 0,46 0,29 γ 1 2 0,40 0,39 0,55 0,37 0,40 0,33 0,44 γ 1 3 0,26 0,23 0,23 0,23 0,26 0,21 0,28 β *ns = normal scores 26

27 Table 4 PARAMETER ESTIMATES, CHI-SQUARE STATISTICS AND RMSEA VALUES FOR THE MODIFIED MODEL Modified model ML GLS MLns* GLSns* Model Chi-sq RMSEA ,049 0,055 0,046 Df γ 1 1-0,12 (n.s) 0,17-0,18 (n.s) 0,24 γ 1 2 1,00 0,67 1,05 0,59 γ 1 3 0,02 (n.s) 0,10 0,03 (n.s) 0,12 β ,81 0,84 0,84 21 *ns = normal scores 27

28 Figure 1 THE CONCEPTUAL MODEL Branch Loan Satisfaction Loyalty Savings 28

29 Figure 2 THE THEORETICAL MODEL x 1 x 2 x 3 x 4 x 5 ξ 1 y 1 x 6 x 7 η 1 y 2 y 3 x 8 x 9 ξ 2 y 4 x 10 x 11 η 2 y 5 y 6 x 12 ξ 3 x 13 x 14 x 15 29

30 References Aish, Anne-Marie and Karl G. Jøreskog (1990), A Panel Model for Political Efficacy and Responsiveness: An Application of LISREL 7 with Wighted Least Squares, Quality and Quantity, 24, Bearden, William O., Jesse E. Teel and Melissa Crockett (1980), A Path Model of Consumer Complaint Behavior, in Marketing in the 80 s: Changes & Challenges, R. P. Bagozzi, ed. Chicago: American Marketing Association. Blattberg, Robert C. and John Deighton (1996), Managing Marketing by the Customer Equity Test, Harvard Business Review 74 (4), Bolton, Ruth N. and James H. Drew (1991a), A Longitudinal Analysis of the Impact of Service Changes on Customer Attitudes, Journal of Marketing, 55 (January), and ---- (1991b), A Multistage Model of Customers Assessment of Service Quality and Value, Journal of Consumer Research, 54 (April), Breckler, Steven J. (1990), Application of Covariance Structure Modeling in Psychology: Cause for Concern? Psychological Bulletin, 107 (2), Browne, Michael W. (1974), Generalized Least-Squares Estimators in the Analysis of Covariance Structures, South African Statistical Journal, 8,

31 ---- (1984). Asymptotically Distribution-Free Methods for Analysis of Covariance Structures, British Journal of Mathematical and Statistical Psychology, 37, Chou, Chi-Ping and Peter M. Bentler (1995), Estimates and Tests in Structural Equation Modeling, in Structural Equation Modeling - Concepts, Issues, and Applications, R. H. Hoyle, ed., Thousand Oaks: Sage Publications. ----, ---- and Albert Satorra (1991), Scaled Test Statistics and Robust Standard Errors for Non-Normal Data in Covariance Structure Analysis: A Monte Carlo Study, British Journal of Mathematical and Statistical Psychology, Curran, Patrick J., Stephen G. West and John F. Finch (1991), The Robustness of Test Statistics to Nonnormality and Specification Error in Confirmatory Analysis, Psychological Methods, 1 (1), du Toit, Stephen and Mathilda du Toit (2000), Interactive LISREL. User`s Guide. Lincolnwood: Scientific Software International. Fornell, Claes (1989), The Blending of Theoretical and Empirical Knowledge in Structural Equations with Unobservables, in Theoretical Empirisism, H. Wold, ed. New York: Paragon Press (1992), A National Customer Satisfaction Barometer: The Swedish Experience, Journal of Marketing, 56 (1),

32 ----, Michael D. Johnson, Eugene W. Anderson, Jaesung Cha and Barbara Everitt Bryant (1996), The American Customer Satisfaction Index: Nature, Purpose and Findings, Journal of Marketing, 60 (4), Hair, Joseph F., Rolph E. Anderson, Ronald L. Tatham and William C. Black (1995), Multivariate Data Analysis. New Jersey: Prentice Hall. Hu, Li-tze, Peter M. Bentler and Yutaka Kano (1992), Can Test Statistics in Covariance Structure Analysis Be Trusted? Psychological Bulletin, 112 (2), Johnson, Michael D. and Claes Fornell (1991), A Framework for Comparing Customer Satisfaction Across Individuals and Product Categories, Journal of Economic Psychology, 12 (2), , Anders Gustafsson, Tor Wallin Andreassen, Line Lervik and Jaesung Cha (2001), The Evolution and Future of National Customer Satisfaction Index Models, Journal of Economic Psychology, 22 (2), Jøreskog, Karl G. (1993), Testing Structural Equation Models, in Testing Structural Equation Models, K. A. Bollen and J. S. Long, eds. Newbury Park: Sage Publications. ----, Dag Sørbom, Stephen du Toit and Mathilda du Toit (1999), LISREL 8: New Statistical Features. Lincolnwood: Scientific Software International. 32

33 Magnus, Jan R. and Heinz Neudecker (1988), Matrix Differential Calculus with Applications in Statistics and Econometrics. New York: Wiley. Martilla, John A. and John C. James (1977), Importance-Performance Analysis, Journal of Marketing, 41 (1), Oliver, Richard L. (1980), A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions, Journal of Marketing Research, 17 (4), (1981), Measurement and Evaluation of Satisfaction Process in Retailer Selling, Journal of Retailing, 57 (Fall), (1992), An Investigation of the Attributes Basis of Emotion and Related Affects in Consumption: Suggestions For a Stage-Specific Satisfaction Framework, in Advances in Consumer Research, J. Sherry and B. Sternhalt, eds. Provo: Association of Consumer Research. Olsen, Sven Ottar, James Wilcox and Ulf Henning Olsson (2002), Consequences of Ambivalence on Satisfaction and Loyalty, working paper, Texas Tech University. Olsson, Ulf Henning, Tron Foss and Sigurd V. Troye (1998), The Performance of Alternate Estimation Methods in Structural Equation Modeling Under Conditions of Misspecification and Non-Normality, working paper, Norwegian School of Management. 33

34 ----, ---- and ---- (2002), Does the ADF Fit Function Decrease When the Kurtosis Increases? British Journal of Mathematical and Statistical Psychology, Forthcoming. ----, ----, ---- and Roy D. Howell (2000), The Performance of ML, GLS, and WLS Estimation in Structural Equation Modeling Under Conditions of Misspecification and Nonnormality, Structural Equation Modeling, 7 (4), , Sigurd V. Troye and Tron Foss (2001), The Role of Fit Assessment in Structural Equation Modeling: Can Fit Be Compared Across Estimation Methods?, working paper, Norwegian School of Management. ----, ---- and Roy D. Howell (1999), Theoretic Fit and Empirical Fit: The Performance of Maximum Likelihood versus Generalized Least Squares Estimation in Structural Equation Models, Multivariate Behavioral Research, 34 (1), Perreault, William D. and Frederick A. Russ (1976), Physical Distribution Service in Industrial Purchase Decisions, Journal of Marketing, 40 (April), Rust, Roland T., Christine Moorman and Peter R. Dickson (2002), Getting Return on Quality: Revenue Expansion, Cost Reduction, or Both, Journal of Marketing, 66 (4), and Richard L. Oliver (1994), Service Quality: Insights and Managerial Implications From the Frontier, in Service Quality: New Directions in Theory and Practice, R. T. Rust and R. L. Oliver, eds. Thousand Oaks: Sage. 34

35 ---- and Anthony J. Zahorik (1993), Customer Satisfaction, Customer Retention, and Market Share, Journal of Retailing, 69 (2), , ---- and Timothy L. Keiningham (1994), Return on Quality (ROQ): Making Service Quality Financially Accountable. Marketing Science Institute. ----, Valarie A. Zeithaml and Katherine N. Lemon (2000), Driving Customer Equity: How Customer Lifetime Value is Reshaping Corporate Strategy. New York: Free Press. Satorra, Albert and Peter M. Bentler (1988), Scaling Correction for Chi-Square Statistics in Covariance Structure Analysis, The Business and Economic Statistics Section of the American Statistical Association. Schneider, Benjamin, Susan S. White and Michelle C. Paul (1998), Linking Service Climate and Customer Perceptions of Service Quality: Test of a Causal Model, Journal of Applied Psychology, 83 (2), Simon, Julian L. (1974), Interpersonal Welfare Comparisons Can Be Made - And Used for Redistribution Decisions, Kyklos, 27 (1), Westbrook, Robert A. (1980), A Rating Scale for Measuring Product/Service Satisfaction, Journal of Marketing, 44 (Fall), Yuan, Ke-Hai and Peter M. Bentler (1997), Improving Parameter Tests in Covariance Structure Analysis, Computational Statistics & Data Analysis, 26,

36 Zeithaml, Valarie A., Roland T. Rust and Katherine N. Lemon (2001), The Customer Pyramid: Creating and Serving Profitable Customers, California Management Review, 43 (4), In the literature WLS is often referred to as the Asymptotically Distribution Free estimator, ADF. 2 A typical element of this weight matrix is a combination of second and fourth order moments: [ ] ( xi xi )( x j x j )( xg xg )( xh xh ) UWLS = s ij gh ijgh sij s, gh i j g h, where sijgh = is an N estimate of σ ijgh = E{( x i Ex i )( x j Ex j )( x g Ex g )( x h Ex h )}. E being the expectation symbol, and s ij is an estimate of the covariance σ ij. 3 Furthermore, under the same ideal conditions, the minimum of the discrepancy function, F, when multiplied with n(sample size 1) will approximate a central chi-square statistic. 4 The normal score is a monotonic transformation of the original score with the same mean and standard deviation. The rank ordering of the cases are the same as for the original scores. 5 The PRELIS 2.30 (Jøreskog et. al 1999) method for creating normal scores was applied. 36