Chapter Analytical Tool and Setting Parameters: determining the existence of differences among several population means

Transcription

1 Chapter-6 Websites Services Quality Analysis of Data & Results 6.1 Analytical Tool and Setting Parameters: 1. Analysis of Variance (ANOVA): I have shown a relative scenario between his results in the results section. To further analyze the data obtained in my results and ascertain the statistical validity of my findings, I chose the ANOVA method. It is an advanced statistical method. ANOVA is a statistical method for determining the existence of differences among several population means (Complete Business Statistics). The statistical method requires the analysis of different forms of variance associated with the random samples under study. Assumptions: There are two assumptions which are required for conducting ANOVA testing, which are: 1. Independent random sampling from each of the populations is assumed. 171

2 2. It is assumed that the populations under study are normally distributed, with means that may or may not be equal, but will have equal variances. The method is used for testing the validity of hypotheses. The hypothesis test of analysis of variance is as follows: H0: µ1= µ2 = µ3 =. = µr H1: Not all µi (I = 1, 2, 3,, r) are equal. Here, H0 is the null hypothesis, and H1 is the alternative hypothesis. Here r signifies the number of populations under study. An independent random sample is drawn from each of the r populations. The size of the sample from population i (i = 1, 2, 3, ; r) is ni, and the total sample size then would be: n = n1 + n2 + n3 + + nr 2. F Distribution: From the r samples I can compute several different quantities which lead to a computed value of a test statistic that follows a known F distribution when the null hypothesis is true. From the value of the statistic and the critical point for a given level of significance, it is possible to make a determination of whether the r population means are equal (Complete Business Statistics, Amir Aczel, 4 th ed.). Based on the assumptions mentioned above, the test statistic of analysis of variance follows an F distribution when the null hypothesis is true. The F distribution has two kinds of degrees of freedom: degrees of freedom for the numerator, and degrees of freedom for the denominator. 172

3 In the analysis of variance, the numerator degrees of freedom are r-1 and the denominator degrees of freedom are n-r. the value of the test statistic is: ANOVA test statistic = F ( r-1, n-r) A rejection of the null hypothesis through ANOVA testing means there is statistical evidence that not all means are equal. However, it will not tell us in which way they are different. Thus, if the null hypothesis is rejected in some cases, then I have need to conduct further statistical tests. This is beyond the scope of this paper. The level of significance or the right-tailed critical point (á) for our analysis = Analytical Constructs: I have desirable a few more constructs to understand the treatment of ANOVA factor analysis. The F distribution is used in determining differences between two population variances, and the appropriate degrees of freedom for the numerator and the denominator of F come from the degrees of freedom of the sample variance in the numerator and the sample variance in the denominator of the ratio. Here, the numerator is MSTR (mean square treatment) and has r-1 degrees of freedom. The denominator is MSE (mean square error) and has n-r degrees of freedom. Thus, the test statistic in analysis of variance is: F(r-1, n-r) = MSTR / MSE When the null hypothesis of ANOVA is true, all r population means are equal. In terms of the expected values of the mean squares I have 173

4 E (MSE) = ó2 And E (MSTR) = ó2 + Óni (µi µ)2 / (r-1) Where µi is the mean of population i and µ is the combined mean of all r populations. If the null hypothesis is not true and differences do exist among the r population means, then MSTR will tend to be larger than MSE. 6.3 ANOVA Testing on the Population Samples and Hypothesis Testing: In this section, I have run the ANOVA test on all five categories grouped by the two populations to find variance among them, and ascertain the significance of the variance. Here, I shall present the data sets and the corresponding results of the statistical analyses; and describe the significance of the results. 1. Tangibility Data Set: The statistical test has been run through the Excel Data analysis set. Here: Refer: Appendix IV A Sources of variation: The sum of squares for treatment (SSTR) = The sum of squares for error (SSE) = Total (SST) = Mean Squares (MSTR) = SSTR / (r-1) = MSE = SSE / (n-r) =

5 This gives us the F Ratio = MSTR / MSE = / = The corresponding critical value for F is Therefore, we see that the F ratio is higher than the F critical value. This signifies a rejection of our null hypothesis. In terms of my research question, this would indicate that there is significant difference in the way customers perceive the tangibility aspect of the services provided by the two companies in his research. 2. Reliability Data Set: Refer: Appendix IV B Sources of variation: The sum of squares for treatment (SSTR) = 103,08 The sum of squares for error (SSE) = 1232,5 Total (SST) = 1334,51 Mean Squares: (MSTR) = SSTR / (r-1) = 103,08 MSE = SSE / (n-r) = This gives me the F Ratio = MSTR / MSE = / = The corresponding critical value for F is Therefore, I see that the F ratio is higher than the F critical value. This signifies a rejection of my null hypothesis. 175

6 In terms of my research question, this would indicate that there is significant difference in the way customers perceive the reliability aspect of the services provided by the two companies in his research. 3. Responsiveness Data Set: Refer: Appendix IV C Sources of variation: The sum of squares for treatment (SSTR) = 295,84 The sum of squares for error (SSE) = 1141,72 Total (SST) = 1437,56 Mean Squares: (MSTR) = SSTR / (r-1) = 295,84 MSE = SSE / (n-r) = 11,6502 This gives us the F Ratio = MSTR / MSE = 295,84 / 11,6502 = 25,39355 The corresponding critical value for F is Therefore, I see that the F ratio is higher than the F critical value. This signifies a rejection of his null hypothesis. In terms of my research question, this would indicate that there is significant difference in the way customers perceive the responsiveness aspect of the services provided by the two companies in my research. 4. Credibility Data Set: Refer: Appendix IV D Sources of variation: 176

7 The sum of squares for treatment (SSTR) = 6,76 The sum of squares for error (SSE) = 480,4 Total (SST) = 487,16 Mean Squares: (MSTR) = SSTR / (r-1) = 6,76 MSE = SSE / (n-r) = 4, This gives me the F Ratio = MSTR / MSE = 6,76 / 4, = 1, The corresponding critical value for F is Therefore, I have see that the F ratio is lower than the F critical value. This signifies an acceptance of my null hypothesis. In terms of my research question, this would indicate that there is no significant difference in the way customers perceive the credibility aspect of the services provided by the two companies in my research. 5. Security Data Set: Refer: Appendix IV E Sources of variation: The sum of squares for treatment (SSTR) = 2,89 The sum of squares for error (SSE) = 758,5 Total (SST) = 761,39 Mean Squares: (MSTR) = SSTR / (r-1) = 2,89 MSE = SSE / (n-r) = 7,

8 This gives me the F Ratio = MSTR / MSE = 2,89 / 7, = 0, The corresponding critical value for F is Therefore, I have see that the F ratio is lower than the F critical value. This signifies an acceptance of his null hypothesis. In terms of my research question, this would indicate that there is no significant difference in the way customers perceive the security aspect of the services provided by the two companies in my research. 6.4 Experimental Study of Website Quality Efficiency To show the Data value of the E-commerce website Qualities evaluation Analysis I have measure website Quality Design efficiency, and generate insights about the effectiveness of different website designs Pattern. 1. Data. Click stream data was collected directly from an online grocer s web servers. The website uses HTTP cookies downloaded onto the visitor s computer to track the customer s shopping behavior at the site. Typical data pre-processing procedures for using web server logs were used to extract navigation path sequences for each individual visitor from the click stream data (Cooley et al., 2009). From the navigation sessions, website usage metrics were extracted to measure the extent to which various areas of the E- commerce website was used in the purchasing process. My data spans two weeks from April 21 to May 04, In this time, a total of sessions were recorded by unique customers. The analysis will focus on 6387 actual completed purchasing transactions from 9949 customers. I selected this period for analysis because there was a design 178

9 change in the middle: the homepage of the website was changed. I have illustrated the value of the proposed methodology by comparing the efficiencies of the two website Qualities designs. 2. Measures. When using DEA, the subsequent analysis is only as good as the initial selection of input and output variables. The input and output variables are to be selected such that the inputs represent the resources consumed by the DMUs and the outputs represent the performance of the DMUs (see Table 6.1). Category Var Measure Description X1 Products Number of product page views X2 Lists Number of product lists views X3 Personal Number of personal list views X4 Order history Number of orders history page views Input X5 Search Number of search conducted X6 Promotion Number of promotional page views X7 Recipe Number of recipe page views X8 Checkout Number of checkout pages X9 Help Number of help page views Output Y1 Basket size Number of items at checkout 179

10 Table 6.1 Showing the Input and Output Variables for Website Efficiency Measurement 3. Results. I have preliminary results of the efficiency estimation (see Table 6.2, Figure 6.1). There is significant variability in efficiency at various levels of output. I have compute inefficiencies (i.e., customer and website inefficiencies) for each of the inefficient DMUs (see Figure 6.2 s histogram of website design inefficiency scores.). Interestingly, the website design inefficiency scores for Week1 vs. Week2 show stark differences. For Week2, more DMUs show less inefficiency due to design: any inefficiency would be more likely due to poor execution from the customers. For Week1, a reversal occurs: there are a large proportion of users where all sources of inefficiency are due to website design, not poor execution. These results suggest Week1 s design outperforms Week2 s changed design. I investigate this issue for a subset of customers transacting both weeks. Efficiency should be stable across multiple transactions an individual (e.g., stable browsing patterns). If I can track how an individual s efficiency changed as a result of different website designs, I have may be obtain insights beyond the overall efficiency statistics. I identified 398 customers that transacted during Weeks 1 and 2. Figure 6.3 shows efficiency score changes from Week1 to Week2 (X to Y-axis). I have showed the diagonal to delineate efficiency increases and decreases. I plot a dashed regression line through the data to see general tendencies of the efficiency score changes. When I have examine different levels of initial efficiency scores (i.e., scores for Week1 website design), I see that the shift is different at different 180

11 initial efficiency levels. Table 6.3 shows the details of the efficiency score changes. This data is graphed in Figure 6.4, which shows the proportion of increasing/decreasing/stable efficiency scores at different efficiency levels (see Table 6.3, Figure 6.4). The results suggest that at low levels of efficiency, the Week2 website design increased customer efficiency. However, as I move up the efficiency levels, I have seen a greater proportion of decreasing efficiencies. This is consistent with the regression results that suggest increases at lower efficiency levels and decreases at higher efficiency levels. I infer from this that Week2 website redesign was more effective for novice/inexperienced shoppers, however, this also had a negative impact on experienced shoppers, who became less effective. 181

12 Efficiency EfficiencyDesign1 EfficiencyDesign2 Overall Minimum Maximum Mean Std Deviation st_Quartile Median rd_ Quartile Total Table 6.2. Showing Efficiency Scores Summary Statistics Output Efficiency Figure 6.1. Showing Website Qualities Efficiency Scores by Output 182

13 Note: Visual inspection gives a summary of Qualities overall website efficiency. If the efficiency marks lie on graph s right edge, the website is highly effective. The current plot, which shows variability at all levels of outputs, suggests website may be ineffective % 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Week1 Week2 Figure 6.2. Showing Histogram of Website Quality Design Inefficiency Scores Note: Website design and customer inefficiency are complementary: 0% design inefficiency corresponds to 100% customer inefficiency. Leftskewed bars mean inefficiency lies with customers not website design. Rightskewed bars mean most measured inefficiency is attributable to design. 183

14 1 Week Week 1 Figure 6.3. Showing Websites Efficiency Score Changes for Paired DMUs Note: Websites Efficiency marks below the diagonal are efficiency decreases, those above are efficiency increases. The regression line can be used to identify efficiency shift tendencies. By comparing with the diagonal, we see for which initial efficiency score range we efficiency increases through redesign (i.e., regression line above diagonal) or decreases (i.e., line below diagonal). 184

15 Website Qualities Efficiency Website Qualities Score Change for Design 2 Total Efficiency, Design 1 Increase Decrease No Change Total Table 6.3. Showing Website Qualities Efficiency Score Changes 185

16 100% 50% 0% Increase Decrease No Change Figure 6.4. Showing Website Qualities Efficiency Score Changes Discussion I have use the theoretical perspective of production economics to measure performance of E-commerce and Internet-based selling websites. I have given model the website as a production information system where customers consume inputs (i.e., use various functionalities of the website) to produce an output (i.e., a basket full of items at checkout). With this basic perspective, my propose an analysis methodology for measuring E-commerce website Qualities efficiency and attributing observed inefficiency to customer inefficiency or website design inefficiency. The application of the proposed evaluation data Analysis to a currently operational e-commerce website demonstrates the value of our technique. I have able to identify several patterns of relationships (e.g., differential impact of redesign for different levels of customer efficiencies) that would have been otherwise difficult (or impossible) to observe with currently available evaluation techniques. One 186

17 limitation to our approach stems from the fact that it is data-driven. Hence, one cannot evaluate a priori whether a particular redesign will be effective. Rather, the intent of the methodology is for post hoc analysis to estimate the impact and nature of Website Qualities change. Several venues for extending the current work are readily apparent. First, given that I may observe recurring transactions by customers over time, deterministic frontier analysis that I have employ needs to be extended to stochastic frontier analysis. Since each customer will have multiple measures of website Qualities efficiency over time, I have need to take into consideration, the measurement and random error components of the efficiency ratings. This would, of course, require a larger sample with many more recurring transactions than the limited sample illustrated in this paper. I have currently in the process of collecting a longitudinal sample of weblog data in order to develop this further. Second, DEA, including those that compare subgroups, assumes a between-group orientation. In other words, a DMU is typically in one subgroup exclusively. However, in our context, the DMUs can be observed in all subgroups. Hence, I have also need to extend the DEA method to better fit these circumstances to solve the website Qualities. And I can also use ANOVA test. 6.5 ANOVA Testing within Population between the Groups: Here I have run a further ANOVA test within the same population between the five service attributes. This will help me identify whether there is any significant co-relation between the components themselves which would indicate an influencing behavior. 187

18 Refer: Appendix V A For Amazon.com: As I can see from the test, the F ratio is which is higher than the F critical value of Thus I conclude there is no significant relationship within the groups. Refer: Appendix V B For Ebay.com: The F ratio for Ebay.com is ; significantly higher than the F critical value of Thus, I conclude that here also there is no significant co-relation between the components within the group. One point to be noted is that the F ratio for Ebay.com is significantly higher (about 3 times) than that of Amazon.com this may indicate a more cohesiveness within the group attributes in case of Amazon.com compared to Ebay.com ******* 188