Performance of Efficiency Measurement Models a Study of Brand Advertising Efficiency in the German Car Industry

Transcription

1 1 KATHOLISCHE UNIVERSITÄT EICHSTÄTT-INGOLSTADT WIRTSCHAFTSWISSENSCHAFTLICHE FAKULTÄT INGOLSTADT Performance of Efficiency Measurement Models a Study of Brand Advertising Efficiency in the German Car Industry Dr. Joachim Büschken Professor of Marketing Marketing Department Catholic University, Germany Ingolstadt, Homepage: CU Working Paper #164 [in progress: please do not cite without author s permission]

2 1 Performance of Efficiency Measurement Models a Study of Brand Advertising Efficiency in the German Car Industry Abstract This paper compares three different models of efficiency measurement on the basis of multiple inputs and outputs. Various data envelopment models are compared with principle component analysis and with a naïve model that derives a single efficiency score for every unit by averaging all possible output-input ratios. The data set for this comparison consists of multiple, media-specific brand advertising expenditures of 21 car makers in the German car market for the years and the resulting advertising effects. The analysis shows that DEA on the basis of a constant return to scales model, principal component analysis and the naïve model, produce almost identical results in terms of the overall advertising efficiency of each brand. It is argued that this similarity is due to the linear approach of these models. Differences are found between VRS and CRS models. This findings suggests that the results of efficiency measurement depend on the use of variable or constant-return-to-scale models and that researchers must select carefully between VRS and CRS according to the specifics of the underlying production function. 1. Introduction In recent literature, there is a growing body of research on efficiency measurements methods. Two streams of research can be identified: non-parametric data envelopment analysis (for a bibliography on DEA see Seiford 1994) and stochastic frontier analysis (SFA) which is a derivate of parametric linear regression. Despite fundamental differences in their approach, both DEA and SFA provide a single aggregate efficiency measure. Originally proposed by Charnes, Cooper and Rhodes (1978), DEA is based on production possibility sets constructed by the observed cases (so called decision making units : DMU). The production possibility set is a convex space consisting of all DMUs and their linear combinations in input-output space. The position of each DMU in this space is identified by finding DMU-specific input and output weights that maximize the combined output-input ratio for every DMU. This is achieved through linear optimization. Efficiency is measured as the vertical (output orientation) or horizontal (input orientation) Euclidian distance of DMUs to the efficiency frontier. The efficiency frontier is the section of the envelop of the production possibility set with a non-negative slope. Since the efficiency frontier is constructed only from

3 2 and possibly only a few efficient DMUs, it is very sensitive to outliers, but very flexible with regard to the frontier s shape. In contrast, SFA is a parametric approach in which the production frontier is estimated simultaneously from all cases. The original model was proposed by Aigner, Lovell and Schmidt (1977) and Meeusen and Van den Broeck (1977). It has a combined error term, one to account for random error and the other to measure technical inefficiency (Coelli, Rao and Battese 1998, Kumbhakar and Lovell 2000). As a parametric method, SFA requires an a priori assumption about the shape of the efficiency frontier, but allows for random influences on DMU efficiency. With DEA, random influence does not exist and consequently the whole of a DMUs distance to the efficiency frontier in input-output-space is interpreted as inefficiency. DEA has been applied to numerous efficiency measurement problems. E.g., there are applications to marketing research concentrating on advertising efficiency (Luo and Donthu 2001, Hershberger, Osmonbekov and Donthu 2001, Tanaka, Takeda, and Nakajima 2002) and distribution efficiency (Kamakura et al. 1998, Horsky and Nelson 1996, Thomas, Barr, Cron, and Slocum 1998, Donthu and Yoo 1998, Hershberger, Osmonbekov and Donthu 2001, Ross and Dröge 2002). The empirical application of SFA is limited primarily to microeconomic issues (e.g. Liu and Zhuang 1998). SFA cannot deal with multiple outputs, which are prevalent in business or economics. It also assumes that the production model (the linear regression model) is the same for all DMUs which may not be true. Hirshberger, Osmonbekov and Donthu (2001) conclude that DEA is better suited to evaluating management performance, because of the inflexibility of the SFA model. Although rigorous and based on inference theory, SFA is limited to linear efficiency frontiers and estimates efficiency in relation to average performance. However, response functions in management are often non-linear (such as in advertising: see Gopalakrishna and Chatterjee 1992, Lodish et al. 1995). In contrast to regression, DEA also identifies specific DMUs that serve as a benchmark. Thus, DEA seems more favorable to measure efficiency, compared to SFA. Zhu (1998) recently proposed principal component analysis (PCA) as an alternative to efficiency measurement. PCA presents another means of estimating a single efficiency measure on the basis of multiple inputs and outputs. It is also based on output-input ratio analysis (Chen and Ali 2002). Firstly, for every DMU, all possible output-input ratios are calculated. For n inputs and m outputs, n m ratios can be calculated for every DMU. Secondly, factor analysis is used to combine multiple output-input ratios. This can be done in a single step if factor analysis finds a single factor which can be interpreted as a combined efficiency measure. For several factors, their Eigenvalues are used to weight the factor scores derived from the ratios. Using PCA with real life data, Zhu (1998) reports efficiency scores that are correlate highly to DEA results. Based on his results, Zhu s study suggests that the lack of statisti-

4 3 cal rigor of DEA compared to PCA is irrelevant with regard to the final result (the DMUs overall efficiency score). Another more simple means of producing a single DMU-specific efficiency score is to average all possible output-input ratios. Instead of performing factor analysis we could simply calculate the arithmetic mean of the ratios. With this approach, the ratios would be weighted equally. Zhu s study suggests that DMU-specific optimal input and output weights are not necessary to estimate efficiency accurately. Instead, factor weights derived from the data set suffice to combine the DMU-specific factor scores. The aim of this study is to analyze the performance of a naïve model in comparison to DEA and PCA. A naïve model needs no weights for inputs and outputs. If performing adequately, it would eliminate the need the use of elaborate linear programming or of statistical methods if the researcher is interested only in establishing a DMUs efficiency. 2. Efficiency Measurement Comparison of Efficiency Measurement Methods The efficiency of a certain DMU in the case of multiple inputs and outputs is established by combining all inputs and outputs into a single efficiency ratio. This ratio reflects a DMUs ability to transform all inputs into all outputs simultaneously. The higher this ratio of outputs to inputs is the more efficient is the DMU. Efficiency measurement methods differ in their approach to combine multiple inputs and outputs: Data Envelopment Analysis (DEA) uses linear programming technique to find DMUspecific optimal weights that maximize the DMUs overall efficiency score. Optimal input and output weights define the DMUs position in input-output space. If that position is on the envelope of the production possibility set, the DMU is defined as efficient. If not, the distance from that position to the envelope gives the DMUs efficiency in relation to efficient benchmarks. For an excellent overview of the mathematical formulation of this model the reader is referred to Cooper, Seiford and Tone (2000). As suggested by Zhu (1998), PCA finds a linear combination of all possible outputinput ratios. It builds on the fact that a DMUs efficiency can be observed with every single output-input-ratio. Thus, every ratio represents some facet of DMU efficiency. If ratios correlate, factor analysis finds independent factors whose Eigenvalues can be used to identify a combined efficiency score through a linear model. In contrast to DEA which calculates optimal weights for inputs and outputs separately, PCA combines individual input-output ratios in a linear fashion. Weights are allocated to the ra-

5 4 tios, not separately to inputs and outputs. For a detailed description of this method, see Zhu (1998). The applicability of PCA to efficiency measurement is contingent upon a data set s suitability for factor analysis. Factors derived must explain a significant portion of all ratio variance. An alternative approach would be to identify all possible output-input ratios as in PCA and then simply to calculate their arithmetic mean. This method not only eliminates the need to assign weights for inputs and outputs ex ante (as in DEA), it would also eliminate the need to perform factor analysis to find linear combinations of the ratios. Here, we do not describe the formal properties of DEA and PCA. For this, the reader is referred to the literature. The remainder of this study is dedicated to establishing the performance of the proposed naïve model compared to DEA and PCA. It is interested only in the similarity of efficiency score results. Note that DEA is capable of producing optimal input and output weights that can be interpreted as their marginal utility or marginal productivity. It also identifies input excesses and output shortfalls and information concerning a DMU s returns to scale (Seiford and Zhu 1999). Thus, DEA produces additional information that can be valuable to managers. Neither PCA nor the proposed naïve model provides that depth of information. Data Luo and Donthu (2001) demand that for DEA, inputs and outputs should correlate with respect to efficiency measurement. Indeed, meaningful efficiency measurement is only possible if a causal relationship between inputs and outputs can be assumed. There must be an underlying production function transforming inputs into outputs. E.g., in marketing research it is widely assumed that this is the case with advertising (input) to create some communication effect (output). This effect refers to communication goals such as brand awareness or brand reputation (Luo and Donthu 2001) or financial goals such as sales (e.g. Gopalakrishna and Chatterjee 1992, Lodish et al. 1995). Thus, in accordance with marketing research results for this study, the underlying production model is that advertising spending is transformed into a communication effect, i.e. the share of potential buyers considering a brand for their next purchase. This effect is contingent upon a DMUs ability to perform that transformation efficiently. The empirical study is based on data from 21 brands in the German car market. All brands combined in this study represent a market share of more than 70% in Germany for For all brands, the following media advertising budgets (in ) were obtained from A.C. Nielsen for the years : TV, radio,

6 5 outdoor, magazine, newspaper. For simplicity, newspaper and magazine advertising were combined to the input print. This leads to a total of 4 inputs. Because brand awareness shows little variance (it is high for many brands) this output was not considered for analysis. Output data were obtained from Stern magazine for the same period (the so called Markenprofile study, see This popular German news magazine conducts an annual analysis of brand profiles in various German industries on the basis of extensive consumer samples. E.g., for the 2001 Markenprofile study a representative sample of respondents were interviewed. Results and process of data collection are similar to J.D. Power in the U.S. (see Table 1 shows brand specific input and output data (averages over 1998 to 2001). As demanded by Luo and Donthu (2001) inputs and outputs are positively correlated. Regression analysis shows that the four inputs explain more than 75% of the variance of brand consideration. 1 Brand Consideration Print TV Radio Outdoor Audi 26,8 38,69 35,29 1,36 2,15 BMW 24,8 36,97 24,19 3,76 3,87 Citroen 3,5 25,42 25,23 8,52 9,25 Daewoo 1,2 5,27 5,54 1,58 1,62 Fiat 6,5 46,13 25,05 2,88 3,93 Ford 21,7 62,01 41,65 11,07 12,47 Kia 1,0 6,29 3,25 0,52 0,53 Mazda 7,7 24,16 9,93 5,11 6,05 Mercedes-Benz 20,5 81,27 27,22 6,90 7,64 Mitsubishi 6,0 19,68 16,83 3,53 3,54 Nissan 7,3 24,88 9,59 8,62 10,47 Opel 34,3 75,38 63,77 8,89 9,79 Peugeot 8,7 44,30 32,72 4,37 4,88 Porsche 4,2 5,03 0,92 0,10 0,02 Renault 11,8 68,46 53,06 12,30 15,03 Saab 2,2 10,57 0,03 0,00 0,55 Seat 3,8 12,09 10,26 2,05 2,13 Skoda 3,3 9,95 10,39 1,20 1,22 Toyota 8,8 43,71 18,71 4,68 5,07 Volvo 6,0 14,62 14,57 2,97 3,89 VW 53,0 92,39 62,75 7,24 13,10 Table 1: Input and Output Data (for brand consideration data is given in %, for inputs in thousand, yearly averages) Thus, the analysis is based upon 1 output and 4 inputs. As a result, 4 separate output-input ratios are possible for each brand. The possibility a brand (Saab) has not used a certain input

7 presents a problem for calculating ratios. For such cases, certain ratios do not exist. Missing values in the data set for PCA are the result. 6 Comparison of Efficiency Measurement Models Following Zhu s (1998) approach, we compare PCA with DEA. However, we consider various DEA models and the proposed naïve model as well. The following DEA models were used: input- and output-oriented models, models with constant (CRS) and variable returns to scale (VRS), super-efficiency models. With a small set of cases, many DMUs can be efficient. Theoretically, the super-efficiency model developed by Andersen and Petersen (1993), differentiates between efficient cases by excluding the DMU under observation from the constraints. Zhu s (1998) study only considered an input-oriented CRS super-efficiency model. Table 2 shows the efficiency scores derived from the efficiency measurement models. Table 3 shows the corresponding efficiency ranks. PCA Naïve VRS-O CRS-O SCRS-O SVRS-O CRS-I VRS-I Audi 0,290 0,342 1,000 0,836 0,801 0,907 0,836 1,000 BMW 0,259 0,308 1,000 0,810 0,795 0,830 0,810 1,000 Citroen 0,000 0,000 0,196 0,166 0,000 0,000 0,166 0,198 Daewoo 0,038 0,046 0,269 0,267 0,378 0,274 0,267 0,953 Fiat 0,007 0,007 0,274 0,170 0,023 0,287 0,170 0,184 Ford 0,100 0,119 0,577 0,421 0,606 0,661 0,421 0,515 Kia 0,016 0,017 0,199 0,192 0,133 0,018 0,192 0,799 Mazda 0,086 0,102 0,630 0,383 0,566 0,690 0,383 0,490 Mercedes-Benz 0,062 0,073 0,758 0,304 0,454 0,742 0,304 0,709 Mitsubishi 0,079 0,094 0,428 0,368 0,548 0,543 0,368 0,394 Nissan 0,074 0,089 0,618 0,355 0,533 0,684 0,355 0,468 Opel 0,152 0,181 0,768 0,549 0,698 0,745 0,549 0,718 Peugeot 0,031 0,036 0,299 0,236 0,296 0,347 0,236 0,264 Porsche 0,837 0,898 1,000 1,000 1,000 1,000 1,000 1,000 Renault 0,017 0,021 0,286 0,208 0,203 0,317 0,208 0,240 Saab 1,000 1,000 1,000 1,000 0,988 1,000 1,000 1,000 Seat 0,084 0,101 0,429 0,382 0,566 0,545 0,382 0,416 Skoda 0,095 0,113 0,446 0,404 0,589 0,561 0,404 0,505 Toyota 0,035 0,041 0,442 0,244 0,319 0,558 0,244 0,330 Volvo 0,125 0,150 0,565 0,495 0,664 0,654 0,495 0,530 VW 0,209 0,249 1,000 0,692 0,760 0,879 0,692 1,000 Table 2: Efficiency Scores 2

8 7 PCA Naïve VRS-O CRS-O SCRS-O SVRS-O CRS-I VRS-I Audi BMW Citroen Daewoo Fiat Ford Kia Mazda Mercedes-Benz Mitsubishi Nissan Opel Peugeot Porsche Renault Saab Seat Skoda Toyota Volvo VW Table 3: Efficiency Ranks Already from the visual inspection of efficiency ranks, it is apparent that all models produce similar or even identical results. We can employ statistical tests to the relationship between efficiency ranks. Table 4 shows rank correlation (Kendall s Tau) between efficiency ranks derived from the various models. All correlation coefficients are significant at the 0.01 level or above. PCA Naïve VRS-O CRS-O SCRS-O SVRS-O CRS-I Naïve 1,000 VRS-O 0,751 0,751 CRS-O 0,998 0,998 0,753 SCRS-O 0,990 0,990 0,751 0,998 SVRS-O 0,768 0,768 0,978 0,770 0,768 CRS-I 0,998 0,998 0,753 1,000 0,998 0,770 VRS-I 0,673 0,673 0,640 0,675 0,673 0,626 0,675 Table 4: Rank Correlation between Efficiency Rankings (Kendall s Tau)

9 8 Results Several results are noteworthy: 1. The naïve model and PCA produce identical results. Thus, factor analysis does not outperform the simple averaging of the separate output-input ratios. Considering the elaborateness of factor analysis compared to the simple approach of the naïve model, this result is surprising. 2. DEA models with constant returns to scale produce efficiency rankings that correlate very highly (r 0.99) with results from PCA and the naïve model. This result is independent of the model orientation (input vs. output) or the super-efficiency property. A closer look at the efficiency rankings shows that with DEA (CRS), a maximum of only two brands (Porsche, Saab) is ranked differently compared to the naïve model. 3. Simple DEA-models correlate highly with their corresponding super-efficiency models. It seems that the ability to differentiate between efficient DMUs does not provide significant additional information in terms of the overall brand ranking. Note that with VRS-O, 5 brands are efficient (Audi, BMW, Porsche, Saab, and VW) and only 2 with CRS-O (Porsche and Saab). Thus, even a higher number of efficient cases does not lead to significant differences in the efficiency ranking. 4. VRS and CRS models produce efficiency rankings that correlate weakly. With those types of models, we observe rank correlations of only 0.64 to Comparing VRS-O with CRS-O, 17 out of 21 brands are differently ranked. For some brands the rank difference are surprising. E.g., for Mercedes-Benz this difference is 7 (number of ranks) and 4 for Korean importer Daewoo and the German car maker Volkswagen. The high performance of the simple naïve model is surprising. Note that analogous to the PCA approach, the naïve model provides information about the determinants of overall efficiency through an analysis of the separate output-input ratios (see Table 5). CONS/Print CONS/TV CONS/Radio CONS/Outdoor Rank Rank Rank Rank Overall (1) (2) (3) (4) (1) (2) (3) (4) Rank Audi 1, , , , BMW 1,664-0,1936-0, , Citroen -1, ,2576-0, , Daewoo -0, , , , Fiat -1, ,2489-0,2327-0, Ford 0, , , , Kia -0, , , , Mazda -0, , ,2364-0, Mercedes- Benz -0, , , , Mitsubishi -0, ,2419-0, , Nissan -0, , , ,

10 Opel 0, , , , Peugeot -0, , , , Porsche 2, , , , Renault -0, , , , Saab -0, , , , Seat -0, , , , Skoda -0, , , , Toyota -0, , ,2345-0, Volvo 0, , , , VW 1, , , , Table 5: Data for the Naïve Model Table 5 presents all output-input ratios for each brand. The ratios are standardized to facilitate comparison. On that basis, ratio-specific ranks are calculated. The overall rank is derived from the arithmetic mean of the standardized ratios. For that reason, the overall rank can differ from the mean of the separate output-input-ratio ranks. A closer look reveals that the weakest performer according to this model (Citroen) shows dismal efficiency on all ratios. It ranks consistently at last for all 4 ratios. Fiat, ranked 20, shows relatively high efficiency on the Consideration/Radio variable (ranked 9), indicating that the efficiency problems lie elsewhere in this case. This also applies to Kia (ranked 19), which performs quite well on Consideration/Outdoor (ranked 9). Several other cases show a significant amount of variation of ratio-specific ranking: Highly ranked Audi (ranked 3) shows some inefficiency with regard to the input TV. For that input, it is only ranked 7. Even more inefficiency can be observed for Saab (overall ranked 1), which shows very poor efficiency for the input print media (ranked only 15). This lack of performance is compensated by very high ratios for TV and radio. The same efficiency problem with print input applies to the brand Mercedes-Benz. This demonstrates that even the naïve model can supply valuable information beyond an overall efficiency ranking similar to PCA. One might question whether the high similarity of PCA and the naïve model holds when different data are used. Therefore, the comparison of the two models was repeated on the basis of the data provided by Zhu (1998) and Ali, Lerme and Seiford (1995). Both studies provide raw data which enables us to check the consistency of our findings for other data sets. Rank correlation between PCA and the naïve model is r=0.987*** for the Zhu data and r=0.891*** for the Ali et al. data. Although similarity between the two models is lower than with the data used for this study, we must conclude that the high similarity of efficiency scores derived from PCA and the naïve model is robust. Additionally, Zhu s (1998) conclusion that DEA and PCA produce very similar results, applies only to DEA models with the constant-return-to-scale property (CRS). VRS models result in efficiency rankings that are show relatively small similarity to DEA-CRS, PCA, or the naïve model. 9

11 10 This clearly is an interesting and relevant finding. It seems that efficiency measurement models fall under two categories. The first category encompasses linear models based on the assumption of CRS. DEA-CRS, PCA and the naïve models all fall under this category. This is because PCA and the naïve model derive efficiency scores directly from output-input ratios. DEA models combine outputs and inputs through linear optimization and then measure the distance of each DMU to the efficiency frontier in output-input space to establish its efficiency score. With DEA-CRS, we employ a linear model for the production frontier. The same holds for PCA and the naïve model which simply combine the separate output-input ratios and then compare them in a linear fashion by establishing the linear difference between the combined ratios. The second type of model is based on the VRS assumption. We may call that the non-linear model. Both PCA and the naïve model cannot satisfy this assumption because of their inherent linear approach. In contrast, VRS is a non-linear model. With this model, the combined output-input ratio does not need to increase proportionally for a DMU to be efficient. This fundamental difference between VRS and CRS (including PCA and the naïve model) seems to be a central issue in efficiency measurement, because it can result in very different efficiency measurements. 3. Conclusion This study compares three alternative approaches to efficiency measurement on the basis of output-input ratios. Real data from the German automobile industry was used to analyze the consistency of advertising efficiency scores derived from DEA, PCA and a naïve model. The empirical results suggest that, with the important exception of DEA models with the variable return-to-scale property, the choice between the alternatives is largely irrelevant. The naïve model and PCA produce identical efficiency rankings. This ranking shows high similarity to DEA scores and DEA (on the basis of CRS). The DEA model orientation (input vs. output) is of minor importance. This result is robust and was established on the basis of three data sets. Note that this study used real data from marketing, whereas Zhu (1998) used real microeconomic data (economic performance of cities). There are fundamental differences between the underlying production functions which allows us to conclude that such differences are not relevant with regard to efficiency scores derived from linear models. Note that no linear model forces researchers to assign weights to multiple inputs and outputs. This is often considered as a major advantage of DEA (Cooper, Seiford and Tone 2000). However, considering our own results, we conclude that this advantage seems to be minor. Table 6 presents the optimal weights for the inputs for the CRS-I model.

12 11 Print TV Radio Outdoor Audi 0,0258 0,0000 0,0000 0,0000 BMW 0,0270 0,0000 0,0000 0,0000 Citroen 0,0393 0,0000 0,0000 0,0000 Daewoo 0,1897 0,0000 0,0000 0,0000 Fiat 0,0217 0,0000 0,0000 0,0000 Ford 0,0161 0,0000 0,0000 0,0000 Kia 0,1589 0,0000 0,0000 0,0000 Mazda 0,0414 0,0000 0,0000 0,0000 Mercedes-Benz 0,0123 0,0000 0,0000 0,0000 Mitsubishi 0,0508 0,0000 0,0000 0,0000 Nissan 0,0402 0,0000 0,0000 0,0000 Opel 0,0133 0,0000 0,0000 0,0000 Peugeot 0,0226 0,0000 0,0000 0,0000 Porsche 0,1582 0,1995 1,0343 0,8462 Renault 0,0146 0,0000 0,0000 0,0000 Saab 0,0675 1,7048 0,5071 0,4149 Seat 0,0827 0,0000 0,0000 0,0000 Skoda 0,1005 0,0000 0,0000 0,0000 Toyota 0,0229 0,0000 0,0000 0,0000 Volvo 0,0684 0,0000 0,0000 0,0000 VW 0,0108 0,0000 0,0000 0,0000 Table 6: Optimal Weights for the CRS-I Model The optimal weights (not only) for the CRS-I model are strictly non-negative (a property achieved by non-zero conditions for the optimization), but are in many cases zero. This result is often observed with DEA. This may indicate that the marginal productivity of such inputs is zero. With PCA and the naïve model, weights are assigned to ratios, not inputs. PCA derives the weights from the Eigenvalue of the factors; the naïve model simply assigns the separate ratios equal weights (the inverse of the number of ratios). For both models these weights are strictly positive: they cannot be zero. In practice, this is a very different weighting approach to DEA, but the impact on the efficiency scores is marginal. This leads us to conclude that the property of the production model (linear vs. non-linear) is more important than the weighting of inputs and outputs. Also, Zhu s (1998) conclusion that DEA s strength in handling multiple inputs and outputs simultaneously, also applies to PCA and the naïve model, without sacrificing accuracy of the efficiency score. Lastly, we conclude that researchers interested in efficiency measurement, must select carefully between VRS or CRS models. The impact of this choice with regard to efficiency scores is significant. The efficiency measured, very much depends on the assumption about the production model. Empirical research in marketing has shown that advertising response functions are non-linear (e.g. Gopalakrishna and Chatterjee 1992, Lodish et al. 1995), for example.

13 12 They typically show decreasing returns to scale. Under such circumstances, the use of a CRS model seems inappropriate. Such a model would assume that high market share brands with large budgets, operate under the same circumstances as niche brands with smaller advertising budgets. However, in reality this is not the case. Such empirical or, alternatively, theoretical considerations should guide researchers in choosing the appropriate model for efficiency measurement. Many production or market-response functions in management, show variable returns to scale. For example, increasing returns to scale apply to many real cost functions. Decreasing returns to scale apply to many market- response functions. The underlying reason is that consumers have decreasing marginal utility from certain product attributes. This study shows that if the underlying production model is non-linear, the group of models identified here as linear (CRS-DEA, PCA and the naïve model) is inappropriate. Researchers cannot expect these models to provide efficiency scores similar to those of appropriate models.

14 13 References: Aigner, D.J., Lovell, C.A.K. and P. Schmidt (1977): Formulation and Estimation of Stochastic Frontier Models, Journal of Econometrics, 6, Ali, A.I.,. Lerme, C.S. and L. M. Seiford (1995): Components of efficiency evaluation in data envelopment analysis, European Journal of Operational Research, 80, Andersen, P. and Petersen, N.C. (1993): A Procedure for Ranking Efficient Units in Data Envelopment Analysis, Management Science, 39., Charnes, A., Cooper, W.W. and E. Rhodes (1978): Measuring the Efficiency of Decision Making Units, European Journal of Operational Research, 3, Chen, Y. and Ali, A.I. (2002): Output-input Ratio Analysis and DEA, European Journal of Operational Research, 142, Coelli, T., Rao, D.S. and G.E. Battese (1998): An Introduction to Efficiency and Productivity Analysis, Kluwer Academic Publishing, Boston at al. Cooper, W.W. and Tone, K. (1997): Measures of Inefficiency in Data Envelopment Analysis and Stochastic Frontier Estimation, European Journal of Operational Research, 99, Cooper, W.W., Seiford, L.M. and K. Tone (2000): Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software, Kluwer Academic Publishing, Boston at al. Donthu, N. and Yoo, B. (1998): Retail Productivity Assessment: Using Data Envelopment Assessment, Journal of Retailing, 74, Gopalakrishna, S. and Chatterjee, R. (1992): A Communication Response Model for a Mature Industrial Product, Journal of Marketing Research, 29, Hershberger, E.K., Osmonbekov, T. and N. Donthu (2001): Benchmarking Marketing Performance, Working Paper, January 21 st 2001, Georgia State University. Horsky, D. and Nelson, P. (1996): Evaluation of Salesforce Size and Productivity through Efficient Frontier Benchmarking, Marketing Science, 15, Kamakura, W.A., Ratchford, B.T. and J. Agarwal (1998): Measuring Marketing Efficiency and Welfare Loss, Journal of Consumer Research, 15, Kumbhakar, S.C. and Lovell, C.A.K (2000): Stochastic Frontier Analysis, Cambridge University Press 2000 Liu, Z. and Zhuang, J. (1998): Evaluating Partial Reforms in the Chinese State Indsutrial Sector: A Stochastic Frontier Cost Function Approach, International Review of Applied Economics, 12, Lodish, M.L. et al (1995): How T.V. Advertising Works: A Meta-Analysis of 389 Real World Split Cable T.V. Advertising Experiments, Journal of Marketing Research, 32, Luo, X. and Donthu, N. (2001): Benchmarking Advertising Efficiency, Journal of Advertising Research, 41, Meusen, W. and Van den Broeck, J. (1977): Efficiency Estimation from Cobb-Douglas Production Functions with Composed Error, International Economic Review, 18, Ross, A., and Dröge, C. (2002): An Integrated Benchmarking Approach to Distribution Center Performance using DEA Modeling, Journal of Operations Management, 20, Seiford, L.M. (1994): A DEA Bibliography, in: Charnes, A., Cooper, W.W., Lewin, A. and L. M. Seiford (eds.), Data Envelopment Analysis: Theory, Methodology and Applications, Kluwer Academic Publishing, New York, N.Y. Seiford, L.M. and Zhu, J. (1999): An Investigation of Returns to Scales in DEA, Omega, 27, Tanaka, K., Takeda, E. and N. Nakajima (2002): Measuring the Performance of Advertising Campaigns based on DEA: an Empirical Study, Working Paper, Setsuan University. Thomas, R.R., Barr, R.S., Cron, W.L. and J.W. Slocum jr. (1998): A Process for Evaluating Retail Store Efficiency: a restricted DEA Approach, International Journal of Research in Marketing, 15,

15 Xue, M. and Harker, P.T. (2002): Note: Ranking DMUs with Infeasible Super-Efficiency DEA Models, Management Science, 48, Zhu, J. (1998): Data Envelopment Analysis vs. Principal Component Analysis: An Illustrative Study of Economic Performance of Chinese Cities, European Journal of Operational Research, 111, More detailed results on this regression are available from the author upon request. 2 Efficiency scores for the super-efficiency models were standardized by dividing the scores through the highest score. As with regular DEA models, the an efficient DMU now has a score of 1.