A: Measuring Efficiency of Georgia Aquarium

Transcription

1 WEB APPENDIX Implementing Integrated Marketing Science Modeling at a Non-Profit Organization: Balancing Multiple Business Objectives at Georgia Aquarium Finalist for the 2014 Gary L. Lilien ISMS-MSI Practice Prize Competition A: Measuring Efficiency of Georgia Aquarium WA-A1: Data Envelopment Analysis Charnes et al.(1978) first proposed DEA as an evolutionary non-statistical and non-parametric tool to measure and compare the efficiencies of the decision making units (DMUs). The major benefits of this method can be classified into two categories: First, this eliminates the possibility of model misspecification and allows the data itself to compute the efficiency frontier using linear programming. Second, DEA is able to compute a single efficiency measure for multiple DMUs, in this case different aquariums, using multiple inputs and multiple outputs (Donthu et al. 2005). The relative efficiency of each DMU is measured by the ratio of weighted inputs and outputs. The DMUs that have a ratio of 1 are referred to as efficient given the required inputs and produced outputs. The units that have a ratio less than 1 are less-efficient relative to the most efficient unit. A technical description of DEA can be found in Seiford (1996). DEA has been extensively used in production (Banker and Maindiratta 1986); public welfare and marketing practices (Kamakura et al. 1988); operation in insurance industry (Mahajan 1991); measurement of salesperson efficiency (Boles et al. 1995) and retail outlets efficiency (Kamakura et al. 1997). WA-A2: Super-Efficiency DEA To discriminate among the units that would otherwise be characterized as the most efficient in standard DEA, we also adopt an input-oriented, constant-returns-to-scale (CRS), super-efficiency DEA approach. In order to discriminate among the performances of efficient DMUs, a super-efficiency DEA model wherein a DMU under evaluation is excluded from the reference set was first developed by

2 Andersen and Petersen (1993). Super-efficiency DEA has been used in sensitivity analysis (Zhu 1996), detection of influential observations (Wilson 1995), and acceptance decision rules (Seiford and Zhu 1998). WA-A3: Technical details of DEA and Super-efficiency DEA A traditional input-oriented CRS DEA uses p inputs and q outputs to measure the relative efficiency of the DMUs-whose results range between 0%-100%. This creates two sets of DMUs: efficient DMUs on the frontier (efficiency score =1) and inefficient DMUs within the frontier (efficiency score<1). Super efficiency DEA further compares the efficient DMUs on the frontier and finds the super-efficient DMUs (efficiency>1). The two-staged procedure is shown below: Suppose we have N DMUs, in our case 20 aquariums. Each DMU j (j = 1,2,.20) produces q different outputs Y rj (r = 1,2, q) using p different inputs X ij (i = 1,2, p). Mathematically, we represent the DEA procedure as follows: Model 1: Data Envelopment Analysis θ 0 0 CRS = min θ CRS N λ j Y rj j=1 N λ j X ij j=1 subject to + q 0 i = θ CRS X i0 ; i = 1,2, p, q + 0 r = Y r0 ; r = 1,2, q; θ CRS, λ j, q i, q + r 0 (In this case X i0 and Y r0 are the i th input and r th output is for DMU 0 under evaluation.) After we solve the linear programming equation as shown in Model 1, we are left with a set of efficient DMUs and a set of inefficient DMUs. In the next phase, we use super-efficiency DEA to differentiate among efficient DMUs. The linear programming approach is shown below: 2

3 Model 2: Super-Efficiency Data Envelopment Analysis θ super super f = min θ f subject to N j=1 λ j X ij θ super N j X i0 ; i = 1,2, p, j=1 λ j Y rj Y r0 ; r = 1,2, q, j 1 θ f super, λ j (j 0) 0 The output of Model 2 provides an efficiency score for the DMUs on the frontier which is greater than (or equal to) 1, eliminating the previous censoring problem and allowing us to discriminate between different, but previously 100% efficient DMUs. To estimate the super-efficiency DEA model, we use the DEA frontier. WA-A4: Variable Selection and results of DEA and Super-efficiency DEA: We chose as inputs the factors that GA cited to typically bear a direct and positive influence on the outputs (Luo and Donthu 2005) of the aquariums. We measured efficiency in terms of ticket price, size of the aquarium, population of the city where the aquarium operates, and the number of animals (species) in the aquarium as inputs and attendance and revenue as two outputs. DEA analysis groups DMUs into two clusters: efficient and non-efficient DMUs. Through the DEA analysis, we were able to identify GA and 10 other aquariums to be efficient. Super efficiency DEA further groups the efficient DMUs from DEA analysis into two clusters: super-efficient (efficiency score>=1) and not super-efficient (efficiency score <1) DMUs. We then ran super-efficiency DEA for the 11 efficient aquariums and found the efficiency scores for each of the 11 aquariums. The mean efficiency scores across DMUs in the super-efficiency DEA was 1.1 (minimum efficiency=0.88; maximum efficiency=1.96). GA is thus found to be super-efficient, confirming that increasing inputs will not increase outputs. The efficiency scores of the 20 aquariums that are under consideration based on the DEA can be found in WA-T1. 3

4 WA-T1: Efficiency Scores of 20 Aquariums based on DEA (Inputs: Size, Population, Animals, Price) Aquarium Dual Output: Attendance & Revenue* Scenario II Aquarium of the Bay 0.95 Aquarium of the Pacific 1.00 Audubon Aquarium of the Americas 0.82 Baltimore National Aquarium 1.00 Birch Aquarium at Scripps 1.00 California Academy of Sciences 0.61 Florida Aquarium 0.55 Georgia Aquarium 1.00 Monterey Bay Aquarium 1.00 Mystic Aquarium 1.00 New England Aquarium 1.00 Oklahoma Aquarium 0.74 Oregon Coast Aquarium 1.00 Seattle Aquarium 0.77 Shedd Aquarium Chicago 1.00 South Carolina Aquarium 0.90 Tennessee Aquarium 1.00 Texas State Aquarium 0.98 The Living Planet Aquarium 1.00 Vancouver Aquarium 0.83 Note: 1.00 = Efficient *Average Monthly Attendance and Revenue WA-F1: DEA and Super-efficiency DEA Problem NO Increase the Outputs Attendance and Revenue YES Super DEA Possible Solutions Modify the Inputs Size Population Animals Ticket Price Is GA efficient with regard to these inputs? DEA NO YES DEAD END! Modification of Inputs not possible PROCEED WITH PLAN B 4

5 B: PLNR Computation Details WA-F2: PLNR Model Computation Timeline MODEL BUILDING DATASET PERFORMANCE EVALUTION TIME PERIOD PROJECTED TIME PERIOD Independent Variables Dependent Variables Independent Variables Predicted Dependent Variables (Model Prediction using Estimated Parameters) Independent Variables WA-B1: Independent variables for Model 3b X it β = β 0 + β 1 (average growth in uses of passes it 1 ) + β 2 (AIT it 1 ) + β 3 (recency it 1 ) + β 4 age it + β 5 (members in the household it 1 ) + β 6 (average number of passes purchased/ repurchased per year it 1 ) + β 7 (uses of passes it 3 ) + β 8 ( uses of passes it 2 ) + β 9 ( uses of passes it 1 ) + β 10 recency it 1 (average number of passes purchased/repurchased per year it 1 ) + β 11 recency it 1 ) (uses of passes it 3 ) + β 12 recency it 1 (uses of passes it 2 ) + β 13 recency it 1 ( uses of passes it 1 ) + β 14 (members in the household it 1 ) (uses of passes it 3 ) + β 15 (members in the household it 1 ) (uses of passes it 2 ) + β 16 (members in the household it 1 ) (uses of passes it 1 ) + β 17 (total marketing expenses it 1 ) WA-B2: Independent variables for Model 3c γz it =γ 0 + γ 1 (average annual revenue it 1 ) + γ 2 (AIT it 1 ) + γ 3 (recency it 1 ) + γ 4 age it + γ 5 (members in the household it 1 ) + γ 6 (average growth in uses of passes it 1 ) + γ 7 (average number of passes purchased/repurchased per year it 1 ) + γ 8 ( uses of passes it 3 ) + γ 9 ( uses of passes it 2 ) + γ 10 (uses of passes it 1 ) + γ 11 (average growth in revenue it 1 ) + γ 12 (recency it 1 ) (average annual revenue it 1 ) + γ 13 (recency it 1 ) (average number of passes purchased/repurchased per year it 1 ) + γ 14 (recency it 1 ) (uses of passes it 3 ) + γ 15 (recency it 1 ) (uses of passes it 2 ) + γ 16 (recency it 1 ) (uses of passes it 1 ) + γ 17 (members in the household it 1 ) (uses of passes it 3 ) + γ 18 (members in the household it 1 ) (uses of passes it 2 ) + γ 19 (members in the household it 1 ) (uses of passes it 1 ) 5

6 WA-T2: Drivers and Parameter Estimates of PLNR Model Drivers Net Revenue it Definition Difference between total revenue contributed by pass holder i at time t and marketing expenses for GA to manage pass holder i at time t Probit Model Parameter Estimates NA Regression Model Parameter Estimates Dependent Variable Relationship with PLNR As net revenue increases, PLNR increases (+) Average annual revenue it 1 Average annual revenues from the pass holder i at time t *** As average annual revenue increases, PLNR increases (+) Recency it 1 Average growth in revenue it 1 Members in the household it 1 How recently the last purchase/repurchase took place Average changes in the year to year revenue contribution from pass holder i at time t *** * *** Age it Age of the pass holder i at t NA Total number of people who have purchased passes in a household to which *** *** pass holder i belongs at time t-1 Average number of passes purchased/repurchased per year it 1 Average number of passes purchased/repurchased per year by pass holder i at time t *** 3.001** If pass holder has purchased/repurchased the pass recently, then PLNR will be lower (-) If revenue from a passholder increases over time, then PLNR of the pass holder will be higher (+) As the number of pass holders in a household increases, the PLNR of the pass holder increases (+) As the average number of passes purchased/ repurchased per year increases, the PLNR of the pass holder i increases (+)

7 Usage of passes it 3 Number of times passes have been used by pass holder i at time t *** ** If a pass holder has used passes multiple times in t-3, then PLNR of the pass holder at time t will be higher (+) Usage of passes it 2 Number of times passes have been used by pass holder i at time t *** *** If a pass holder has used passes multiple times in t- 2,then PLNR of the pass holder at time t will be higher (+) Usage of passes it 1 Average growth in usage of passes it 1 Number of times passes have been used by pass holder i at time t-1 Average changes in the year to year usage frequency of passes for passholder i at time t *** NA If a pass holder s usage frequency increases over time, then its PLNR will be lower (-) Recency it 1 Average annual revenue it 1 Interaction between recency of purchase/repurchase and average annual revenue *** For a pass holder, who recently purchased/repurchased a pass and has given high revenue in the past, the expected PLNR of the pass holder is higher (+) Recency it 1 Usages of passes it 3 Interaction between recency and number of times passes have been used by pass holder i at time t ** *** For a pass holder, who recently purchased/repurchased pass and who used the passes multiple times at time t-3, the expected PLNR at time t is low (-) 7

8 Recency it 1 Usage of passes it 2 Interaction between recency and number of times passes have been used by pass holder i at time t *** *** For a pass holder, who recently purchased/repurchsed a pass and who used the passes multiple times in t-2, the expected PLNR at time t is low (-) Recency it 1 Usage of passes it 1 Total marketing expenses it 1 Interaction between recency and Number of times passes have been used by pass holder i at time t-1 Total amount spent by GA on marketing on pass holder i at time t *** - NA As the total marketing amount GA spends on a pass holder increases, the PLNR increases (+) AIT it 1 Recency it 1 Average number of passes purchased/repurchased per year it 1 Average inter-purchase time: time difference in years in purchasing/repurchasing pass for pass holder i at time t-1 Interaction between recency and Average number of passes purchased/repurchased per year by a pass holder at time t *** *** *** If a pass holder has higher AIT, then his/her PLNR will be higher (+) If a pass holder has recently purchased/repurchased the pass and average number of passed purchased/ repurchased per year is high, then his/her expected PLNR at time t will be higher (+) Members in the household it 1 Uses of passes it 3 Interaction between total number of people who have purchased/repurchased passes in a household to which pass holder i belongs at time t-1 and Number of times passes have been used by pass holder i at time t *** ** As the number of pass holders in a household at time t-1 and the usage frequency of passes by the pass holder at time t-3 increase, the PLNR of the pass holder at time t 8

9 decreases (-) Members in the household it 1 Usage of passes it 2 Members in the household it 1 Usage of passes it 1 Interaction between total number of people who have purchased/ repurchased passes in a household to which pass holder i belongs at time t-1 and Number of times passes have been used by pass holder i at time t-2 Interaction between total number of people who have purchased/repurchased passes in a household to which pass holder i belongs at time t-1 and Number of times passes have been used by pass holder i at time t * As the number of pass holders in a household at time t-1 and usage frequency of passes by the pass holder at t-2 increase, the PLNR of the pass holder in t decreases (-) NA IMR ***significant at 0.1% level ; **significant at 5%, *significant at 10% For variables: Total marketing expenses it 1, Average growth in revenue it 1, Recency it 1, Average growth in usage of passes it 1, Average annual revenue it 1, Average number of passes purchased/repurchased it 1, Members in the household it 1 and AIT it 1, (t-1) represents the data for the period to model the repurchase probability and net revenue for WA-T3: Descriptive Statistics of Independent Variables used in Models 3b and 3c Variables Mean Standard Deviation Average annual revenue it Recency it Average growth in revenue it Age t Members in the household it

10 Average number of passes purchased /repurchased per year it 1 Usage of passes it Usage of passes it Usage of passes it Average growth in usage of passes it Total Markerting expenses it AIT it WA-T4: Correlation among Independent variables used in Models 3b and 3c Independent Variables (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Average Annual Revenue it 1 (1) 0.22*** 0.67*** *** 0.44** 0.13** * 0.77** 0.59** -0.16** Recency it 1 (2) 0.51** -0.04** 0.08** 0.23** * 0.37** 0.33* 0.35* Average growth in the revenue it 1 (3) *** 0.01** 0.22** -0.42* 0.019** ** Age it (4) Members in (5) the household it 0.78* -0.15** 0.23** 0.24** 0.29** 0.09** 0.14*** Average number of -0.57** 0.45** 0.26** 0.11** -0.15** 0.57** passes purchased/ repurchased it 1 (6) per year AIT it 1 (7) -0.42** -0.27** 0.17** 0.36** Usage of (8) -0.10** -0.11** -0.62** 0.46** passes it 3 Usage of (9) -0.08** ** passes it 2 Usage of (10) 0.84* -0.20** passes it 1 Average growth in usage of passes it 1 (11) -0.41** Total Marketing Expenses it 1 (12) 1 *Significant at 5% level **Significant at 1% level ***Significant at 0.1% level 10

11 WA-T5: Parameter Estimates (for the Models with Transformed Variables) Probit Model Regression Model Parameter Parameter Variables Estimates Estimates Net Revenue it NA Dependent Variable Average annual revenue per visit it *** Recency it *** * Average growth in revenue per visit it *** Age it Members in the household it *** 22.83*** Average number of passes purchased/repurchased per year it *** 4.013** Usage of passes it *** ** Usage of passes it *** *** Usage of passes it Average growth in usage of passes it *** Recency it 1 Average annual revenue per visit it *** Recency it 1 Usages of passes it ** *** Recency it 1 Usage of passes it *** *** Recency it 1 Usage of passes it Total marketing expenses per visit it *** - AIT it *** *** Recency it 1 Average number of passes purchased/repurchased per year it *** Members in the household it 1 Uses of passes it *** ** Members in the household it 1 Usage of passes it * Members in the household it 1 Usage of passes it IMR ***significant at 0.1% level ; **significant at 5%, *significant at 10% WA-T6: Hit Ratio WA-T7: MAPE Model Probability of Pass Repurchase (Level I of Tobit Model) Calibration Sample Hit Ratio Validation Sample Hit Ratio 82.3% 81% Model Predicted Net- Revenue (Level II of Tobit Model) Calibration Sample MAPE Validation Sample MAPE 20.5% 28% 11

12 Retention Percentage PLNR ($) WA-F3: Profiling of Pass Holders based on their values 2,500 2,000 2,065 PLNR for Annual Pass Holders S p High PLNR S g t 1,500 1,000 High PLNR Medium PLNR Low PLNR Very Low PLNR Decile WA-F4: Retention of Pass Holders Retention of Pass Holders Year 12

13 Segments Super High and High PLNR Medium PLNR Low and Very Low PLNR WA-T8: Pass Holder Retention Strategies Retention Strategies Nurture their existing relationship, and Offer best-in-class services to cultivate attitudinal loyalty Upsell/cross-sell products at GA s café and gift shop Focus on encouraging word-of-mouth publicity and rewarding behavioral loyalty Do not over-spend on marketing activities Reduction of overall marketing costs Attempt to upsell/cross-sell products Source referrals and encourage to repurchase WA-F5: Indicators of a high PLNR pass holder A high PLNR is obtained at time t when Pass Repurchase Average number of passes purchased/repurchased per year is high t-2 t-3 Pass Usage Pass is used more Pass is used more Pass Holder Revenue Revenue from a pass holder increases over time Pass Holders in Household There are multiple pass holders in the household Frequency of Purchase Pass holder repurchases his/her pass more frequently A-3: Spatial Analysi 13

14 C: Spatial Analysis WA-F6: Overview of Spatial Analysis RETENTION STRATEGY 40 Common Zip Codes for retaining visitors and pass holders ACQUISITION STRATEGY Look-A-Like Zip Codes for Targeting and Acquiring Look-A-Like Zip Codes for Targeting Figure 6: Findings of Zip Code Analysis conducted in Phase II: Where do Georgia Aquarium s most 7 valuable pass 17 holders come from? A. Pass holders 8 with Super-high PLNR:

15 WA-F7: Where do Georgia Aquarium s most valuable pass holders come from? A. Pass Holders with Super-High PLNR: B. Pass Holders with High PLNR: C. Pass Holders with Medium PLNR: 15

16 D: Media Optimization Analysis WA-T9: Select Literature Review for Media Optimization Model AUTHORS CONTEXT FINDINGS OF THE STUDY Clarke (1976) The duration of the impact of The cumulative effect of advertising on sales is short-lived, and lasts for Advertising (Offline) on sales months rather than years. Blattberg and The aggregate sales- Abel (1981) advertising relationship There is a positive relationship between advertising and aggregate sales. A Meta-Analysis of Assmus et al. econometric models of the (1984) advertising-sales relationship The effect of advertising on a firm s sales is specific and quantifiable. Tellis and Aggregating data over time and households may create a misleading Offline Ads (TV ads) Weiss (1995) impression of advertising having a statistically significant effect on sales. The ideal way to schedule advertising is by implementing a pulsing strategy. Naik et al. The importance of a media The right media scheduling strategy can enhance media effectiveness and (1998) scheduling strategy thereby the firm s visibility, recall and sales. JA (2000) Offline media Offline media spend are important even for online businesses. Klein and Ford (2003) Chatterjee et al. (2003) Dreze and Hussherr (2003) Manchanda et al. (2006) Tellis and Amber (2007) Naik and Peters (2009) Stephen and Galak (2012) Li and Kannan (2014) Our Study Offline and online media Online media effectiveness (An analytical model) Click-through rates in online media and measuring online media effectiveness Online media Offline media Online and offline media synergies Quantifying and comparing the relative impacts of traditional and social earned media on sales Online media Online and offline media Consumers are increasingly replacing traditional offline information search with online search. Hence the importance of online media in the purchase decision making process and actual sales is increasing. Effective placement of dynamic online ads, selection of online media vehicles and measurement of advertising responses. Click-through rates are low in the online environment. However, repeated exposure to online banner ads influences both aided and unaided recall of advertising, brand recognition, and brand awareness. Advertising affects the purchase behavior of a firm s existing and new customers. The number of pages in a website, number of exposures to an online advertising message and the number of websites advertised on, are all positively related to the purchase probabilities of customers. Offline media are very effective in generating sales. There are definite synergies between intra-media and cross media (onlineoffline). There is a larger effect of traditional earned media on per event sales than that of social earned media. The social earned media sales elasticity is greater than the traditional earned media sales elasticity. The performance of traditional earned media is driven by social earned media. A new methodology to measure the incremental effect of each media channel in the online media environment. Accounts for interactions of all media (online and offline). Identifies the most effective media. Recommends optimal spend in each effective media type. 16

17 Attendance VARIABLE WA-T10: Parameter Estimates of Media-mix Optimization Model PARAMETER ESTIMATES VARIABLE PARAMETER ESTIMATES Log(1+radio) **** D *** Log(1+TV) *** D Log(1+Online) **** D *** Log(1+Magazine) ** e e-07** Log(1+Outdoor) * e e-7** Log(1+Newspaper) e e-07** Log(1+Othermedia) e e-07* Log(Ticket Price) *** e e-07* Log(Visitors Satisfaction) *** e e-05** Log(Coke Attendance) ** e7-2.26e-06* Log(Braves Attendance) *** Log(Changes in Quarterly GDP) * Significant at 10% level **Significant at 5% level ***Significant at 1% level ****Significant at 0.1% level E: Quantifying the impact of the study WA-F8: Performance of GA in terms of Annual Attendance after implementing our recommendations Aquarium Attendance 2,400,000 2,300,000 2,200,000 Actual Updated Prediction based on recommendations 2,100,000 Status Quo Prediction 1,000, ,000 Average of 19 Aquariums 10% Status Quo Prediction Updated Prediction Actual Average of 19 Other Aquariums Year 17

18 WA-F9: Performance of GA in terms of Annual Revenue after implementing our recommendations $85,000,000 Annual Revenue Actual Revenue $80,000,000 $75,000,000 $70,000,000 $21,000,000 Updated Prediction based on recommendations Status Quo Prediction 12% $20,000,000 Average of 19 Aquariums Year Status Quo Prediction Updated Prediction Actual Average of 19 Other Aquariums Note: Charts are truncated at the dotted line WA-T11: Findings of GA s Annual Visitor Survey Report, Metrics Actual Realization Previous Year Benchmark by GA in 2013 Level Average Overall Visitor Satisfaction 90% 87% 75% Purchases at Café 57% 46% 43% Purchases at Gift Shop 60% 52% 46% Advertising Recall * 86% 82% 64% Word-of-mouth awareness 72% 68% 57% Entertainment Experience 93% 88% 77% Educational Experience 93% 92% 75% Perceived Admission Value # 84% 77% 62% Likelihood of Returning 79% 71% - Satisfaction with onsite Crowd management 81% 80% 63% Net Promoter Score 96% 96% 75% * The primary information sources were the Internet (40%, up from 30%) and television (34%, up from 25%). (Source: Georgia Aquarium Visitor Survey 2012/2013 Morey Group). # This rating is typically less reflective of the admission price and more reflective of overall satisfaction; it is significant since the Aquarium s admission price is high compared with those of the Benchmark aquariums. (Source: Georgia Aquarium Visitor Survey 2012/2013 Morey Group) 18

19 References: Andersen P, Petersen NC (1993) A procedure for ranking efficient units in data envelopment analysis. Management Sci. 39(10): Assmus G, Farley JU, Lehmann DR (1984) How advertising affects sales: Meta-analysis of econometric results. J. Marketing Res. 21 (1): Banker RD, Maindiratta A (1986) Piecewise loglinear estimation of efficient production surfaces. Management Sci. 32(1): Blattberg RC, Abel PJ (1981) A Micromodeling Approach to Investigate the Advertising-Sales Relationship. Management Sci. 27 (9): Boles JS, Donthu N, Lohtia R (1995) Salesperson evaluation using relative performance efficiency: The application of data envelopment analysis. The J. Personal Selling and Sales Management Charnes A, Cooper WW, Rhodes E (1978) Measuring the efficiency of decision making units. European J. Oper. Res. 2(6): Chatterjee P, Hoffman DL, Novak TP (2003) Modeling the Clickstream: Implications for Web- Based Advertising Efforts. Marketing Sci. 22(4): Clarke DG (1976) Econometric Measurement of the Duration of Advertising Effect on Sales. J Marketing Res. 13 (4): Donthu N, Hershberger EK, Osmonbekov T (2005) Benchmarking marketing productivity using data envelopment analysis. J. Bus. Res. 58(11): Dreze X, Hussherr FX (2003) Internet Advertising: Is anybody Watching? J. Interactive Marketing. 17(4):8-23. JA Bellizzi (2000) Drawing Prospects to E-commerce Websites. J. Advertising Res. 41(1/4): Kamakura WA, Ratchford BT, Agrawal J (1988) Measuring market efficiency and welfare loss. J. Consumer Res. 15 (3): Kamakura WA, Lenartowicz T, Ratchfrord BT (1997) Productivity assessment of multiple retail outlets. J. Retailing. 72(4): Klein LR, Gary TF (2003) Consumer search for information in the digital era: An empirical study of prepurchase search for automobiles. J Interactive Marketing. 17(3):

20 Li H, Kannan PK (2014) Attributing Conversions in a Multichannel Online Marketing Environment: An Empirical Model and a Field Experiment. J. of Marketing Res.50.1:40-56 Luo X, Donthu N (2005) Assessing advertising media spending inefficiencies in generating sales. J. Business Res. 58(1): Mahajan J (1991) A data envelopment analytic model for assessing the relative efficiency of the selling function. European J. Oper. Res. 53(2): Manchanda P, Dube JP, Goh KY, Chintagunta PK (2006) The effect of Banner advertising on internet purchasing. J. Marketing Res. 43(1): Naik PA, Raman K, Winer RS (2005) Planning marketing-mix strategies in the presence of interaction effects. Marketing Sci. 24(1): Naik PA, Kay P (2009) A Hierarchical Marketing Communications Model of Online and Offline Media Synergies. J. Interactive Marketing. 23 (4): Seiford LM, Zhu J (1998) An acceptance system decision rule with data envelopment analysis. Computers & Oper. Res. 25(4): Seiford LM (1996) Data envelopment analysis: the evolution of the state of the art ( ). J. Productivity Analysis. 7(2-3): Stephen AT, Galak J (2012) The effects of traditional and social earned media on sales: A study of a microlending marketplace. J. Marketing Res. 49(5): Tellis GJ, Amber T (2007) Handbook of Advertising. Sage Handbook of Advertising, Gerard J. Tellis and Tim Amber, eds. London: Sage Publications, 3-16 Tellis G J, Weiss DL (1995) Does TV Advertising Really Affect Sales? The Role of Measures, Models, and Data Aggregation. J. Advertising. 24 (3): Wilson PW (1995) Detecting influential observations in data envelopment analysis. J. Productivity Analysis. 6(1): Zhu J (1996) Robustness of the efficient DMUs in data envelopment analysis. European J.Oper.Res. 90(3):