The State of Research Methods in Personal Selling and

Transcription

1 The State of Research Methods in Personal Selling and Sales Management Literature Anthony K. Asare, Jing Yang, and Thomas G. Brashear Alejandro This study examines the state of research methods in the personal selling and sales management (SALES) literature and assesses the trends and changes in these methods that have taken place in recent decades. The authors examine the research methods described by 1,346 sales articles published in 15 prominent journals over a 29-year period ( ). They compare articles published in Journal of Personal Selling & Sales Management with the aggregate of sales articles published in other journals as well as with articles published in Industrial Marketing Management to reveal any common trends or differences. The results indicate some important shifts in the use of research methods in SALES literature as well as significant differences in the research methods published across different research outlets. This paper outlines the implications of the findings and offers suggestions for how the discipline might improve its use of various research tools. Personal selling and sales management (SALES) research has matured, and in recent decades its importance as a subdiscipline of marketing has increased considerably (Moncrief et al. 2000). As any discipline matures and its research expands, a critical examination of research methods used becomes necessary since the questions to be addressed in maturing disciplines become more complex and the methods needed to assess them become more sophisticated (Crook et al. 2010). Thus, researchers in other management fields have assessed the use of research methods in their disciplines, including accounting (Dyl and Lilly 1985), organizational research (Aguinis et al. 2009; Stone-Romero, Weaver, and Glenar 1995), management (Scandura and Williams 2000), retailing (Brown and Dant 2009), and strategic management (Ketchen, Boyd, and Bergh 2008; Shook et al. 2004). Marketing researchers (nonsales) have also examined the state of the art in marketing research. Greenberg, Goldstucker, and Bellenger (1977) consider research methods employed by business practitioners and find that marketing researchers in the late 1970s mostly used simple analytical measures. Bellenger and colleagues (1990) revisit Greenberg, Anthony K. Asare (Ph.D., University of Massachusetts Amherst), Assistant Professor of Marketing, School of Business, Quinnipiac University, Hamden, CT, anthony.asare@quinnipiac.edu. Jing Yang (Ph.D., University of Massachusetts Amherst), Assistant Professor of Marketing, School of Business Administration, Pennsylvania State University at Harrisburg, Middletown, PA, juy22@ psu.edu. Thomas G. Brashear Alejandro (Ph.D., Georgia State University), Associate Professor of Marketing, Eugene M. Isenberg School of Management, University of Massachusetts Amherst, Amherst, MA, brashear@mktg.umass.edu. Goldstucker, and Bellenger s study and discover increased sophistication and complexity in research methods since that time. Germain (1993) focuses his examination on research methods in marketing research textbooks and finds that they mainly utilize basic statistical and research methods. Baumgartner and Homburg (1996) and Steenkamp and Baumgartner (2000) investigate the use of a single analytical tool (structural equation modeling [SEM]), whereas Wind, Rao, and Green (1991) and Svensson and Wood (2007) focus on multiple analytical tools, such as factor analysis, analyses of variance (ANOVA), multivariate analyses of variance (MANOVA), and causal modeling. Like their counterparts, SALES researchers have conducted studies to examine research trends. Moncrief et al. (2000) examine scholars publication records and backgrounds to explore trends in SALES research and found that scholarly activity was becoming more robust. Carter, Dixon, and Moncrief s (2008) study examines the complexities of SALES research from 1990 to 2005 by focusing on response rates in both sales and marketing journals. They find that response rates were lower for SALES research than marketing (nonsales) research. Richardson, Swan, and McInnis-Bowers (1994) investigate the extent to which sample frames in sales force research cover different industries and types of salespeople and conclude that the breadth of sales force samples utilized in SALES studies is limited. Finally, Swan, Powers, and Sobczak (1991) conduct a citation analysis of the Journal of Personal Selling & Sales Management (JPSSM), and Bush and Grant (1991) offer a contributor analysis of SALES research articles. Although these research summaries have provided useful information, none of them focused on the use of research methods and tools in the SALES literature until Bush and Grant (1994), whose study covers 13 years ( ) of Journal of Personal Selling & Sales Management, vol. XXXII, no. 4 (fall 2012), pp PSE National Educational Foundation. All rights reserved. Permissions: ISSN (print) / ISSN (online) DOI: /PSS

2 474 Journal of Personal Selling & Sales Management research published in JPSSM, Industrial Marketing Management (IMM), Journal of Marketing (JM), and Journal of Marketing Research (JMR ). Their study advanced knowledge in the use of research methods in the SALES literature, but their focus was limited to research design, analytical tools, and sample units, leaving out important research areas such as construct development and measurement. Their examination was also influenced by the analytical tools that were popular prior to Tools such as SEM, which have since become very common, were not included. Since Bush and Grant (1994) offered their assessment, the only study that we know of that assesses the use of a wide range of research methods in SALES research was conducted by Williams and Plouffe (2007). Plouffe, Williams, and Wachner (2008) also examine various research methods in their investigation of publishing trends in SALES research. However, their discussion of research methods used in SALES research is relatively brief. We extend work by Bush and Grant (1994) and Williams and Plouffe (2007) and cover a wider selection of research methods than has been previously covered in any of the analysis of SALES research methods. We examine 15 journals over 29 years, which is the longest period covered by any review of SALES research methods. We also cover a wider range of research methods (53) versus William and Plouffe (27 methods) and Bush and Grant (22 methods). For example, we cover important areas in research development such as construct measurement and development, areas that the other studies do not cover. Our analysis of research methods are also more in-depth than the other articles. While the other articles utilize only frequency counts to analyze their results, we utilize frequency counts, percentage use index, t tests, and trend analysis to statistically examine the differences and similarities in use of research methods starting in 1980 (the inaugural year of JPSSM ). We also compare the different journals utilized in our study to examine similarities and differences in their use of research methods in SALES articles that have been published. As a result of our exclusive focus on research methods, the number of years we covered, as well as the breadth and depth to which we examine the use of research methods in the SALES literature, our study is by far the most comprehensive study conducted to examine the use of research methods in SALES research. This study enables us to respond to the following questions: 1. How frequently have various research methods been used by SALES researchers in recent decades? 2. Have the basic methods and designs in SALES research shifted significantly over time? 3. Are there differences in SALES research methods published in different journals? 4. What are the possible implications of trends in research method usage in SALES research? To address these questions, we examine research methods in 1,346 sales articles published in 15 prominent journals from 1980 to We also compare articles published in JPSSM to sales articles published in journals other than JPSSM. Finally, we compare articles published in JPSSM to articles published in IMM, to reveal any common trends and differences between the two journals that publish the most SALES research articles. This study contributes to sales research and practice in several ways. It contributes to a better understanding of changes in research design and analysis, which should help researchers and practitioners improve and adjust their research methods to respond to new trends and sales environments. We provide the discipline s gatekeepers, including editors, reviewers, and members of editorial boards, with information about the status of research methods, which may assist them in their evaluations of articles submitted for review (Stone-Romero, Weaver, and Glenar 1995). The insights from this study should also help researchers gain greater awareness of the methodological procedures preferred by sales journals to guide them as they conduct their research targeted at those journals (Scandura and Williams 2000). Finally, this study provides sales research institutions and training programs with information about the state of research methods to assist them in training sales researchers. The remainder of this paper is structured as follows. We describe the methods and tools we used to collect and analyze the data, then discuss the results of our analysis. We also note the implications of our findings for selling and sales management literature. We conclude with a brief summary and discussion of potential research areas. Data Collection and Coding Methodology To provide an overview of research in SALES literature, we examined the methods employed in 1,346 articles published in 15 prominent journals over a 29-year period ( ). The journals included in this survey were JPSSM, IMM, JM, JMR, Journal of Business & Industrial Marketing (JBIM), Marketing Science (MS), Journal of the Academy of Marketing Sciences (JAMS), Journal of Business Research (JBR), International Journal of Research in Marketing (IJRM), Marketing Letters (ML), Psychology & Marketing (P&M), Journal of Marketing Theory and Practice (JMTP), Journal of Applied Psychology (JAP), Journal of Consumer Marketing (JCM), and Journal of Business Ethics (JBE). Six general categories of information were collected from these journals: research design, triangulation, data collection methods, construct validity and reliability, dependent variable properties, and data analytic techniques.

3 Fall SALES studies were selected using a procedure similar to that used by Moncrief, Marshall, and Watkins (2000). We initially searched for academic articles that had sales, selling, sales force, and sales management in their abstract or title. We then looked for articles that focus on topics that are typically found in academic sales articles and in the JPSSM Abstracts section. We did not include articles that focused on sales as a measure of a firm s performance or sales promotions. However, we included articles that used salespeople as their sample of study, even if the primary focus of the study was not sales. Two reviewers were involved in the coding process, and to help ensure consistency in their coding, the two coders initially coded 20 of the same articles. They then cross-checked their coding, and when there were inconsistencies, they discussed them with the help of a third reviewer until there was unanimity among the coders. After all the articles were coded, we checked the reliability of the results by asking 2 other people to code 200 articles (15 percent) that had already been coded. The ratio of coding agreement between the data coded by the two new coders and the originally coded data were 91 percent and 94 percent. These results are in line with the median of interjudge reliability rates of 90 percent reported in the literature (Bush and Grant 1994; Kassarjian 1977). Research Design For the research design category, we focused on the primary research strategies for each study. The research design categories reflected the framework proposed by McGrath (1982) and Wind, Rao, and Green (1991). We thus coded 12 types of research designs: conceptual studies, survey research, lab experiment, experimental simulation, field experiment, field study, pure methodology, judgment task, case study, qualitative methods, meta-analysis, and measurement development. Triangulation This category includes both types of triangulation identified by Denzin (1978), namely, in-method and between-method. For our research, in-method triangulation refers to the use of multiple studies in a single research article, whereas betweenmethod triangulation represents the use of different research methods in an article. Thus, an article with multiple lab studies (common in consumer research) offers in-method triangulation, whereas one that combines qualitative and quantitative methods would be considered a between-method triangulation article. Data Collection Methods In the data collection category, we examined different aspects of data collection, including the data source (primary or secondary), the time frame (cross-sectional, dual periods, or longitudinal), the number of key informants (single or multiple), and the geographic location of the respondents (United States or international [non United States]). Construct Validation and Reliability We coded the different approaches that researchers used to test and report construct validity and reliability, including Cronbach s alpha, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), composite reliability, discriminant validity, and average variance extracted (AVE). The AVE shows the amount of variance captured by a construct in relation to the amount of error that is the result of measurement error (Diamantopoulos and Siguaw 2000). We also examined articles to determine if they reported tests for response bias, common method bias, and interrater reliability. Dependent Variable Properties For the dependent variable properties category, we reviewed the number and types of dependent variables used. Following Scandura and Williams (2000), we classified the dependent variables as tangible outcomes, performance ratings, or perceptual and attitudinal variables. Data Analytic Techniques We coded for different data analytic techniques, including univariate analytical techniques, such as t tests, and chi-square tests; bivariate techniques, such as correlations, ANOVA, and simple regression; and multivariate techniques, including multiple regression, stepwise regression, hierarchical regression, MANOVA, SEM, conjoint analysis, logit/tobit, and discriminant analysis. Finally, we examined the analytical techniques used to cluster and group different subjects: multidimensional scaling and clustering techniques. Analysis of Coded Data To answer our key research questions, we started with an evaluation of the overall usage and trends of different research methods. Thus we employed a usage index, similar to previous authors (Crook et al. 2010; Stone-Romero, Weaver, and Glenar 1995). The percentage use index (PI) details the number of uses of a research method as a percentage of all the articles coded in that year. For example, conceptual articles earned a PI of during , meaning percent of all the articles we coded between 1980 and 1984 are conceptual. The empirical percentage use index (EPI) is similar but includes only empirical articles. For our analyses, we used PI to calculate the research design category, which features empirical

4 476 Journal of Personal Selling & Sales Management and nonempirical articles, then employed EPI for all the other sections, which included only empirical articles. To determine trends in the use of different sales research methods, trend analysis was used to determine how the different methods have been used over time. The study summarizes each coding variable s growth trajectory by fitting a separate regression model to each variable s data. A linear individual change trend was adopted because it provides a good description of the trajectories for the variables in this study. We regressed the outcome PI/EPI on YEAR. YEAR was centered before it was entered into the regression to improve the interpretability of the intercept (Singer and Willett 2003). We used 1994 as the central year. Each variable s estimated intercept and slope summarizes its growth trajectory, where the fitted intercept estimates the value of the variable in 1994 and the fitted slope estimates the annual rate of change in the outcome. In longitudinal data, residuals tend to be autocorrelated over time. We conducted the Durbin Watson bounds test for each regression model, following the decision rules suggested by Durbin and Watson (1951), then proceeded with the ordinary least squares (OLS) estimation. In situations where we found autocorrelation, we used the Cochrane Orcutt estimation method to analyze the data (Cochrane and Orcutt 1949). We also conducted independent samples t tests to compare research methods (PI/EPI) used in JPSSM with those used in SALES articles published in journals other than JPSSM (non-jpssm articles). Finally, we compared articles published in JPSSM to SALES articles published in IMM. Results In Table 1 we provide a summary of the SALES articles published in different journals. We find a clear upward trend in the number of SALES articles published, such that the average in the 1980s was 34.8 articles per year, the 1990s climbed to 50.8 articles a year, and the average reached 54.4 articles annually. Before 1987, 40 percent of the journals we examined had never published a single SALES article. As expected, JPSSM published the most articles by far (567), representing 42 percent of all articles examined in this study, followed by IMM (243) with 18 percent. None of the other research outlets reached more than 10 percent. For example, JBIM (77), with the third-largest number of SALES articles, accounted for only 6 percent of the articles examined in this study. The fourth-place journal, JAMS (74) provided a surprise, in that this general journal is not usually considered a popular outlet for publishing sales and business-to-business (B2B) articles. Yet JAMS published almost as many SALES articles as JBIM, a B2B journal that is considered a popular SALES research outlet. Finally, JM (59) was the most receptive of the three premier journals, whereas JMR published 44 articles and MS 21. Our findings also show that SALES articles appear in a wide variety of outlets, which implies that researchers cover a considerable range of topics. In addition to traditional sales, B2B, and marketing journals, SALES articles appear in consumer and psychology journals such as JAP, P&M, and JCM; practitioner-oriented journals such as JMTP; and JBE. The 39 SALES articles in JBE demonstrate sales researchers interest in examining ethical issues and also an interest by researchers in general, in using sales as a context for ethics research. Research Design In Table 2, we list the combined research design results for all 15 journals. Of all the empirical research design methods, survey research is the most commonly used, and it underlies a little more than half (54 percent) of the SALES articles published. Since , when survey research accounted for 50 percent of research designs, its use has increased steadily to percent in Other quantitative research designs, such as experimental simulation, field experiments, judgment tasks, measurement development, meta-analysis, and field studies, are not used as often as surveys. Laboratory experiments make up only 4.09 percent of all SALES articles, and their use has declined from a peak of 5.47 percent in to just 1.74 percent in The use of qualitative methods has increased though, after initially dropping from 7.28 percent ( ) to a low of 3.55 percent ( ), to reach 8.70 percent in the most recent (2005 8) period. At the journal level, the rate of conceptual article publication varies widely. The three premium journals (JM, JMR, MS) rarely publish conceptual articles; none of the 44 SALES articles in JMR were conceptual. However, specialized sales and B2B journals such as JPSSM, JBIM, and IMM publish more of them, such that approximately one-third of all their SALES articles are conceptual. Survey research consistently emerges as the most commonly used empirical research approach. Even in psychology journals, such as JAP, JCM, and P&M, which tend to be noted for their extensive use of experiments, survey design was the top approach for conducting SALES research. However, contrary to our expectations, consumer- and psychology-oriented journals (JAP, JCM, and P&M) were not the most frequent publishers of lab experiments. Rather, ML offered the highest proportion of its SALES articles using lab experiments, with 25 percent, followed by P&M (23 percent). Nonpsychology journals such as JBR (12 percent), JBE (10 percent), and JMTP (10 percent) published more articles using lab experiments than JAP (9 percent), and JCM did not publish a single SALES article based on lab experiments.

5 Fall Table 1 Number of SALES Articles Published in Different Journals JPSSM IMM JBIM JAMS JBR JM JMTP JMR JBE JAP P&M IJRM MS ML JCM Total Percent Total JPSSM IMM JBIM JAMS JBR JM JMTP JMR JBE JAP P&M IJRM MS ML JCM Total ,346 Percent

6 478 Journal of Personal Selling & Sales Management Table 2 All Journals Combined Total Research Design (PI) Conceptual Survey Judgement task Laboratory experiment Experimental simulation Field experiment Field study Pure methodology Case study Qualitative Meta-analysis Measurement development Triangulation (EPI) Between-method In-method Data Collection (EPI) Primary data Secondary data Multiple respondents International sample Cross-sectional Two time periods Longitudinal Construct Measurement and Development (EPI) Exploratory factor analysis Confirmatory factor analysis Cronbach s alpha Composite reliability Average variance extracted (AVE) Discriminant validity Interrater reliability Early late response bias evaluated Common method bias evaluated Dependent Variable Properties (EPI) Tangible outcomes Performance rating Perceptual/attitudinal Single Two Three Four or more Data Analytic Techniques (EPI) t-tests Chi-square Correlations (hypothesis testing only) Simple regression Multiple regression Stepwise regression Hierarchical regression ANOVA/ANCOVA

7 Fall Total MANOVA/MANCOVA Nonparametric Clustering techniques Structural equation modeling Conjoint analysis Discriminant Logit/tobit Multidimensional scaling Qualitative methods are less common in SALES research. Several journals (e.g., JAMS, IJRM, ML, JMR, MS ) did not publish a single SALES article with qualitative approaches during the 29-year period. The largest proportions appeared in IMM (13 percent) and JBIM (13 percent), followed by JM (10 percent). Meta-analysis is also not common in the SALES literature. Only six journals (JBIM, JAP, P&M, JPSSM, JMR, and JBR) published any meta-analyses, and even these featured at most three such articles (i.e., JMR and JPSSM ). We also recognize that the development of measures has not been a common approach utilized in the SALES literature. Of the journals included, seven never published articles whose primary purpose was to develop measures. Eight percent of SALES articles published in P&M focus primarily on developing measures and the journal publishes the highest proportion of articles that develop measures, followed by JMR (7 percent). Triangulation Approximately 11 percent of the empirical articles we examine used a form of triangulation, and such usage has increased from 9 percent ( ) to 14 percent (2005 8). While in-method triangulation usage has increased from 2 percent to 7 percent in the same periods, respectively, between-method triangulation has been relatively steady, averaging around 7 percent per year. The increase in in-method triangulation mirrors the increased use of multistudy research. At the journal level, triangulation is not widespread, such that a few journals published relatively many articles with triangulation approaches, but a lot of them did not publish any. Thus JCM, JAMS, ML, JMR, and MS included no SALES articles using between-methods triangulation, and IMM, JBIM, JMTP, JBE, and ML did not publish any in-methods triangulation SALES articles. Thirty-one percent of the SALES articles in P&M used between-methods triangulation; it is by far the journal with the largest proportion of articles using between-methods approaches, followed by JPSSM with 8 percent, IMM with 7 percent, and JAP with 6 percent. As expected, the psychology journals featured the highest proportions of in-method articles because they tend to publish articles with multiple experiments: P&M at 12 percent and JAP with 9 percent. Data Collection Methods Primary data are by far the leading source, accounting for 87 percent of data collected. Although the collection of primary data has declined from its high of 91 percent ( ) to 85 percent (2005 8), it continues to dominate. Secondary data account for 8.96 percent of the articles we examine, and its use increased from 7 percent ( ) to 12 percent (2005 8). Approximately 9 percent of SALES articles used multiple respondents; thus, the majority of articles use single respondents. On average, 10 percent of studies use international samples, though such usage has increased considerably, from 2 percent ( ) to 20 percent (2005 8). We also find a predominance of empirical articles using cross-sectional data (75 percent), though with a gradual decrease from a peak of 83 percent ( ) to 74 percent (2005 8). Data collected over two or more time periods appear in a relatively small percentage of empirical articles in our study (8.38 percent), though this usage has been gradually growing from 4 percent in to 10 percent in At the journal level, the use of primary data is consistently high; none of the journals featured less than 60 percent of their SALES studies with primary data. Although usage rates for secondary data were low overall, we find relatively high rates in two premium journals, JMR (21 percent) and JM (18 percent). Articles with international samples are increasing, and in six journals more than 10 percent of their SALES empirical articles used them: IJRM (31 percent), JBR (22 percent), JBE (20 percent), JAP (18 percent), IMM (15 percent), P&M (15 percent), and JBIM (11 percent). Cross-sectional data are the overwhelming favorite across the board, whereas longitudinal data collection is not common. However, JAP (25 percent) shows the highest proportion of published SALES articles using longitudinal data collection, followed by JMR (14 percent) and IJRM (12 percent). Most journals (11) featured longitudinal data collection in two or fewer articles.

8 480 Journal of Personal Selling & Sales Management Construct Validity, Reliability, and Response Bias EFA is commonly used in SALES articles, such that 21 percent of SALES empirical articles used it. The use of EFA has been consistently high, and since the period, the journals we examine have featured it in at least 20 percent of their SALES articles. The use of Cronbach s alpha also has been consistently high, in that 48 percent of the articles test scale reliability using Cronbach s alpha. This rate has increased noticeably from 29 percent ( ) to 64 percent (2005 8). In the period, percent articles reported a CFA, 30 percent used AVE, and 42 percent employed discriminant validity. Tests for response bias were reported by only 3 percent of articles between 1980 and 1984, but their use increased considerably to 33 percent (2005 8). Tests for common method bias also increased: Before 1990, no articles reported testing for common method bias, whereas 15 percent did so between 2005 and In the individual journals, we can detail the popularity of these approaches. Cronbach s alpha is widespread across all the journals, as are EFA, CFA, and discriminant validity. However, MS published no articles using EFA or discriminant validity and only one with CFA. The use of AVE and composite reliability measures are less popular: Six journals (MS, JCM, JAP, JMTP, JBE, and ML) did not publish a single SALES article that used AVE, and six (JCM, JBIM, JMTP, JBE, JAP and ML) did not publish any with composite reliability measures. In contrast, JAMS (13 percent) and IJRM (15 percent) published the largest proportion of SALES articles with composite reliability. These two journals also featured the highest percentage of articles using AVE, at 17 percent and 12 percent, respectively. Dependent Variable Properties The majority of SALES articles use perceptual/attitudinal dependent variables, and this usage has been consistently high, averaging 63 percent. Performance ratings are another common dependent variable, appearing in 18 percent of all empirical articles. Its use has increased considerably, from 10 percent ( ) to 26 percent (2005 8). The use of tangible outcomes similarly increased from 8 percent ( ) to 14 percent (2005 8), after dipping to a low of 6 percent between 1995 and Although the number of studies using single dependent variables decreased only slightly from 20 percent ( ) to 19 percent (2005 8), the use of multiple dependent variables increased. Empirical articles that used two dependent variables constituted 5 percent in but 11 percent in Similar increases occurred among articles using three (6 percent to 13 percent) and four or more (17 percent to 31 percent) dependent variables. At the individual journal level, perceptual/attitudinal dependent variables are by far the most common dependent variables. In six journals (JMR, JBE, JBR, JAP, IMM, and JPSSM) we also find more articles with multiple dependent variables than with single dependent variables. Data Analytic Techniques Our analysis reveals an increase in the use of several multivariate techniques, including SEM, multiple regression, MANOVA/ MANCOVA (multivariate analysis of covariance), and logit/ tobit. Overall, ANOVA/ANCOVA (analysis of covariance) is the most widely used technique, in 21 percent of all SALES empirical articles, followed closely by multiple regressions (20 percent). Over the 29-year study period, SEM (19 percent) has been the third-most-common technique, though in the most recent period (2005 8), it became by far the most common (45 percent). In , less than 1 percent of empirical articles in the SALES literature used SEM, so this growth has been extraordinary. Furthermore, MANOVA is the fourth most popular analytical technique (10 percent). During the most recent period, the four most commonly used multivariate tools (ANOVA, MANOVA, SEM, and multiple regression) appeared in 85 percent of all empirical articles, which demonstrates the increasing dominance of multivariate techniques in the SALES literature. Other multivariate techniques, such as logit/tobit, discriminant, hierarchical regression, multidimensional scaling, and conjoint analyses, are less commonly used, and we find no clear trends in their use. Unlike multivariate techniques, the use of simpler analytical tools, such as t tests, chi-square, and correlations, has been decreasing, except for simple regressions, which marginally increased from 0 percent ( ) to 1 percent (2005 8). Although the four major multivariate methods (multiple regression, ANOVA, MANOVA, and SEM) account for the majority of data analytic techniques in individual journals, their usage proportions differ greatly. Among the premier journals, they accounted for a significant proportion of data analysis tools, though in MS only one article used multiple regression. In JM, the overwhelming majority of articles (93 percent) included at least one of the four methods; SEM was most popular (29 percent), closely followed by ANOVA (25 percent) and multiple regressions (24 percent), whereas MANOVA (15 percent) was the least popular. The same overall assessment holds for JMR, though the proportion was lower (80 percent), and the popularity of the four methods differed: multiple regression (32 percent), followed by ANOVA (21 percent), SEM (18 percent), and MANOVA (9 percent). Another highly ranked journal, JAMS, emphasized the four methods, such that 90 percent of its SALES articles used at least one. As in JM, SEM accounted for the largest proportion (37 percent), followed by multiple regression (23 percent), MANOVA (6 percent), and ANOVA

9 Fall Table 3 Trend Analysis for All Journals Initial Status Rate of Change Dependent Variables Estimate Standard Error Estimate Standard Error R 2 Research Design (PI) Survey 55.31* ** Judgement task 1.91* * Pure methodology 2.81* * Triangulation (EPI) In-method 3.55* * Data Collection (EPI) International sample 11.47* * Construct Measurement and Development (EPI) Confirmatory factor analysis 21.41* * Cronbach s alpha 48.72* * Composite reliability 5.83* * Average variance extracted (AVE) 5.90* * Discriminant validity 13.80* * Interrater reliability 2.16* ** Early late response bias evaluated 13.23* * Common method bias evaluated * * Dependent Variable Properties (EPI) Perceptual/attitudinal 61.96* * Single 19.77* * Two 8.43* ** Data Analytic Techniques (EPI) Correlations 9.08* * Nonparametric * * Structural equation modeling 17.60* * Logit/tobit * * Notes: Independent variable is mean-centered Year. t-test: two-tailed. 1 Estimation method is Cochrane Orcutt, and all the others are OLS. ** Significant at p < 0.10; * significant at p < (4 percent). The psychology journals also featured the four approaches but in different proportions. As expected, ANOVA (39 percent) accounted for the largest proportion in psychology journals, followed by multiple regression (19 percent), SEM (18 percent), and MANOVA (8 percent), though JAP published more articles with multiple regressions (30 percent) than ANOVA (24 percent), SEM (21 percent), or MANOVA (3 percent). Trend Analysis In Table 3 we summarize the results of our trend analysis to test for statistically significant shifts in the use of the different research methods. The results show some notable trends. In terms of research design, for example, three methods underwent statistically significant shifts: survey research increased (β c = 0.50, p < 0.10), but pure methodology articles (β c = 0.33, p < 0.05) and judgment tasks (β c = 0.18, p < 0.05) significantly decreased over the years. In the triangulation section, we find that the use of inmethod triangulation has significantly increased (β c = 0.33, p < 0.05); for data collection, international samples (β c = 0.90, p < 0.05) have increased significantly too. There has been a considerable increase in the use of different methods to validate and determine construct reliability. With the exception of EFA, all the techniques in the construct validation section show significant increases, namely, CFA (β c = 1.94, p < 0.05), Cronbach s alpha (β c = 1.51, p < 0.05), AVE (β c = 0.94, p < 0.05), composite reliability (β c = 0.71, p < 0.05), and discriminant validity (β c = 1.21, p < 0.05). The efforts to test for bias follow a similar pattern, such that the rates of change for response bias (β c = 0.95, p < 0.05) and common method

10 482 Journal of Personal Selling & Sales Management Table 4 JPSSM and Non-JPSSM Comparison Dependent Variables t Significance (Two-Tailed) Non-JPSSM JPSSM Standard Error Difference Research Design (PI) Conceptual Judgement task Data Collection (EPI) International sample Data Analytic Techniques (EPI) Stepwise regression Multidimensional scaling bias (β c = 0.39, p < 0.05) are positive and significant. Interrater reliability is the only approach with a negatively significant shift (β c = 0.14, p < 0.05). In the dependent variable section, the rates of change in the use of perceptual (β c = 0.75, p < 0.05), single (β c = 0.46, p < 0.05), and dual (β c = 0.27, p < 0.10) dependent variables are positive and significant. The trend analysis for the various data analysis techniques reveals four statistically significant shifts: Both SEM (β c = 1.57, p < 0.05) and logit/tobit (β c = 0.23, p < 0.05) usages have significantly increased, but the use of correlations to test hypotheses declined (β c = 0.52, p < 0.05), as did the use of nonparametric approaches (β c = 0.27, p < 0.05). JPSSM Versus Non-JPSSM Articles We compare articles published in JPSSM against aggregate of the SALES articles published in all the other substantive journals. With this comparison, we compare the modal research design, measurement, and analysis characteristics of articles published in the premier sales journal with those of journals that also publish SALES research articles. Of the 1,346 journal articles we examine, 567 (42 percent) were published in JPSSM. As we show in Table 4, t tests for the use of different research methods reveal numerous differences, though few of those differences are statistically significant. An area of significant difference is the publication of conceptual articles: Non-JPSSM journals published significantly fewer conceptual articles than JPSSM (t = 2.27, p < 0.05). Furthermore, non-jpssm articles included significantly more international samples than JPSSM articles (t = 2.31, p < 0.05). Non-JPSSM journals published significantly fewer articles using stepwise regression than JPSSM (t = 1.82, p < 0.10). Finally, the non-jpssm journals did not publish SALES articles with multidimensional scaling, whereas JPSSM published many studies using that approach (t = 7.08, p < 0.01). In addition, we conducted trend analyses to identify and compare patterns of stability and change in the use of research methods. Table 5 summarizes the results for JPSSM articles, and Table 6 summarizes those for non-jpssm articles. They indicate similar trends for construct validation and reliability, in that both JPSSM and non-jpssm articles show significant growth in the use and reporting of CFA, Cronbach s alpha, composite reliability, AVE, discriminant validity, response bias, and common method bias. They also reveal significant increases in the use of in-method triangulation methods, which implies that published articles used more multiple studies. Other areas of significant, common growth entailed the use of SEM and international samples. Yet these comparisons also reveal some key differences. Although both types of journals published fewer conceptual articles over time, this reduction was significant only for JPSSM. Similarly, the significant decline in the use of stepwise regressions was limited to JPSSM. The greater use of survey research in the trend analysis is significant in JPSSM but not in non-jpssm articles. Other areas of significant growth that occurred only in JPSSM include the use of meta-analysis, performance-based dependent variables, and simple regression. In contrast, areas of significant growth that occurred only in the non-jpssm journals include the use of judgment task based research, pure methodology, perceptual dependent variables, chi-square, and logit. We find significant reductions in interrater reliability and the use of correlations to test hypotheses only in non-jpsmm articles. IMM Versus JPSSM Articles We compared SALES articles published in IMM with those published in JPSSM because these two journals publish the most SALES articles by far (243 and 567, respectively). Comparing research methods in these two leading outlets helps us reveal similarities and differences, as well as identify and compare patterns of stability and change in research methods used by SALES articles published in IMM and JPSSM. The t tests reveal considerable differences for 40 percent of the research methods, as we summarize in Table 7. In the

11 Fall Table 5 JPSSM Trend Analysis Initial Status Rate of Change Dependent Variables Estimate Standard Error Estimate Standard Error R 2 Research Design (PI) Conceptual * * Survey 52.41* ** Meta-analysis 0.44** * Triangulation (EPI) In-method 5.29* ** Data Collection (EPI) International sample 2.57* * Construct Measurement and Development (EPI) Confirmatory factor analysis 21.26* * Cronbach s alpha * * Rho * Average variance extracted (AVE) ** * Discriminant validity 16.67* * Early late response bias evaluated 18.06* * Common method bias evaluated * Dependent Variable Properties (EPI) Performance rating 24.39* * Two 12.07* ** Data Analytic Techniques (EPI) Simple regression * * Stepwise regression 3.65* ** Structural equation modeling * * Notes: Independent variable is mean-centered Year. t-test: two-tailed. 1 Estimation method is Cochrane Orcutt, and all the others are OLS. ** Significant at p < 0.10; * significant at p < research design section, IMM published significantly fewer SALES articles with judgment tasks (t = 3.52, p < 0.01), laboratory experiments (t = 3.86, p < 0.01), experimental simulation (t = 1.87, p < 0.10), meta-analysis (t = 1.82, p < 0.10), and measure development focuses (t = 2.01, p < 0.05), but more articles centered on pure methodology (t = 2.02, p < 0.05). Furthermore, IMM published significantly fewer articles that used in-method triangulation than did JPSSM. In terms of data collection, IMM published significantly fewer articles with secondary (t = 2.51, p < 0.01) and longitudinal (t = 2.48, p < 0.05) data. However, it contained far more articles that used international samples (t = 2.92, p < 0.01) compared with JPSSM. We find a significantly lower proportion of SALES articles in IMM that report the use of EFA (t = 3.24, p < 0.01), Cronbach s alpha (t = 2.22, p < 0.05), and discriminant validity (t = 2.33, p < 0.05), as well as of articles using chi-squares, multiple regression, SEM, or multidimensional scaling. The trend analyses summarized in Table 8 reveal, regarding the growth and decline in the use of different research methods over the years, that both journals followed similar trends when it came to construct validation and reliability. Both JPSSM and IMM sales articles exhibited significantly more use and reporting of CFA, Cronbach s alpha, composite reliability, AVE, discriminant validity, response bias, and common method bias. They also both significantly increased their publication of articles with SEM and international samples. Despite these commonalities, there were some areas where only one or the other had significant growth or declines over the years. For example, both journals reduced their use of conceptual articles, but the reduction was significant only in JPSSM. Another decline significant only in JPSSM was the use of stepwise regressions, whereas the growth in survey research was significant in JPSSM but not in IMM. The use of meta-analyses, performance-based dependent variables, and simple regressions also increased significantly but only in

12 484 Journal of Personal Selling & Sales Management Table 6 Trend Analysis of Non-JPSSM Articles Initial Status Rate of Change Dependent Variables Estimate Standard Error Estimate Standard Error R 2 Research Design (PI) Judgment task 1.63* * Pure methodology 2.92* * Triangulation (EPI) In-method 3.45* * Data Collection (EPI) International sample 12.49* * Construct Measurement and Development (EPI) Confirmatory factor analysis 21.69* * Cronbach s alpha 49.50* * Composite reliability 5.79* * Average variance extracted (AVE) 5.63* * Discriminant validity 13.71* * Interrater reliability 2.18* ** Early late response bias evaluated 12.79* * Common method bias evaluated * * Dependent Variable Properties (EPI) Perceptual/attitudinal 61.45* * Single 19.45* * Two 8.00* ** Data Analytic Techniques (EPI) Chi-square 3.56* ** Correlations 9.16* * Nonparametric 2.41* * Structural equation modeling 17.70* * Logit/tobit ** * Notes: Independent variable is mean-centered Year. t-test: two-tailed. 1 Estimation method is Cochrane Orcutt, and all the others are OLS. ** Significant at p < 0.10; * significant at p < JPSSM. In contrast, articles using three or more dependent variables grew significantly in IMM, and the journal revealed significant reductions in the use of pure methodology and nonparametric articles. Discussion Our main goal has been to examine the use of different research methods and techniques in SALES literature, and our results reveal some important trends and shifts. In particular, the data indicate a clear trend toward more empirical focus in sales force research, as illustrated best by the increase in empirical articles at the expense of conceptual ones. This increase reflects trends in academia in general, including a general push for more theory testing research. Another trend that indicates the increasing rigor of SALES research is the dominance of multivariate statistical techniques, particularly SEM, MANOVA, ANOVA, and multiple regression, at the expense of basic methods, such as t tests, correlations, and chi-squares. Although 38 percent of SALES empirical articles published in used at least one of the four multivariate methods, we find that the rate increased considerably to 85 percent by The most notable shift is the substantial increase in the use of SEM, from less than 1 percent in to 45 percent in The use of more sophisticated analytical tools demonstrates progress in the discipline s efforts to conduct more rigorous research; however, the dominance of just a few methods is troubling. The finding that 45 percent of empirical articles (2005 8) use a single analytical tool (SEM) is particularly disturbing because it, like any analytical tool, has limitations, so that a significant proportion of SALES studies might suffer the same types of errors. In this sense,