Causation or covariation: an empirical re-examination of the link between TQM and financial performance

Transcription

1 Journal of Operations Management 22 (2004) Causation or covariation: an empirical re-examination of the link between TQM and financial performance Kenneth M. York, Cynthia E. Miree 1 School of Business Administration, 316 Elliott Hall, Oakland University, Rochester, MI , USA School of Business Administration, 413 Elliott Hall, Oakland University, Rochester, MI , USA Received 11 February 2003; received in revised form 1 January 2004; accepted 25 February 2004 Available online 30 April 2004 Abstract Total Quality Management (TQM) is an integrated management system designed to focus an organization s resources on increasing the quality of a firm s products/services, satisfying customer needs and improving the efficiency of the processes that produce the firm s products/services. Advocates of TQM have suggested that there should be a positive relationship between implementing TQM practices and financial performance measures. The empirical evidence supporting this assertion, however, is limited at best. Most of the research has been limited to surveys of managers perceptions of the effect of TQM on financial performance. A few empirical studies using financial performance measures have been done, and have shown that TQM firms have better financial performance than other firms. However, better performing companies may be more likely to adopt TQM, so that rather than being a path to improved financial success (causation), TQM merely comes along for the ride (covariation). This study examined the relationship between TQM and financial performance, using a sample of Baldrige Award winners and replicated with a second sample of state quality award winning companies, and three different sets of financial performance measures. Both Baldrige and state quality award winners generally had better financial performance than their peers after winning a quality award, and before Elsevier B.V. All rights reserved. Keywords: Quality; Financial performance; Empirical research 1. Introduction Total Quality Management (TQM) is a set of management methods and tools focused on providing superior value to the customer through identification of customers expressed and latent needs, responsive- Corresponding author. Tel.: ; fax: addresses: york@oakland.edu (K.M. York), miree@oakland.edu (C.E. Miree). 1 Tel.: ; fax: ness to changing markets, as well as on improving the efficiency of the processes that produce the product or service. Over the past few decades, TQM has run the cycle from initial obscurity, through rising popularity and unrealistic expectations, to skeptical re-evaluation (Fuchsberg, 1992, 1993; Greising, 1994; Main, 1991; Mathews, 1992). At the same time, the literature on TQM has also developed, thereby shaping our understanding of the role and impact of TQM in organizations. Some of the research on TQM has focused on the importance of effective design to the success of TQM programs and initiatives (Samson and /$ see front matter 2004 Elsevier B.V. All rights reserved. doi: /j.jom

2 292 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) Terziovski, 1999; Ahire and Dreyfus, 2000). Other studies have examined TQM failures and the various precursors to such outcomes (Murugesh et al., 1997), still other studies have explored the combined impact of TQM and other initiatives like JIT (Sriparavastu and Gupta, 1997; Flynn et al., 1995), business process re-engineering (Gonzalez-Benito et al., 1999) and other types of business innovation (Martinez-Lorente et al., 1999) on various aspects of business performance such as worker productivity, cycle times, defect rates and competitive advantage (Flynn et al., 1995). While these studies do seem diverse, the unifying empirical question underlying much of this literature stream seeks to determine if TQM actually impacts firm performance and if so, how. Advocates of TQM have suggested a link between the implementation of TQM methods and financial performance based on the costs associated with poor quality, rework, and warranties. In essence, rather than increasing costs, improving quality should actually reduce costs, and therefore have a positive effect on measures of financial performance. In addition, increasing the quality of the product or service should increase the retention rate for current customers and attract new customers, thereby improving market share and revenues (Rust et al., 1994, p. 5; Jaworski and Kohli, 1993; Kohli and Jaworski, 1990; Narver and Slater, 1990; Slater and Narver, 1995; Sterman et al., 1997). Further, superior quality products or services should also produce greater customer loyalty, higher stock prices, reduced service calls, and greater productivity (Omachonu and Ross, 1994, p. 14; Hendricks and Singhal, 1996; Rust et al., 1994, p. 1). The purpose of this study is to re-examine the link between TQM and financial performance both in the literature and through additional empirical examination. This paper builds on and improves on previous studies in the following ways. First, this paper compares the financial performance of each quality award winner against its entire SIC group. Second, financial performance is compared against the reference group both before and after winning a quality award. Third, three sets of financial performance measures are used to evaluate performance. Finally, we replicate our results using a sample of state quality award winners. 2. Literature review The basic theoretical foundation for the link between the effective implementation of TQM methods and financial performance is based on two expected relationships. First, TQM focuses on the organization s efforts to create and retain customers, which leads to increased revenues by gaining a market advantage, and reduced costs through product design efficiency. Second, TQM focuses on the organization s efforts to improve the processes that produce their products and services, which leads to increased revenues through product reliability, and reduced costs through process efficiency (George and Weimerskirch, 1998, p. 7;Reed et al., 1996; Anderson et al., 1995). In short, doing the right thing, and doing it more efficiently, should have a positive effect on many measures of the financial performance of the firm. The conceptual model of TQM is shown in Fig. 1. Customer Satisfaction Market Orientation Value for customers Responsiveness to changing marketplace needs Improved Efficiency Reduced waste, rework, reprocessing Reduced cost of conformance and cost of non-conformance Financial Performance Sales Market share Profits Fig. 1. TQM and financial performance model.

3 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) While the theoretical framework behind TQM may be intuitively appealing, designing a rigorous empirical test of this relationship is difficult at best. These difficulties primarily arise from some of the unique challenges associated with this research question. First, TQM winners are often subsidiaries of larger corporations, thus requiring the use of aggregated data to assess TQM s impact on performance. Second, as the number of TQM awards and the variety of criteria for winning an award has increased over the years, developing a consistent and replicable sample of TQM firms remains a methodological challenge. Further, the proliferation of quality awards coupled with the differences in the degree to which TQM must be implemented within the firm to obtain one of these awards makes the comparison of results within the literature difficult at best (e.g., Forker, 1997; Hendricks and Singhal, 1997). Third, international or cultural-specific differences can create differences in a company s quality improvement approach and resultant quality and financial performance (Adam et al., 1997). Finally, choosing an appropriate time frame for the study such that firms are able to experience the theoretical changes in market share or efficiency that can impact the data used to assess TQM s impact on financial performance remains a challenge. To this end, the general literature about TQM s impact on firm performance is mixed (see Martinez-Lorente et al., 2000) and the true nature of relationship between TQM and financial performance remains unclear. To test the relationship between TQM and financial performance, a number of researchers have used surveys or interviews to measure financial performance by collecting respondents opinions about the firm s financial performance (e.g., Forker, 1997; Mann and Kehoe, 1994; Anderson et al., 1995; Mohrman et al., 1995; Adam, 1994; General Accounting Office, 1991). The limitation of this approach is the reliance on survey respondents to provide opinions (which may have a strong demand characteristic), rather than using externally reported information which is subject to standard accounting practices and scrutinized by professionally qualified auditors. A typical example of this approach is Powell (1995), who analyzed data from 36 surveys and 18 on-site interviews of public and private companies. Financial performance was measured subjectively, using a five-item scale addressing profitability, sales, growth, and overall financial performance. This study is suggestive, but not persuasive evidence of the link between TQM and financial performance because respondents self-selected into the study, and then provided only opinions about the financial performance of their own companies, rather than actual financial performance data. Similarly, in an analysis of 20 studies of the impact of TQM conducted by a variety of consulting firms, business associations, and membership organizations such as the Conference Board and the American Society for Quality Control, Hiam (1993) concluded that TQM can improve company performance on market share, process costs, profits, and return on investments. However, most of these studies were also based on surveys of management perceptions, with little independent measurement of the impact of TQM practices on performance. Hiam (1993) concludes that more detailed and objective studies are needed to refine the understanding of the general impact of TQM. In an effort to overcome this limitation and provide evidence of a positive relationship between TQM and financial performance, Rust et al. (1994) report that a series of studies using the Strategic Planning Institute s PIMS (Profit Impact of Marketing Strategy) database of 3000 business units, found positive relationships between return on investment and reported quality levels, and between quality and market share growth. More specifically, Buzzell and Gale (1987, pp ) claim that there is no doubt that relative perceived quality and profitability are strongly related, whether the profit measure is return on sales or return on investment. While many of the aforementioned studies have significant design limitations, there are other studies that have applied more rigor to this research question with fairly good results. Hendricks and Singhal (1997) explored the impact of effective TQM implementation on the operating performance of firms over a 10-year period and found that firms winning quality awards outperformed firms in a matched control sample in terms of operating performance and sales growth. In addition they found that these same firms were also more successful in controlling costs. Their sample was based on 394 firms from 42 distinct 2-digit SIC codes matched with firms with similar fiscal year ends and the same 2-digit SIC codes (1-digit codes were used for larger firms). In total, nearly 100 different types of quality awards

4 294 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) were represented in their sample and they examined 10 years of data (6 years before receiving the award and 3 years after winning the quality award). While they do provide some evidence that TQM does have a positive effect on performance, the key limitations of their study are typical of more rigorous examinations of TQM and financial performance. First, the use of firm-wide financial data does not adequately capture the true financial impact of TQM implementation when the award winner is a single division of a larger firm. Nevertheless, the use of firm-wide data reflects a common practice. There are difficulties in finding subsidiary data; the Standard and Poor s data set includes only firm-wide data, and other data bases which contain subsidiary data have other limitations, such as self-selection into the data base. If researchers use firm-wide data, they should first test for significant differences in financial performance between when a company or a subsidiary wins a quality award, and then test for significant differences between quality award winners and other companies. Second, while the use of winning a quality award as a proxy for effective TQM implementation is appropriate, when multiple types of quality awards are included in the sample (with varied criteria, standards, and levels of competition) there may be some cause for concern. Finally, the results do not provide compelling evidence of causation by failing to examine any other conditions that may lead some firms to benefit from the adoption of TQM over others. Nonetheless, their study filled an important gap in the TQM literature as it adopted the long-term perspective necessary to truly understand the impact of organizational change on performance. In a similar study, Easton and Jarrell (1998) looked at 108 firms from 32 different industries over the 5-year period after winning a quality award to determine the effect of successfully adopting TQM on firm performance. Like Hendricks and Singhal (1997), this study also uses parent company-level Compustat data to detect changes in financial performance, includes a number of different non-specific types of quality awards and ignores potential enabling factors. At the same time, Easton and Jarrell advance our understanding of TQM s effect on financial performance by dividing the firms in their sample into two groups: those with more advanced TQM systems and those with less advanced TQM systems. Easton and Jarrell s (1998) approach was to compare the performance of 108 TQM firms to a benchmark performance measure of predicted performance without TQM. A firm s unexpected (i.e., better than expected) performance was the difference between the firm s actual performance and an analyst s forecast, which was then compared to the unexpected performance of a matched (on industry, time period, analysts s projection of future performance, market size, debt-to-equity ratio, and market risk factor) portfolio of three firms that had not implemented TQM. In this approach, each company serves as its own control, with each company compared first to itself (i.e., its predicted performance), and then to a matched sample of non-tqm companies. Easton and Jarrell found that financial performance was improved for firms that adopted TQM, and the improvement was greater for firms with more advanced TQM systems. Further, during the time period under investigation, Easton and Jarrell found that the long-term performance of TQM firms was improved. More recently, Hendricks and Singhal (2001a,b) expand their previous work and our understanding of the relationship between TQM and financial performance. Hendricks and Singhal (2001b), move beyond their 1997 work which considered TQM s impact on operating performance and track the long run stock price performance of firms with effective TQM implementation both before and after winning a quality award. They also employ a more precise measurement technique that utilizes buy and hold abnormal returns and overcomes some of the measurement issues associated with previous measures of abnormal returns used in the literature (e.g., Easton and Jarrell, 1998). Using three control groups (industry-matched, N = 603; industry-size-matched, N = 507; and industry-size book-to-market value ratio-matched, N = 595), they found that firms do experience long-term financial benefits in terms of buy and hold returns. In Hendricks and Singhal (2001a), the authors make another significant contribution to the TQM literature by considering the impact of various firm characteristics on TQM s ability to improve financial performance. In particular, they examine the moderating effect of size, capital intensity, degree of diversification, and type of award (independent versus supplier) on performance. Their results are based on a sample of 435 firms. While they find generally that firms that have effectively implemented TQM experience improvement in their financial performance,

5 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) Table 1 Comparison of three key empirical studies of TQM and financial performance Dimensions Hendricks and Singhal (1997) Easton and Jarrell (1998) Hendricks and Singhal (2001a) Sample size Data source, data used Compustat Annual Industrial File, Parent Firm Compustat Annual Industrial File, Parent Firm Compustat Annual Industrial File, Parent Firm Time frame 10 years (6 years before to 3 years after winning quality award) 5 years after winning quality award 4 5 years after winning quality award Types of quality awards included Categories of performance measures Key findings Not explicitly stated: various quality awards from nearly 100 award providers Not explicitly stated: various quality awards Baldrige Award winners, more than 26 independent and more than 26 supplier quality awards 1. Operating Income 1. Net income 1. Operating Income 2. Annual percent change in sales 2. Operating Income 2. Annual percent change in sales 3. Percent change in total cost per dollar of sales 3. Sales 3. Percent change in cost per dollar of sales 4. Continuously compounded stock returns Firms winning quality awards outperform the control firms on Operating Income-based measures and sales growth, and are more successful at controlling costs Long-term performance of TQM firms is improved, as measured by median cumulative positive abnormal return at the end of 5 years Overall, TQM firms experience better financial performance than control firms; smaller firms do better than larger firms; less capital intensive firms do better than more capital intensive; less diversified firms outperform more diversified firms; no significant differences between early and later adopters of TQM they also find that small firms tend to benefit more from TQM when compared to larger firms, and high capital-intensive firms do not benefit as much from TQM as lower capital-intensive firms. In addition, they find that more focused firms tended to benefit more from TQM than more diversified firms, and that independent award winners demonstrate significant improvement in cost per dollar of sales. Nevertheless, the authors highlight the benefits of winning supplier awards as a precursor for pursuing national and independent quality awards. A summary of these key empirical studies is shown in Table Hypotheses The above literature review, though not intended to be exhaustive, does provide a representation of TQM studies and lends evidence to a relationship between TQM and financial performance. At the same time, each of these analyses of TQM firms have neglected to rule out the additional and equally important hypothesis TQM firms had better financial performance before they won the award, perhaps even for years before they adopted TQM methods. In other words, rather than being a path to improved financial success (causation), TQM merely comes along for the ride (covariation). Any analysis which shows that TQM-managed firms currently outperform their non-tqm competitors should attempt to rule out the alternative hypothesis that TQM-managed firms also had better financial performance before they won the award, thereby instigating the argument that better performing companies are not only more likely to adopt TQM, but are also more likely to have the resources necessary to successfully implement the practices and reap its accompanying benefits. To this end, we tested the following hypotheses: Hypothesis #1. Baldrige Award winning firms will experience better financial performance than

6 296 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) non-baldrige Award winning firms, both before and after winning the quality award. Hypothesis #2. State quality award winning firms will experience better financial performance than non-winning firms, both before and after winning the quality award. In the next section of the paper we discuss how the samples were created, and how the financial performance measures were developed. Our approach was to do three different analyses with three different time frames. Starting with the year of winning the award, analyses were done for 5 years after winning the award (excluding from the analysis companies which won a quality award after 1992), a second set of analyses were done since the company won the award (the earliest Baldrige winner was in 1988, the earliest state winner was in 1983), and a third set of analyses were done for the years 1997, 1992, 1987, and Sample, data, methods 4.1. Data sets Compustat PC Plus v6.31 was used to create the data sets, using data. We chose this time frame because it roughly overlaps the time frames used in the comparison studies listed in the literature review, and does not include Baldrige Award winners under the most recent revision of the criteria which places greater emphasis on business results, a change that could potentially skew the results of this study. The list of Baldrige Award winners was obtained from the National Institute of Standards and Technology (NIST, 2004) web site. The list of state quality award winners was assembled from web sites for the awards, and by responses to and letters to 32 state quality award administrators requesting a list of past winners. We did not include quality awards given by companies to their suppliers because those awards might be based on very different criteria than the Baldrige Award, or political criteria. A list of the state quality awards used in the data set is shown in Table 2. In most cases, a subsidiary or division of a company has won the quality award (75% of Baldrige Winners, 74% of state quality award winners). In all Table 2 List of state quality awards in state quality award data set Arkansas Quality Award California Quality Award Commonwealth of Kentucky Quality Award Connecticut Award for Excellence Edgerton Quality Award (Nebraska) Florida Sterling Award Georgia Ogelthorpe Award Governor s Golden State Quality Award Governor s Award for Excellence (New York) Granite State Quality Award (New Hampshire) Hawaii State Award of Excellence Idaho Quality Award Kansas Award for Excellence Lincoln Award (Illinois) Louisiana Quality Award Margaret Chase Smith Maine State Quality Award Maryland Quality Award Massachusetts Quality Award Michigan Quality Leadership Award Mississippi Quality Award Missouri Quality Award New Jersey Quality Achievement Award New Mexico Quality Award North Carolina Quality Leadership Award Oklahoma Quality Award Winners Oregon Quality Award Rhode Island Quality Award South Carolina Governor s Quality Award Tennessee Quality Award Texas Quality Award U.S. Senate Productivity and Quality Award for Maryland U.S. Senate Productivity and Quality Award for Virginia Washington Quality Initiative Achievement Award Wyoming State Quality Award cases, however, corporate parents of divisions which won the award were included in the data set (e.g., GM data was used for Cadillac). Therefore, the data set includes companies where the award winning units later failed or were sold off (e.g., AT&T Universal Card Services won the Baldrige Award in 1992 and was sold to Citibank in 1998). Given the potential implications of these decisions we were initially concerned that this might confound our results. We therefore tested for a difference between when companies or a subunit won a quality award before proceeding with further analysis. A multivariate ANOVA (MANOVA) was calculated for each of the three sets of financial performance measures, using whether a subunit won as the classification variable. There were no significant effects for the Hendricks and Singhal financial performance mea-

7 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) Table 3 MANOVAs on financial performance measures for parent vs. subsidiary winning quality award Performance measures Baldrige winners State winners F-ratio Probability F-ratio Probability Hendricks and Singhal Fortune Untestable Untestable 1982 Untestable Untestable Kaplan and Norton Note. Some MANOVAs were untestable due to too much missing data. Statistically significant at 0.05 level. sures, the Fortune measures, or Kaplan and Norton measures for the 1997, 1992, 1987 and 1982 data, except for the Hendricks and Singhal measures in 1987 and 1982 for the Baldrige Award data set (the Fortune measures were untestable for 1987 and 1982 due to too much missing data). When outliers were identified for these two tests (Studentized Residual greater than 3.0) and removed from the analysis, there was no significant effect for the Hendricks and Singhal measures in 1982 (F(3, 11) = 2.76, P = 0.092). These results indicate that there were few significant differences in the financial performance measures between companies that won quality awards and companies where a subsidiary won a quality award, and this result was consistent across both the Baldrige and state quality award winners. The MANOVAs for the Baldrige and state quality award winners are shown in Table 3. Following the work of Hendricks and Singhal (1997, 2001a), winning a quality award was used as a proxy for effective TQM implementation. Hendricks and Singhal (2001a, p. 274) give five reasons to make their case that winning a quality award is a proxy for effective TQM implementation, to which we add a sixth: (1) The core concepts and values emphasized are typically those that are widely considered to be the building blocks of effective TQM implementations; (2) Quality awards are given after the applicant typically goes through a multi-level evaluation process where a group of experts judge the applicants against the criteria, thereby ensuring award winners are effectively implementing and practicing TQM; (3) There is a fair amount of academic literature that has used various quality award criteria to describe and research various aspects of TQM (see e.g., Garvin, 1991; Easton, 1993; Hendricks and Singhal, 1996, 1997, 1999); (4) Many organizations use quality award criteria to define and benchmark their TQM practices and to conduct internal self-assessments (Hiam, 1993); (5) Most books that cover quality management typically discuss quality awards and their role in TQM (e.g., Omachonu and Ross, 1994; Petrick and Furr, 1995); and (6) The application process for the Baldrige Award, after which most state quality awards are modeled, requires evidence of major shifts in management philosophy, practices, and policies related to the pursuit of improved quality (Blackburn and Rosen, 1993). We had considered other approaches, such as a survey where managers perceptions are used to categorize companies as TQM-managed or non-tqm-managed (e.g., Mann and Kehoe, 1994; Mohrman et al., 1995; General Accounting Office, 1991), but the managers are not necessarily objective observers, and a survey asking about the use of TQM management methods has a strong demand characteristic influencing managers to say that their company is TQM-managed. The Easton and Jarrell (1998) approach of using interviews conducted by TQM experts is, as Hendricks and Singhal (2001a, p. 274) point out, similar to the process award givers follow to identify firms that deserve their quality awards. But it also relies upon subjective judgments rather than the hard financial data in Compustat. Before beginning our study, we concluded that using winning a quality award as a proxy for TQM implementation was the best approach to use, and would build on the work other researchers had published. Again following the work of Hendricks and Singhal (1997) we use the date of winning the Baldrige or state quality award as the indicator of effective TQM implementation. Hendricks and Singhal (1997, p. 1261) describe their rationale as follows: Easton and Jarrell report performance changes over a 5-year

8 298 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) period beginning at the approximate time when firms in their sample started serious efforts to implement TQM. Given that it can easily take 3 5 years to implement an effective TQM program, their results are more representative of what could be expected during a period of implementing TQM. In contrast, by using the year of winning the first quality award as an indicator that firms in our sample have an effective TQM program in place at that point, we examine both their TQM implementation period (a 5-year period prior to their first award) and the post implementation period (a 4-year period after their first award). We followed this approach and improved upon it, by doing multiple analyses using different time frames and three different sets of financial performance measures, and replicating the findings in the sample of Baldrige winners with a sample of state quality award winners. Although previous researchers (e.g., Hendricks and Singhal, 2001a,b) have included quality awards given by companies to their suppliers to get a larger sample size, the samples used in this study were restricted to Baldrige and state quality award winners. The Baldrige Award data set included 20 publicly traded companies which won the Baldrige Award during and all publicly traded companies with matching SIC codes (N = 906, with 19 4-digit SIC codes). The state quality award data set included 131 publicly traded companies which won a state quality award during and all publicly traded companies with matching SIC codes (N = 10,241, with digit SIC codes). The scope of years was limited to 1998 and prior years winners for two reasons: (1) to allow a period of time after winning the quality award to see an effect on financial performance measures, and other researchers have used 5 years after winning an award (see e.g., Easton and Jarrell, 1998; Hendricks and Singhal, 1997); and (2) to exclude from the analysis companies which won a quality award after the most recent revision in the Baldrige Award criteria, which puts more emphasis on process management and Business Results (Hertz, 1997) Financial performance measures The Compustat database contains a large number of financial measures. Three different sets of financial performance measures were created from the available variables. The Hendricks and Singhal (1997) measures are based on the authors three hypotheses about the link between TQM and financial performance, testing whether an effective TQM program would improve the profitability of the firm, increase revenues, and reduce costs, as measured by: Operating Income ($million), Net Sales ($million), and Cost of Goods Sold/Sales ($). The Fortune measures follow a broader set of financial performance measures reported in the annual Fortune (1999) list of the 500 largest U.S. corporations: Revenues ($million), Profits (Net Profit Margin 100), Assets ($million), Shareholders Equity ($million), Market Value ($million), Earnings Per Share ($), Earnings Per Share over 10 Years (%), Total Return (%), and Total Return over 10 Years (%). Although included in the Fortune 500 list, Profits as a Percent of Revenue, Profits as a Percent of Assets, and Profits as a Percent of Equity were not included because they are ratios of the other variables in the set of measures. The Kaplan and Norton (1992) measures are based on the authors Balanced Scorecard (Financial Perspective) and is measured by: Cash Flow ($million); Annual Sales Growth (%) and Operating Income ($million); Market Share (%) and Return on Equity (%). The Kaplan and Norton Balanced Scorecard specifies quarterly sales growth, the closest available variable in Compustat is Annual Sales Growth. Market Share is not in the Compustat database, so it was created by dividing Net Sales for each company by Total Net Sales for the SIC code for that company, i.e., Market Share is calculated within SIC code. There is only one variable in common across the three sets of financial performance measures: Operating Income in the Hendricks and Singhal and the Kaplan and Norton sets. Thus, these sets of performance measures provide three different perspectives on measuring the financial success of the firm. 5. Empirical results To address the two hypotheses, a number of different types of analyses need to be done. First, the

9 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) performance of TQM companies should be examined over time to determine whether their financial performance has been better than other companies in their industry, from the year they won the quality award to 5 years later. Second, the performance of TQM companies should be compared beyond the 5-year mark to determine the sustainability of any TQM advantage (Powell, 1995). If both these analyses indicate that TQM companies had better financial performance than non-tqm companies in their industries after winning a quality award, one must still address the alternative hypothesis that TQM companies also had better financial performance than non-tqm companies before winning the award. And third, to finish the analysis, the financial performance of TQM companies should be compared to non-tqm companies over an interval that includes a time period both before and after the award, across industries and by industry. The results of these analyses follow Correlations among financial performance measures Given the nature of the measures, we expected to find a number of intercorrelations among the financial performance measures. We therefore tested for intercorrelations to assess the viability of further analysis. Pearson Correlations were calculated on the 1997 data between each of the performance measures using listwise deletion and Bonferroni corrected probabilities. Using the Baldrige winners data set, for the Hendricks and Singhal performance measures, Operating Income was significantly correlated with Net Sales 0.84, and there was a mean intercorrelation (ignoring the sign) of For the Fortune performance measures, 19 of the 36 correlations were statistically significant, with a mean intercorrelation of The mean intercorrelation for the Kaplan and Norton performance measures was Similar results were found for the state quality award winners, with mean intercorrelations of Table 4 Pearson correlations among performance measures 1997 Baldrige data set OI NS CGS/S Hendricks and Singhal (N = 891) Operating Income (OI) Net Sales (NS) 0.84 Cost of Goods Sold/Sales (CGS/S) R P A SEQ MV EPS E10 TR TR10 Fortune (N = 2583) Revenues (R) Profits (P) 0.08 Assets (A) Shareholder Equity (SEQ) Market Value (MV) Earnings Per Share (EPS) Earnings Per Share over 10 Years (E10) Total Return (TR) Total Return over 10 Years (TR10) CF SG OI ROE MS Kaplan and Norton (N = 8984) Cash Flow (CF) Annual Sales Growth (SG) 0.01 Operating Income (OI) Return on Equity (ROE) Market Share (MS) Statistically significant at 0.05 level using Bonferroni corrected probabilities.

10 300 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) Table 5 Pearson correlations among performance measures 1997 state quality award data set OI NS CGS/S Hendricks and Singhal (N = 9210) Operating Income (OI) Net Sales (NS) 0.81 Cost of Goods Sold/Sales (CGS/S) R P A SEQ MV EPS E10 TR TR10 Fortune (N = 2583) Revenues (R) Profits (P) 0.02 Assets (A) Shareholder Equity (SEQ) Market Value (MV) Earnings Per Share (EPS) Earnings Per Share over 10 Years (E10) Total Return (TR) Total Return over 10 Years (TR10) CF SG OI ROE MS Kaplan and Norton (N = 8984) Cash Flow (CF) Annual Sales Growth (SG) 0.01 Operating Income (OI) Return on Equity (ROE) Market Share (MS) Statistically significant at 0.05 level using Bonferroni corrected probabilities. 0.27, 0.16, and Although these results limited the scope of the possible analyses, the majority of the intercorrelations were small; the median intercorrelation for the Baldrige sample was 0.18 and for the state sample The correlations among the performance measures for the Baldrige Award winners is shown in Table 4, and for the state quality award winners in Table Financial performance 5 years after winning a quality award The effect of TQM management methods may only be seen after they have been in place for a number of years. It would be unrealistic to expect a company to win a quality award and then see an immediate financial impact beyond reactions from the stock market. Guidelines in the literature suggest that 5 years after winning an award is an acceptable point to discern any impact of TQM practices on a firm s financial performance (see e.g., Easton and Jarrell, 1998; Hendricks and Singhal, 1997). To test for the effect of TQM methods 5 years after winning a quality award, we calculated the percent change in each of the measures (for all three performance sets of performance measures) for the Baldrige winners that won the award prior to 1993 and the matching SIC code sample (including itself, to make a more conservative test), using medians to control for extreme values. This analysis controls for the effect of industry by calculating percent change by SIC code, and comparing each quality award winner to the percent change for all companies in its SIC code. For example, Motorola s percent change in Operating Income compared to SIC 3663 was 126.3% compared to 22.8%; Net Sales 105.6% compared to 42.9%; and Cost of Goods Sold/Sales 51.4% compared to 8.3%. This analysis included 10 (50%) of the Baldrige winners and 22 (17%) of the state quality award winners prior to 1993; 50% of the Baldrige winners won prior to 1993 and 57% of the state quality award winners won prior to The results for the Baldrige Award winners are shown in Table 6, and for the state quality award winners are shown in Table 7.

11 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) Table 6 Mean percent change in financial measures for Baldrige winning companies for 5 years after winning the Baldrige Award Performance measures Variable Baldrige winners SIC match Hendricks and Singhal Operating Income 145.7% 46.9% Net Sales Cost of Goods Sold/Sales Fortune Revenues 134.9% 34.8% Profits Assets Shareholders Equity Market Value Earnings Per Share Earnings Per Share over 10 Years Total Return Total Return over 10 Years Kaplan and Norton Cash Flow 30.8% 18.4% Annual Sales Growth Operating Income Return on Equity Market Share Note. Mean percentage change is based on the difference in medians from Baldrige Award winning year and 5 years later, averaged across all companies. SIC match is the sample of matching SIC codes for each Baldrige Award winner; Baldrige Award winner percent change is compared to the median percent change for each Baldrige Award winner s matching SIC code sample separately. A larger percentage change for Cost of Goods Sold/Sales indicates a larger reduction in Cost of Goods Sold/Sales. Bold values are where the Baldrige winners had a greater mean percentage change in medians than SIC match. Table 7 Mean percent change in financial measures for state quality award winning companies for 5 years after winning the award Performance measures Variable State winners SIC match Hendricks and Singhal Operating Income 26.2% 24.1% Net Sales Cost of Goods Sold/Sales Fortune Revenues 40.7% 24.8% Profits Assets Shareholders Equity Market Value Earnings Per Share Earnings Per Share over 10 Years Total Return Total Return over 10 Years Kaplan and Norton Cash Flow 33.6% 9.8% Annual Sales Growth Operating Income 26.2% 24.1% Return on Equity Market Share Note. Mean percentage change is based on the difference in medians from state quality award winning year and 5 years later, averaged across all companies. SIC match is the sample of matching SIC codes for each state quality award winner; state winner percent change is compared to the median percent change for each state quality award winner s matching SIC code sample separately. A larger percentage change for Cost of Goods Sold/Sales indicates a larger reduction in Cost of Goods Sold/Sales. Bold values are where the state quality award winners had a greater mean percentage change in medians than SIC match.

12 302 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) For the Baldrige winners data set, the mean percent changes over the 5-year period in the Hendricks and Singhal set of performance measures were 98.8% greater for Operating Income, 100.1% greater for Net Sales, and 1.6% greater for Cost of Goods Sold/Sales (i.e., greater reductions in Cost of Goods Sold/Sales). Similar results were obtained for the Fortune and Kaplan and Norton performance measures. Thus, in general, the quality award winners had better financial performance 5 years after winning the award compared to the companies in their SIC code. For the Hendricks and Singhal measures, the Baldrige winning company had better financial performance than its SIC code sample in 59.3% of cases, in 69.6% of cases in the Fortune measures, and 46.7% for the Kaplan and Norton measures. For the state quality award winners the percentages were similar for the three measures, with 59.5, 58.7, and 58.6% of cases. When we compared these results to those of state quality award winners, we found that state winners experienced better financial performance in Operating Income and Net Sales (Hendricks and Singhal measures); Revenues, Assets, Shareholders Equity, Market Value, Earnings Per Share, Earning Per Share over 10 Years, Total Return, and Total Return over 10 Years (Fortune measures); and Cash Flow, Operating Income, and Market Share (Kaplan and Norton measures). Although these results give a general indication that TQM-managed firms had better financial performance 5 years after winning a quality award, they should be interpreted with caution because of the small sample size. These results support Hypothesis #1 and Hypothesis #2; Baldrige Award winning firms experienced better financial performance than non-baldrige Award winning firms, and state quality award winning firms experienced better financial performance than non-winning firms Financial performance more than 5 years after winning the award To determine whether TQM companies had better financial performance beyond the 5-year mark, we looked at 1997 data and calculated a MANOVA for each of the three sets of financial performance measures described above, once again comparing TQM firms to non-tqm firms. For the Baldrige winners, the MANOVAs for all three sets of performance measures were statistically significant, with 8.0, 22.0, and 43.3% of the variance in the financial measures accounted for by winning the Baldrige Award, respectively. For the state quality award winners, the MANOVAs were also significant for all three sets of performance measures, with variance accounted for 3.5, 4.5, and 5.0%, respectively. The only differences between the Baldrige and state quality award winner results were in the Fortune set of performance measures, where Earnings Per Share was significant only for the Baldrige winners, and Total Return over 10 Years was significant only for the state quality award winners, although the variance accounted for was consistently less for the sample of state quality award winners. In sum, these results indicate that TQM firms were still better performers than non-tqm firms even more than 5 years beyond initially winning a quality award, although the results are less strong for state quality award winners. These results also support Hypothesis #1 and Hypothesis #2; Baldrige Award winning firms experienced better financial performance than non-baldrige Award winning firms, and state quality award winning firms experienced better financial performance than other firms. The results of the MANOVAs are shown in Table 8. Table 8 Multivariate analysis of variance of Baldrige and state quality award winners and matching SIC companies 1997 data Sample Performance measures R F-ratio Probability d.f. Baldrige winners Hendricks and Singhal , 887 Fortune , 250 Kaplan and Norton , 862 State winners Hendricks and Singhal , 9206 Fortune , 2573 Kaplan and Norton , 8978 Note. The canonical correlation is reported as the multiple R. Statistically significant at 0.05 level.

13 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) Financial performance in earlier years Any attempt to capture a causal relationship between TQM and financial performance must also rule out the alternative hypothesis that TQM firms had superior financial performance, relative to their competitors, prior to winning a quality award, perhaps even for years before they adopted TQM methods. It may be that better performing companies are more likely to adopt TQM, so that rather than being a path to improved financial success (causation), TQM merely comes along for the ride (covariation). We therefore compared the financial performance of the TQM firms and non-tqm firms prior to the TQM firms winning a quality award. To test whether quality award winners had better financial performance in earlier years, even before they won the quality award, we ran MANOVAs on the three sets of financial performance measures for the 1997, 1992, 1987, and 1982 data sets. The MANOVAs were statistically significant for Baldrige winners in each of the four time periods, for the Hendricks and Singhal set, accounting for 8.0, 10.8, 9.7, and 16.1% of the variance respectively. For the Fortune set, the MANOVAs for 1997 and 1992 were statistically significant, accounting for 22.2 and 28.9% of the variance (there was too much missing data in the 1987 and 1982 data sets). For the Kaplan and Norton data sets, all of the MANOVAs were statistically significant, accounting for 43.3, 50.3, 42.9, and 47.2% of the variance. Once again, similar results were found for the state quality award winners, with lower levels of variance accounted for ranging from 5.8 to 2.6%. These results suggest that the TQM-firms did indeed have better financial performance, when compared to their peers, prior to winning a quality award. These results support Hypothesis #1 and Hypothesis #2, Baldrige Award winning firms experienced better financial performance than non-baldrige Award winning firms, both before and after winning the quality award, and state quality award winning firms experienced better financial performance than other firms, both before and after winning the quality award. The MANOVAs for the Baldrige winners are shown in Table 9 and for the state quality award winners in Table Classification analysis To confirm that the quality award winners are actually distinguishable from other companies on the basis of financial measures, a discriminant analysis was performed using whether the company won a Baldrige or state quality award as the grouping variable and financial data from 1982, 1987, 1992, and The percent of correct classifications for the Hendricks and Singhal set of performance measures using Baldrige winners was greater than 90% for all of the years that were tested. We obtained the same result for the Fortune and the Kaplan and Norton sets of performance Table 9 Multivariate analysis of variance of Baldrige winners versus other companies in 1997, 1992, 1987, and 1982 Performance measures Year R F-ratio Probability d.f. N Winners Hendricks and Singhal , , , , Fortune , , Untestable due to too much missing data 1982 Untestable, due to too much missing data Kaplan and Norton , , , , Note. Multiple correlations are canonical correlations. Statistically significant at 0.05 level.

14 304 K.M. York, C.E. Miree / Journal of Operations Management 22 (2004) Table 10 Multivariate analysis of variance of state quality award winners versus other companies for 1997, 1992, 1987, and 1982 Performance measures Year R F-ratio Probability d.f. N Winners Hendricks and Singhal , , , , Fortune , , Untestable, due to too much missing data 1982 Untestable, due to too much missing data Kaplan and Norton , , , , Note. Multiple correlations are canonical correlations. Statistically significant at 0.05 level. measures for each of the years tested (1982 could not be calculated because of missing data). Similar results were obtained for the state quality award winners, with the percentage of correct classifications for the three sets of financial performance measures greater than 85% for all of the years that were tested. Thus, the Baldrige and state quality award winners were different from other companies, for all three set of financial performance measures and for all years tested. These results also support Hypothesis #1 and Hypothesis #2: Baldrige Award winning firms experienced better financial performance than non-baldrige Award winning firms, and state quality award winning firms experienced better financial performance than other firms, both before and after winning the quality award Z-score analysis The analyses reported above either aggregated across SIC codes, comparing all quality award winning companies to the rest of the companies in the data set, or compared quality award winners to companies in their SIC code without a test for statistical significance. To compare Baldrige winning companies to the set of other companies within their SIC codes, Z-scores were calculated for each variable in the three sets of financial performance measures by SIC code, i.e., a positive Z-score for a company on any variable indicates that the company was above the mean on that variable within their SIC code. As a rule of thumb, a Z-score greater than is significantly different from the mean at the 0.05 probability level. This analysis allows an industry-specific comparison of quality award winning companies to other companies within their SIC code. With few exceptions, the Baldrige winners had positive Z-scores from 1978 to 1997 (or negative Z-scores for Cost of Goods Sold/Sales where a smaller number is better), including years before they won the award. The state quality award winning companies had better financial performance than the industry average on most of the measures for most of the 20-year range of the data set, although the effect was less pronounced. The results for the Baldrige Award winners are shown in Table 11 and for the state quality award winners in Table 12. In general, the financial performance of Baldrige winning companies was better than the industry average on almost all of the measures for the 20-year range of the data set. Many of the values are greater than +1.96, especially in later years. These results also support Hypothesis #1 and Hypothesis # Effect of industry Finally, the analyses above comparing the financial performance of quality award winners to the set of companies in its SIC code assumed there would be differences in these measures by industry. To test whether there is an effect of industry on the differences between quality award winning companies and other