COORDINATING DEMAND FORECASTING AND OPERATIONAL DECISION-MAKING WITH ASYMMETRIC COSTS: THE TREND CASE

Transcription

1 COORDINATING DEMAND FORECASTING AND OPERATIONAL DECISION-MAKING WITH ASYMMETRIC COSTS: THE TREND CASE ABSTRACT Robert M. Saltzman, San Francisco State University This article presents two methods for coordinating demand forecasting and operational decisionmaking when time series data display a trend and the unit costs of under-forecasting (c u ) and over-forecasting (c o ) may differ. The first method, called ordinary least squares with adjustment (), makes a simple additive adjustment to the forecasts generated by regression. The second method generates forecasts by fitting a weighted least absolute value () regression line through the data, where the weights correspond to the unit underage and overage costs. Using simulation, both methods are tested against regression forecasts made in the presence of one of four types of error distributions, and numerous combinations of c u and c o. The performance measure used for evaluation is the mean opportunity loss (MOL) over the postsample periods. Simulation results show that the benefits of both approaches increase rapidly as c u and c o diverge, regardless of the error distribution. In particular, when the ratio of costs is 3:1, can reduce post-sample MOL of by 19-23%. Even when the ratio of costs is closer to 2:1, can still reduce post-sample MOL by 6-7%. In a few scenarios, outperforms, but generally is not as robust as. I. INTRODUCTION Traditionally, articles and texts concerned with demand forecasting tend to ignore the related decision-making aspects of the situation. Similarly, those concerned with operational decisionmaking tend to ignore how the relevant demand forecasts were developed. Many assume that a manager somehow already possesses a demand forecast, or that demand can be represented by some specific probability distribution (Brown, 1982). In reality, demand forecasting and decisionmaking should not be activities performed in isolation from one another. A few articles bridge this gap and show how these two activities interact, especially in the context of inventory control. For example, Gardner (1990) demonstrated how the choice of forecasting model plays a key role in determining the amount of investment needed to support a desired customer service level in a large physical distribution system. Eppen and Martin (1988) showed how various stationary demand distributions and lead time length affect forecasting error and, thus, error in the proper amount of safety stock required to meet a given service level. Watson (1987) studied how lumpy demand patterns can lead to large fluctuations in forecasts, which, in turn, increase ordering and holding costs as well as distort the actual level of service provided. Importantly, positive and negative forecasting errors often have asymmetric impacts on operations and, therefore, may be viewed differently by the manager who actually uses the forecasts to make a decision. Symmetric performance measures (e.g., MAD, MSE, MAPE) may be misleading if the cost of over-forecasting is substantially different from that of underforecasting. Consequently, the methods presented here are evaluated from the viewpoint of a decision-maker who adopts mean opportunity loss (MOL) as the best measure of forecasting performance. Unfortunately, articles and texts that use symmetric measures of forecasting performance and do not consider the role played by forecasts within the larger decision-making context dominate the forecasting literature. One exception is the situation described by Lee and Adam (1986), who report that some MRP systems can reduce total production costs if demand for final product is over-forecast by 10-30%. Conversely, class schedulers at a large university (SFSU) prefer to under-forecast enrollment in multi-section courses, finding it easier to add a section at the last minute if demand turns out to be higher than expected, rather than cancel an under-filled section if demand is lower than expected. Another common situation in which the costs of overforecasting (c o ) and under-forecasting (c u ) may be quite different is in the classical single-period California Journal of Operations Management Volume I, Number 1,

2 inventory problem. Gerchak and Mossman (1992, p. 805) claim that c u is usually much greater than c o. In all of these settings, planners could estimate the relative costs of over- and underforecasting and adjust the given forecasts in a manner that appropriately balances the costs. Rech, Saltzman and Mehrotra (2002) present an easy-to-use, general procedure for doing this that still allows the forecaster/decision-maker to first uncover the underlying pattern in the time series data. The aim of this article is to extend that integrative approach by demonstrating how demand forecasting and decision-making can be effectively coordinated when the time series of interest displays a linear trend. II. MODEL AND FORECASTING METHODS This study used the linear model y t = α + βt + ε t, for t = 1, 2, n, where y t is the t th observation of the time series and ε t is a random error for the t th observation. Four different ε t distributions, described in the next section, were tested. The parameters α and β were estimated (by a and b, respectively) by each of the three forecasting methods, i.e., (1) : Ordinary Least Squares regression; (2) : Ordinary Least Squares regression with Adjustment a method first described in Rech, Saltzman and Mehrotra (2002) that makes a simple additive adjustment to the forecasts; and (3) : Weighted Least Absolute Value regression a new approach that finds the line that minimizes the total opportunity loss. Of course, minimizes the sum of the squared residuals Σ(y t f t ) 2, where f t = a + bt. For example, the regression equation for the time series data (with n = 18) listed in the first two columns of Table 1 is f t = t, graphed as the long-dashed line in Figure 1. In the simulation experiments, served as a benchmark for comparison with the other two methods. makes a simple additive adjustment to the forecasts that takes into account the weights placed on positive and negative errors (c u and c o, respectively) by the decisionmaker. Given any set of forecasts and the associated errors, Rech, Saltzman and Mehrotra (2002) show how to easily find the adjustment a* that, when added back into the original forecasts, would minimize the historical MOL. This adjustment a* is, in essence, just the S th percentile of the distribution of forecast errors, where S = c u /(c o + c u ) is called the critical ratio. TABLE 1: EXAMPLE OF FORECASTING METHODS & THEIR OPPORTUNITY LOSSES Forecasting Method Opportunity Loss ($) t Actual MOL California Journal of Operations Management Volume I, Number 1,

3 FIGURE 1: PLOT OF THE DATA AND FORECASTS WHEN S = Demand Actual Period t To illustrate, let c u = $65 and c o = $35, so that S = Then, based on the set of errors generated by using the regression equation to forecast the 18 historical periods, a* is set to 5.653, the value of the 12 th smallest error (i.e., the 65 th percentile of the error distribution). The resulting equation is f t = t + a* = t, shown as the solid line in Figure 1. Because under-forecasting is nearly twice as costly as over-forecasting, the forecasts are considerably higher than the forecasts and thereby reduce the historical MOL of by 12.2% (from $ to $282.13). extends the Least Absolute Value criterion that others have investigated (e.g., Dielman and Rose 1994, Dielman 1986, and Rech, Scmidbauer and Eng 1989) to allow positive and negative errors to have different costs. Its objective is to find the line that minimizes the sum of weighted absolute errors, that is, c ( y f ) + c ( f y ), where e t = y t f t. Thus, u t t o et > 0 e t < 0 minimizes both the total and mean opportunity loss over the historical periods. To solve the problem, we reformulate it as the following linear programming problem. Minimize c u Σ P t + c o Σ N t subject to P t N t + a + bt = y t, for t = 1, 2,, n P t 0, N t 0, for t = 1, 2,, n The variables P t and N t represent the magnitude of positive and negative errors, respectively, i.e., the vertical distances from the actual data point (t, y t ) down and up to the fitted line. The difference P t N t defines the residual for period t. With appropriate costs in the objective, the problem combines the forecasting and decision-making activities into a single problem and gives the decision-maker the flexibility to adapt to any situation with asymmetric costs. Applying this approach to the data given in Table 1 results in a equation of f t = t, shown as the short-dashed line in Figure 1. As with, the forecasts are also well above those generated by. Moreover, they reduce the historical MOL to $280.70, a 12.7% reduction over that of and a slight improvement over. t t III. EXPERIMENTAL DESIGN To test these methods on a wide variety of sample data, and to assess their performance on future or post-sample data, a Monte Carlo simulation experiment was conducted for the model presented in the previous section. The experimental design included two factors: the error distribution and the critical ratio. The following four error distributions, each with a mean of zero, California Journal of Operations Management Volume I, Number 1,

4 were tested in the experiments: (a) Normal, with mean µ = 0, standard deviation σ = 5, or N(0, 5); (b) Laplace (or double exponential), with each exponential having σ = 5; (c) Contaminated Normal, where the ε t are drawn from a N(0, 5) distribution with probability 0.85 and from a N(0, 25) distribution with probability 0.15; and (d) Uniform, with floor -15 and ceiling +15, or U(-15, 15). A Normal error distribution is typically assumed to hold for applications; the latter three distributions permit investigation of how sensitive the results are to increasing weight in the tails of the error distribution. The critical ratio S = c u /(c o + c u ) expresses the cost of under-forecasting relative to that of over-forecasting. For example, a c u :c o ratio of 3:2 is represented by S = In order to determine how the results change over a wide range of values that S might take on in practice (i.e., from 0.10 to 0.90), the value of c u was varied from $10 to $90, in increments of $5, while setting c o = $100 c u. Results in the next section are presented as a function of S. All of the experiments were run on a personal computer in Microsoft Excel using Visual Basic for Applications (Harris, 1996). Each experiment used the same parameter values of α = 0 and β = 5, as well as the same sample size of n = 24 historical periods, and forecasting horizon of h = 6 post-sample periods. This relatively small amount of data is typical of business situations in which a time series may exhibit a trend, e.g., 2 years of monthly data (projecting up to half a year ahead) or 6 years of quarterly data (projecting up to a year and a half ahead). A scenario here refers to a particular combination of critical ratio and error distribution. Within each replication of a scenario, the traditional design for testing forecasting methods on a time series (Makridakis, et al., 1982) was followed. First, a time series of the desired length (n + h = 30 periods) was randomly generated using one of the four error distributions. The first n periods were then used to estimate (fit) the parameters of each forecasting method, while the last h periods were used by each method to generate a set of post-sample forecasts. Successively applying each forecasting method to the data yielded both a set of n historical forecasts and a set of h post-sample forecasts. Performance was then measured separately for each set of forecasts, i.e., the MOL across the n historical periods was recorded, as was the MOL over the h post-sample periods. Finally, an average MOL over 1000 replications was calculated, and reported upon next. IV. SIMULATION RESULTS A few general observations can be made before examining the results in detail. First, in every scenario tested, always achieves the lowest historical MOL. This is not surprising because it's objective is precisely to find the line that minimizes the MOL over the historical periods., on the other hand, is more constrained than in that it takes the forecasts from the line as fixed and then adjusts only the intercept of this line by adding a* to all the historical forecasts. Regardless of the error distribution, significantly lowers the historical MOL of. However, does nearly as well as in reducing the historical MOL, (only about 2% higher). Since both and reduce the historical MOL of in all scenarios tested, the historical MOL case won t be discussed further. Second, the relative performance of and over the historical periods tended to be reversed over the post-sample periods in most scenarios. The simulation showed that, for the scenarios with Normal and Uniform errors, always outperforms ; with Laplace and Contaminated Normal errors, outperforms for some values of S. While unexpected, this reversal in performance between sample and post-sample periods is not uncommon. Makridakis (1990, p. 505) references a number of empirical studies in which the method that best fit the historical data did not necessarily provide the most accurate post-sample forecasts. Third, for each error distribution tested, the relative benefits of using either or over are roughly symmetric about S = 0.50 (as can be seen in Figures 2-5), i.e., the percentage reductions in MOL at S = 0.25 and S = 0.75 are quite similar to one another, as are those at S = 0.35 and S = Therefore, statements made below for values of S a certain distance above 0.50 are also approximately true for values of S the same distance below California Journal of Operations Management Volume I, Number 1,

5 Normal errors. Two important differences from the historical data case emerge in the postsample (Figure 2). First, outperforms for all values of S, and second, there are some values of S for which neither nor outperforms. In particular, improves upon when either S 0.60 or S Otherwise, yields a slightly higher MOL than. Meanwhile, reduces the MOL of only when S 0.65 or S Still, both methods improve on as the costs diverge from one another. For instance, at S = 0.65 (where c u is almost twice c o ), reduces the MOL by 5.8% (from $ to $214.14), while reduces it by 2.1% (to $222.63). At S = 0.75 (where c u is three times c o ), reduces the MOL by 20.1% (from $ to $181.30), while reduces it by 13.5% (to $196.07). FIGURE 2: POST-SAMPLE MOL VS. S WITH NORMAL ERRORS FIGURE 3: POST-SAMPLE MOL VS. S WITH LAPLACE ERRORS Mean Opportunity Loss ($) Critical Ratio S Mean Opportunity Loss ($) Critical Ratio S FIGURE 4: POST-SAMPLE MOL VS. S WITH CONTAM. NORMAL ERRORS FIGURE 5: POST-SAMPLE MOL VS. S WITH UNIFORM ERRORS Mean Opportunity Loss ($) Critical Ratio S Mean Opportunity Loss ($) Critical Ratio S California Journal of Operations Management Volume I, Number 1,

6 Laplace errors. Figure 3 shows that and both improve upon for all S values. However, for 0.40 S 0.60, does slightly better than, while for S 0.30 and S 0.70, does increasingly better than. As before, both methods rapidly improve on as the costs diverge from one another. For instance, at S = 0.65, reduces the MOL of by almost 6%, while reduces it by 5%. At S = 0.75, reduces the MOL by 17%, while reduces it by just under 13%. Contaminated Normal errors. Figure 4 shares two key features with Figure 3. First, both and improve upon for every value of S, and second, is superior to for some values of S ( ). In many cases, is quite a bit better than, e.g., at S = 0.45, where it reduces the MOL by 8.3% (from $ to $352.49) while only reduces the MOL by 1.4% (to $379.08). Uniform errors. Figure 5 shows consistently outperforming, and by larger amounts than it did in the Normal error case. appears to do slightly worse than when S is However, at S = 0.65, reduces the MOL by 6.2% (from $ to $385.32), while actually increases it by 2% (to $419.12). At S = 0.75, reduces the MOL by nearly 23% (from $ to $315.70), while reduces it by 15%. As S approaches 0.50, performs much poorer here than it did in the other cases. In particular, s postsample MOL is 2 12% higher than that of for values of S in Clearly, one would not use in this case unless S was at least 0.70 or at most TABLE 2: SUMMARY OF POST-SAMPLE MOL RESULTS FOR SELECTED VALUES OF S S Forecasting Method Error Distribution Contaminated Normal Laplace Normal Uniform $ $ $ $ $ $ $ $ % 16.6% 16.1% 22.0% $ $ $ $ % 13.3% 18.6% 15.9% $ $ $ $ $ $ $ $ % 7.1% 6.5% 6.7% $ $ $ $ % 7.2% 11.4% 1.2% $ $ $ $ $ $ $ $ % 5.8% 8.0% 6.2% $ $ $ $ % 5.0% 10.1% -2.0% $ $ $ $ $ $ $ $ % 17.0% 19.1% 22.9% $ $ $ $ % 12.9% 18.7% 15.3% California Journal of Operations Management Volume I, Number 1,

7 Table 2 summarizes the post-sample MOL results for all of the tested error distributions for four selected values of S. The percentages below the MOL figures for and represent the relative changes from the MOL with. V. CONCLUSION This article has pursued two main themes. First, demand forecasting and operational decisionmaking can be easily coordinated in some circumstances. Second, when positive and negative forecasting errors have significantly different costs for the decision-maker, it's usually well worthwhile to adjust the forecasts to account for the difference. This article has shown that the straightforward adjustment approach described in Rech, Saltzman and Mehrotra (2002) works well in the trend case across a variety of error distributions and critical ratios. It appears here that the best forecasting method depends on the type of data being analyzed, as well as the relative costs of under- and over-forecasting. In particular, if the pattern of errors about the trend line is well approximated by either a Normal or Uniform distribution, and the critical ratio S is not too close to 0.50, using provides a consistently greater reduction in MOL than does. Specifically, when the ratio of costs is 3:1 (i.e., S = 0.75 or 0.25), reduces post-sample MOL by 19 23%. Even when the ratio of costs is closer to 2:1 (i.e., S = 0.65 or 0.35), still reduces post-sample MOL by 6 7%. If errors appear to follow either a Normal or Uniform error distribution, and S is in the range, it's probably not worthwhile to use either or. When the errors fit either a contaminated Normal or Laplace distribution, both and improve upon the post-sample MOL of for all values of S. However, tends to outperform for values of S close to 0.50, while does better for values of S closer to 0 or 1. In general, the benefits of using either or over increase dramatically as the costs become more asymmetric, regardless of the error distribution. Given the uncertainty in actually knowing the underlying error distribution, and that 's performance is more robust than 's with respect to the error distribution, I d recommend that always be used, especially if S is outside of the range. is much easier to implement than and outperforms under most scenarios. While has the pleasing property of finding the forecasting line and accounting for asymmetric costs simultaneously, it requires setting up and solving a linear program. Moreover, it significantly improves upon only when the errors are contaminated Normal and 0.30 S This article has looked at long-term forecasting methods (i.e., regression) for the linear case. An area for further research would be to investigate the effect of adjusting short-term, adaptive forecasting methods in the trend case. For example, using exponential smoothing or moving average forecasts with some type of moving adjustment may well provide benefits similar to those described here relative to more sophisticated approaches such as Holt's method. VI. REFERENCES Dielman, T., A Comparison of Forecasts from Least Absolute Value and Least Squares Regression, Journal of Forecasting, Vol. 5, 1986, Dielman, T. and Rose, E., Forecasting in Least Absolute Value Regression with Autocorrelated Errors: A Small-Sample Study, International Journal of Forecasting, Vol. 10, 1994, Eppen, G. and Martin, R., Determining Safety Stock in the Presence of Stochastic Lead Time and Demand, Management Science, Vol. 34, 1988, Gardner, E., Evaluating Forecast Performance in an Inventory Control System, Management Science, Vol. 36 (4), 1990, Gerchak, Y. and Mossman, D., On the Effect of Demand Randomness on Inventories and Costs, Operations Research, Vol. 40 (4), 1992, California Journal of Operations Management Volume I, Number 1,

8 Harris, M., Teach Yourself Excel Programming with Visual Basic for Applications in 21 Days, SAMS Publishing, Indianapolis, Lee, T. and Adam, E., Forecasting Error Evaluation in Material Requirements Planning Production-Inventory Systems, Management Science, Vol. 32 (9), 1986, Makridakis, S. et al., The Accuracy of Extrapolation (Time Series) Methods: Results of a Forecasting Competition, Journal of Forecasting, Vol. 1, 1982, Makridakis, S., Sliding Simulation: A New Approach to Time Series Forecasting, Management Science, Vol. 36 (4), 1990, Rech, P., Saltzman, R. and Mehrotra, V., Coordinating Demand Forecasting and Operational Decision Making: Results from a Monte Carlo Study and a Call Center, Proceedings of the 14 th Annual CSU-POM Conference - San Jose, CA, February 2002, Rech, P., Schmidbauer, P. and Eng, J., Least Absolute Regression Revisited: A Simple Labeling Method for Finding a LAR Line, Communications in Statistics - Simulation, Vol. 18 (3), 1989, Watson, R., The Effects of Demand-Forecast Fluctuations on Customer Service Level and Inventory Cost when Demand is Lumpy, Journal of the Operational Research Society, Vol. 38 (1), 1987, California Journal of Operations Management Volume I, Number 1,