SIOPRED performance in a Forecasting Blind Competition

Transcription

1 SIOPRED performance in a Forecasting Blind Competition José D. Bermúdez, José V. Segura and Enriqueta Vercher Abstract In this paper we present the results obtained by applying our automatic forecasting support system, named SIOPRED, over a data set of time series in a Forecasting Blind Competition. In order to apply our procedure it has been necessaro develop an interactive strategy for the choice of suitable seasonal pattern, exponential smoothing method and nature of the errors, which can be additive or multiplicative. In contrast, using SIOPRED all the model parameters (initial conditions and smoothing parameters) are chosen by applying Soft Computing techniques without intervention bhe forecaster in a completely automated way. For the choice of the essential characteristics of seasonality of data set and for selecting the exponential smoothing method which is used for forecasting, a multi-objective formulation which minimizes several measures of fitting is used, while keeping the updating equations of the exponential smoothing methodology. I. INTRODUCTION It is common in business and economics to find time series which can exhibit different trend and seasonal patterns. Forecasting the behavior of these time series plays an important role in several areas of decision making. The choice of a suitable forecasting procedure depends on the properties of the data or the objectives of the study ([1], [13]). Exponential smoothing (ES) methods are widely used in business planning because of their robustness and well performance in forecasting competitions, also for non-conventional data sets (see, for instance, [12], [9], [6], [7]). Using a number of modifications to the generalized Holt-Winters method we have developed a forecasting support system, SIOPRED, which permits the simultaneous consideration of a wide range of forecasting methods based on a full optimization framework ([2], [5]). Time data series arise whenever a given variable is observed over time, therefore the corresponding information about the characteristics of the variable and the observation period are well known and facilitate the analysis of the data set. This is not the situation in a forecasting blind time series competition where researchers must analyze and provide suitable forecasts in the absence of the proper specifications of systems. We propose then to forecast these time series using an interactive selection strategy based on SIOPRED, in such a wahat the fitting errors calculated for different scenarios assess both in the selection of the size of the José D. Bermúdez is with the Department of Statistics and Operational Research, University of Valencia, Spain (phone: ; fax: ; bermudez@uv.es). José V. Segura is with the Center of Operational Research, University Miguel Hernández de Elche, Spain (phone: ; jvsh@umh.es). Enriqueta Vercher is with the Department of Statistics and Operational Research, University of Valencia, Spain (phone: ; fax: ; vercher@uv.es). seasonal cycle and in the decision about the data pattern for the model fitting. This paper is organized as follows. We next describe the main features of exponential smoothing procedures and the characteristics of our forecasting support system. Section III introduces our interactive proposal for the selection of the forecasting model based on multi-objective approaches. In Section IV we show the results obtained with our forecast strategy for a set of eight time series in a forecasting blind competition. II. EXPONENTIAL SMOOTHING METHODS The exponential smoothing is a class of forecasting methods that produce point-wise forecasts with simple formulae, taking into account level, trend and seasonal components of the data, which are updated based on new observations. The introduction of a class of state-space models underlying exponential smoothing methods enabled them to enjohe advantages that forecasting procedures based on a proper statistical model have ([14], [3], [11], [8]). Let us introduce the updated equations of the Holt-Winters method, the most general form of exponential smoothing, assuming that the observed data, { },...,n have an additive damped trend and a multiplicative seasonality, in such a way that the updating equations for level, trend and seasonality are: L t = α + (1 α)(l t 1 + b t 1 ) I t p b t = β(l t L t 1 ) + (φ β)b t 1 I t = γ + (1 γ)i t p L t (1) where L t, b t, and I t in (1) represent the level of the series, the trend and the seasonal index at time t, respectively, α, β, γ and φ are their smoothing parameters, and p is the length of the seasonal cycle. Smoothing parameters are usually constrained to the range 0 1, and for the damping parameter φ the constraint β φ is added. The h-step-ahead forecast at time n, for h 1, is then given by: ŷ n+h = (L n + φ i 1 b n )I n+h p (2) i=1 Notice that we are using a damped formulation (due to the inclusion of φ parameter), although there is no dampening effect for the first forecast period. When the selected forecast method takes φ = 1 the standard Holt-Winters procedure appears. The generalized Holt-Winters additive seasonal form works changing the ratios by differences (for instance,

2 I t p instead of equations are: I t p ), in such a wahat the updating L t = α( I t p ) + (1 α)(l t 1 + b t 1 ) b t = β(l t L t 1 ) + (φ β)b t 1 I t = γ( L t ) + (1 γ)i t p (3) and the forecasts made at time n for h periods ahead is obtained from the expression: ŷ n+h = L n + φ i 1 b n + I n+h p (4) i=1 It is well-known that both additive and multiplicative Holt- Winters form are over parameterized, so their parameters are unidentifiable. In order to obtain identifiability, Winters [19] recommended normalizing the seasonal indices at the beginning of the series, in such a wahat j=0,...,1 p I j = p for the multiplicative form of the Holt-Winters method while j=0,...,1 p I j = 0 for the additive one. In order to use the exponential smoothing scheme both smoothing parameters (α, β, γ, φ) and initial values (L 0, b 0, I 1 p,..., I 0 ) must be provided, but these values are unknown and have to be estimated from the observed data. There is a wide variety of heuristic criteria for calculating these values ([15], [16]). Alternatively, our decision support system, SIOPRED, considers both the smoothing parameters and initial values as decision variables in a nonlinear optimization problem where the one-step-ahead fitting errors {e t },...,n are minimized, where e t = F t 1 and F t 1 is the one-step-ahead forecast made at time t 1. Using a multi-objective soft computing approach for three different measures of error (MAPE, RMSE and MAD) and a crisp constraint on the Theil s U-statistic value, it determines the solution with the maximum degree of global satisfaction. These measures of error are defined as: The root of the mean squared error (RMSE) is given by: RMSE(e t ) = 1 e 2 t n The mean absolute percentage error (MAPE) is computed as follows: MAP E(e t ) = 100 n e t The mean absolute deviation (MAD) is as follows: MAD(e t ) = 1 n e t Finally, Theil s U-statistics measure the effort of determining the parameter values with respect to the naive forecast, that is using the previous observed value as the forecast for the next period [18]. n U(e t ) = t=2 ( et ) 2 n t=2 ( yt 1 yt ) 2 The use of fuzzy methodology for managing those several sources of fitting errors has been very fruitful and it favours the automation of the forecasting procedure (see [2] for details). III. INTERACTIVE MODEL SELECTION STRATEGY In order to apply SIOPRED in a completely automated wahe forecaster must provide the length of the seasonal cycle and decide if the seasonal component has either a multiplicative or an additive from, the procedure then returns suitable forecasts for the specified time horizon, h, by using (2) or (4), respectively. The selection of p, the length of the seasonal cycle, is usually based both on the knowledge the users have about the time series and on the nature of the data set. However, if this value has to be obtained from the observed data we can apply our procedure over a range of possible values of p and select those with minimum fitting error trying to avoid the over parametrization of the model. Something similar happens with the decision about applying additive or multiplicative seasonality models. Let us briefly describe the ideas in which are based our model selection strategy. Our goal is identifying the most appropriate exponential smoothing method from a set of possible choices once the length of the seasonal cycle is given. For each one of these models we will use the output information provided by SIOPRED about the measures of goodness of fit RMSE, MAPE and MAD and the U-Theil statistic defined above. Figure 1 gives a sketch of the interactive method which has been implemented in order to facilitate the use of our forecasting support system in a blind competition; that is, when some specifications of data set are unknown. II. Forecast Data analysis Data SIOPRED Data Ia. Selection of seasonal cycle: p Measures of fitting Data errors Forecasting: Data SIOPRED Fig. 1. Ib. Model selection: ES method Flowchart of the selection strategy

3 Let us comment the main features of the procedure: Stage Ia. Selection of the seasonal cycle In order to properly select the size of the seasonal cycle for each time series data { },...,n, we analyze the solutions provided bhe multiplicative Holt-Winters method with several usual sizes. Here we used p {1, 3, 4, 7, 12, 24, 52, 365}, whenever n 2p, in such a wahat the most usual single seasonal patterns are considered. For each run SIOPRED returns the three measures of goodness of fit based on the one-step-ahead errors and the U-statistic value. We reject those p values whose associated U- statistics are 1 or take a value near 1. Doing that allows us to drastically reduce the number of candidate values for p (see Table I). Stage Ib. Model selection For the selected seasonal cycles in Stage Ia we again apply SIOPRED, now using the additive Holt-Winters method both for the raw data and for a logarithmic transformation of the data set. Note that the multiplicative one has been analyzed in the previous stage. We obtain a set of solutions for which the objective vector (RMSE, MAPE, MAD) values are known. Every solution corresponds to the best exponential smoothing method for the given specification of the system (see Table II). We select a non-dominated solution by ranking the obtained solutions using their objective vector values (in a standard multi-objective framework) and applhe corresponding ES method for forecasting the future values of the time series in the Stage II. Stage II. Forecasting Once an exponential smoothing method has been selected for a given time series in the previous Stage, we use SIOPRED in order to provide the h future forecasts. In fact, for the input we only need to fix p, h, the additive or the multiplicative version of the Holt-Winter method and the set of observations with which we are working with, that is raw or logarithmic transformed data. Although our procedure may perform both additive or multiplicative forms of exponential smoothing methods, in Stage Ia we only work with the multiplicative one because on average it seems to work a little better. On the other hand, it is a common approach for dealing with time series with multiplicative seasonality (when seasonal variation is roughly proportional to the local mean level) to consider a logarithmic transformation of raw data to make seasonality additive. Then, we also consider this transformation in order to decide which method is the proper exponential smoothing method. IV. FORECASTING RESULTS Here we present the results obtained by applying the scheme proposed in Fig. 1 and comment some important issues. First, in order to explain with an example the interactive model selection strateghat we introduce in the previous section, we will develop in detail the analysis of the first data series. Table I shows the results for the runs of SIOPRED for different p values in Stage Ia for the ICTSF-001 time series. which has been plotted in Fig. 2. At this stage the procedure selects p = 7. Note that the Theil s U-statistic takes a similar value for p = 365 but this last solution is a dominated one if we consider the vector of fitting errors. In the case that both solutions were no dominated with similar values for the objective vector the procedure also should select p = 7, because of the parsimony principle that looks for the simplest model, avoiding over parameterized models (in this case: 365 seasonal indices). Fig. 2. Time plot of the time series ICTSF-001 for which n = 893 TABLE I FITTING ERRORS AND U-STATISTICS FOR THE ICTSF-001 TIME SERIES Cycle size RMSE MAPE MAD U-Theil Table II shows the one-step ahead fitting errors of the solution provided bhe Holt-Winters method for the time series ICTSF-001 assuming that the data have weekly seasonality, p = 7. Concerning the column named Method we have written M ultiplicative when the exponential smoothing multiplicative seasonality form has been selected by our procedure and analogously we write Additive for the additive seasonality form.

4 TABLE II FITTING ERRORS OF THE SOLUTIONS PROVIDED BY SIOPRED FOR ICTSF-001 TIME SERIES WITH p = 7 Method Data RMSE MAPE MAD U-Theil Multiplicative raw Additive raw Additive ln(raw) In order to select exponential smoothing method that will provide the predictions for a time horizon of h = 150 periods, we rank the solutions showed in Table II. We consider a scalar function, ϕ, which adds the four fitting errors, previously scalarized bheir maximum observed value in Stage Ia, and select those method with the minimum value of ϕ. In this case, the multiplicative seasonality form has been selected and the corresponding forecasts are plotted in Fig. 3, where the seasonal cycle of length seven is clearly observed ICTSF 001 allow to clearly observe the length of their seasonal cycles, the same situation may also appear for the time plot of their point-wise forecasts, where the data pattern is sometimes hidden. This fact may be due to the relationship between the values of n, p and h (see Table IV). TABLE IV NUMBER OF OBSERVATIONS, LENGTH OF THE SEASONAL SIZE AND FORECASTING HORIZON Time series n p h ICTSF ICTSF ICTSF ICTSF ICTSF ICTSF ICTSF In contrast with Fig. 3, the monthly seasonal cycle adjusted for the ICTSF-003 time series can not be observed in Fig. 4, while it seems clearly fixed for the weekly forecasts of the ICTSF-004 time series Fig. 3. Point-wise forecasts for the time series ICTSF-001, h = ICTSF ICTSF We have applied the above selection strategy for all the time series in the competition except for the ICTSF-005 time series. Table III presents the fitting results obtained with the finally selected method for all of the time series but ICTSF We have analyzed this last series in a different manner because of our assumption about the presence of missing data. TABLE III CYCLE SIZE, EXPONENTIAL SMOOTHING METHOD AND FITTING ERRORS Time series Method Data p RMSE MAPE MAD ICTSF-001 Mult. raw ICTSF-002 Addit. raw ICTSF-003 Addit. raw ICTSF-004 Mult. raw ICTSF-006 Mult. raw ICTSF-007 Addit. ln(raw) ICTSF-008 Addit. raw It must be note that the time plot of these series do not Fig. 4. Point-wise forecasts using SIOPRED. Up, time series ICTSF-003 for h = 7. Down, time series ICTSF-004 for h = 25 Concerning the data analysis of the ICTSF-005 time series (Fig. 5), it must be noted that the selection of the length of the seasonal cycle by using all the proposed values in the Stage Ia for the seasonal size were not satisfactory. Then we tro apply SIOPRED for a exponential smoothing method with double seasonality (see, for instance, [17], [4], [10]). In particular, assuming an scenario similar to 24 hourly data observations per seven days, that is p = 168, but this adjust neither works well. Then we decide to intervene the time series assuming that p is around 168 and that the data have not been observed in a perfect regular time pattern, in such a wahat the presence of both missing and extra data in the observed seasonal cycles for this time series could be possible. In order to suitable adjust the peaks of each cycle we have trimmed the first observation and applhe Stage Ia for

5 series. This procedure selects the best exponential smoothing method in an automated way, using an optimization-based scheme which unifies the stages of estimation of parameters and model selection. Our proposal also uses a multi-objective modeling approach which jointly minimizes three measures of the onestep ahead fitting errors once the best fit for a given time series has been provided for raw or transformed data. REFERENCES Fig. 5. Time plot of the time series ICTSF-005 for n = 1031 p {165, 166, 167, 168}, then p = 165 is selected and Stage Ib is run. See Table V for the results obtained at Stage Ib. TABLE V FITTING ERRORS OF THE SOLUTIONS PROVIDED BY SIOPRED FOR THE ICTSF-005 TIME SERIES WITH p = 165 Method Data RMSE MAPE MAD U-Theil Multiplicative raw Additive raw Additive ln(raw) ICTSF Fig. 6. Point-wise forecasts for the time series ICTSF-005 for h = 200 Fig. 6 plots the forecasts for the ICTSF-005 time series with a time horizon h = 200, we have used the Holt-Winters method with multiplicative seasonality for p = 165. Note that the forecast pattern seems to preserve both the weekly and hourly seasonality of the raw data set. [1] J.R.T. Arnold, S.N. Chapman, and L.M. Clive, Introduction to Materials Management, 6th edn., New Jersey: Pearson Prentice Hall, [2] J.D. Bermúdez, J.V. Segura JV, and E. Vercher, A decision support system methodology for forecasting of time series based on Soft Computing, Computational Statistics & Data Analysis, vol. 51, pp , [3] J.D. Bermúdez, J.V. Segura JV, and E. Vercher, Holt-Winters forecasting: an alternative formulation applied to UK air passenger data, Journal of Applied Statistics, vol. 34, pp , [4] J.D. Bermúdez, J.V. Segura JV, and E. Vercher, Predicción del consumo de agua con SIOPRED, II Congreso Español de Informática, SICO 2007/CEDI 2007, Zaragoza, Spain, 2007, pp [5] J.D. Bermúdez, J.V. Segura JV, and E. Vercher, SIOPRED: a prediction and optimisation integrated system for demand, TOP, vol. 16, pp , [6] J.D. Bermúdez, J.V. Segura JV, and E. Vercher, Análisis de series de datos no convencionales con SIOPRED, 1st International Workshop on Mining of Non-Conventional Data, Sevilla, Spain, November 2009, pp [7] J.D. Bermúdez, J.V. Segura JV, and E. Vercher, Bayesian forecasting with the Holt-Winters model, Journal of Operational Research Society, vol. 61, pp , [8] A. Corberán-Vallet, J.D. Bermúdez, J.V. Segura JV, and E. Vercher, A Forecasting Support System based on Exponential Smoothing, in Handbook on Decision Making (Chapter 8), Edited by L. C. Jain and C. P. Lim, Berlin: Springer-Verlag, [9] E.S. Jr. Gardner, Exponential smoothing: the state of the art-part II, International Journal of Forecasting, vol. 22, pp , [10] P. Gould, A.B. Koehler, F. Vahid-Araghi, R.D. Snyder, J.K. Ord, and R.J. Hyndman, Forecasting time series with multiple seasonal patterns, European Journal of Operations Research, vol. 191, pp , [11] R.J. Hyndman, A.B. Koehler, J.K. Ord, and R.D. Snyder, Forecasting with Exponential Smoothing. The State Space Approach, Berlin: Springer, [12] S. Makridakis, and M. Hibon, The M3-competition: results, conclusions and implications, International Journal of Forecasting, vol. 16, pp , [13] D.C. Montgomery, C.L. Jenkins, and M. Kulahci, Introduction to Time Series Analysis and Forecasting, New jersey: Wiley, [14] J.K. Ord, A.B. Koehler, and R.D. Snyder, Estimation and prediction for a class of dynamic nonlinear statistical models, Journal of American Statistical Asssociation, vol. 92, pp , [15] C. T. Ragsdale, Spreadsheet Modeling and Decision Analysis, 3rd. Ed., Cincinnati, Ohio: South-Western, [16] J.V. Segura, and E. Vercher, A spreadsheet modeling approach to the Holt-Winters optimal forecasting, European Journal of Operational Research, vol. 178, pp , [17] J.W. Taylor, Short-term electricity demand forecasting using double seasonal exponential smoothing, Journal of the Operational Research Society, vol. 54. pp , [18] H. Theil, Applied Economic Forecasting, Amsterdam: North Holland, [19] P.R. Winters, Forecasting sales by exponentially weighted moving averages, Management Science, vol. 6, pp , V. CONCLUSIONS This paper presents an interactive selection strategy for forecasting time series based on SIOPRED assuming blind scenarios with respect some proper specifications of the time