Quality and Uncertainty in Seasonal Adjustment Draft 1

Transcription

1 1(29) Quality and Uncertainty in Seasonal Adjustment Draft 1 Gross domestic product (GDP) Per cent change from previous quarter, annual rate. Constant prices. Seasonally adjusted TRAMO/SEATS ARIMA(110)(010) X12 ARIMA(011)(011) Source: Statistics Sw eden Data up to and including the Second Quarter 2006 Abstract The idea of a best method in seasonal adjustment has since long time been a framework of research and comparisons of software. It has produced recommendations of software providing a route for harmonization of methods among countries. It seems that the idea of best method and harmonization would guarantee comparability among time series between nations. In this paper, model uncertainty and quality in seasonal adjustment is discussed within the framework of model based approaches. The estimated unobserved components for a specific model for a seasonal time series are just one amongst many competing models within a space of models. So are the seasonal adjusted series and their changes over time. For every model there is a vector of observed properties, statistical properties and user defined properties. Examples of statistical properties are the AIC, BIC-measures and the estimated test statistics of the residuals from the model. Examples of user defined properties are the revision errors, smoothness, outliers, trading-day effects and distance measures between the changes of the seasonal adjusted series and the yearly changes of the original series. An outline of this approach is given. Looking at the 'whole' space is called the 'superpopulation approach' to seasonal adjustment. Quality components of official statistics with special reference to seasonal adjustment are discussed.

2 2(29) 1. Introduction and Summary Seasonal Adjustment at Statistics Sweden The Quality Concept of Official Statistics in Sweden The Characteristics of a Time Series Uncertainty in Seasonal Adjustment A Superpopulation Approach to Seasonal Adjustment Some Examples of the Superpopulation Approach On a Quality Measure in Seasonal Adjustment Guidelines issued by the Council for Official Statistics in Sweden Statistical Properties of Seasonal Adjustment User related properties of Seasonal Adjustment Smoothness Revisions Outliers Calendar effects Yearly Changes of the original series Uncertainty in Seasonal Adjustment Software and IT...26 References...28 Appendix 1 Criteria for sufficient quality for official statistics Introduction and Summary The search on the words 'seasonal', 'seasonal adjustment' and 'seasonal adjustment quality' with Google results in , and 73 hits respectively. The corresponding with Google Scolar gives , 4870 and 9. Looking for the phrase 'data quality' results in hits using Google and in Google Scholar. In other words, 'seasonality' is a very widely used concept. That is also the case for the concept seasonal adjustment. and a demand for its use. However, to write about the concept of 'sesonal adjustment quality' seems to be difficult and may bee a risky affair, especially in a scientific way. Even if Google does not give the whole picture, the search results indicates the magnitude of the difficulties about the quality concept of seasonal adjustment. One of the reasons seems to be the non-existence of observable true components, like the population mean. All components discussed in the seasonal adjustment framework are un-observable components similar to the components in

3 3(29) factor analysis. However, the search for 'factor analysis quality' in Google gives 836 hits, ten times more than the phrase 'seasonal adjustment quality'. So, there must be something more. Statistics Sweden (SCB) has a long tradition of being in the frontline of quality discussions of data at the national and an international level. When the report on quality of data 1 was published at SCB, I asked one of the authors: 'Where can I find a discussion on the quality of a seasonal adjusted number?'. You are welcome to write about it. This conversation took place many years ago! Summing up: 'Seasonality affects every one', seasonal adjustment is widely used while the 'quality in seasonal adjustment' is rarely discussed. The purpose of this paper is to start a discussion of 'seasonal adjustment quality' within the context of model-based seasonal adjustment 2 and the official statements on quality of official statistics given by the Council of Official Statistics 3. The paper is organized as follows. In chapter 2, a short overview of the system of seasonal adjustment at Statistics Sweden is given, as a background. In chapter 3, the official statements on the quality of statistics are given. The characteristics of a time series are in focus in chapter 4. These characteristics are divided into two classes, characteristics stemming from nature and characteristics stemming from the producer of statistics. Uncertainty in statistics is discussed in chapter 5 within the context of model-based seasonal adjustment. The superpopulation setup for seasonal adjustment is outlined in chapter 6. In chapter 7, this view is shown in some empirical examples taken from Swedish statistics. Suggested components of a quality measure are introduced in chapter 8. It is consistent with the guidelines issued by the Council for Official Statistics in Sweden. Summary Model-based seasonal adjustment with TRAMO/SEATS was implemented at Statistics Sweden (SCB) in 1999 for the national accounts. An interface written in SAS was used with EXCEL for input/output. This production system has been in use without any technical problem. The most important feature of model-based seasonal adjustment is the choice of a statistical model for a time series. Given the estimated model, it is used for forecasts/backcasts to extend the series in order to use appropriate estimation methods for the unobserved component - the seasonal, the trend, trading-day and working-day effects and outliers. Special attention has been made at SCB for attain and maintain good quality of the system for seasonal adjustment and its output. At present about 500 time series are individually adjusted by TRAMO/SEATS covering national accounts and other economic statistics. There is a surveillance of all system in real time'. Every year, extensive analyses are made, update of the models, etc. An outline of used criteria is given in chapter 2. These analyses are made in the context of a superpopulation approach to seasonal adjustment, which is briefly indicated in chapter 6. Roughly, it means that a large set of models, a superpopulation of models, are used to bring up a whole distribution of e.g., seasonal adjusted GDP or other unobserved components. The dispersion in this distribution is seen as a measure of uncertainty in seasonal adjustment for a particular series. 'Official ' models and seasonally adjusted statistics are compared to the distribution of this statistic over the superpopulation. as a checkpoint, before publication. The maintenance of models is very time-consuming and expensive. SCB is searching for a new design of seasonal adjustment, which can be made in an automatic way and still be of high quality. The superpoulation setup in lines with chapter 6 will be a proposition for tests during Quality and uncertainty is of major concern in the paper. Quality in seasonal adjustment should be part and consistent with other quality properties as officially taken. The official statements 1 See [2] 2 That includes the X-11-ARIMA and X-12 and X-13S approaches although they are only partly model-based. 3 See [ ].

4 4(29) on this is outlined in chapter 3. Seasonal adjustment is to a limited extent part of these guidelines. Quality in seasonal adjustment consistent with the official view on quality of official statistics is discussed in chapter 8. In 8.1 the guidelines for quality are given for seasonal adjustment. The objectives should be clearly formulated, the main users be involved and a yearly quality report is of great importance. The components of a quality declaration/report are introduced and discussed in chapter The quality components are classified as statistical properties, user-related properties, uncertainty, software and IT. The objectives of seasonal adjustment are explicit formulated by proposals. 2. Seasonal Adjustment at Statistics Sweden Model-based seasonal adjustment (SA) was implemented for the National Accounts (NA) in A lot of research was made before this decision was taken. The evaluation at SCB has shown that SA with the software TRAMO/SEATS (TRS) applied to Swedish time series produces SA series with high quality. For that reason SCB has chosen to use TRS for SA of official time series. The implementation of TRS has been made for all national accounts series, for short term industrial indicators and other statistics at SCB. X-11-ARIMA in still in use but will be replaced in Although restricted resources, SCB has put much attention and effort to achieve and maintain high quality in SA within the framework of model-based SA. Special attention has been made to calendar, working-day effects, outliers and the identification of models. Every single time series has been treated individually. Many decisions have been taken to attain and maintain good quality in SA in order to make it easy for the user to analyse the Swedish economy and its development over time, the identification of turning points but also separate the effects of exogenous and random disturbances. The arguments on the choice of software at Statistics Sweden have been discussed in [4]-[13]. That does not mean that we have solved all problems of SA. One of the 'remaining' t challenges is to deal with four issues, uncertainty in seasonal adjustment, the assessment of a quality measure, maintenance or surveillance of models and to inform the user. The format of a quality report for SA and its 'organisation' is also in front of us. The TRS software has been implemented within a production system using SAS as an interface. There are many parameters to be set, i.e., choices for SA of a particular series. There are two separate routes to follow. Either we can use the default parameters or we can make such choices individually/manually. SCB has taken the last approach, so far, for all time series. That means that every series has been individually analysed in order to obtain and maintain good quality. The implementation of TRS is further discussed in this chapter. The Choice of the ARIMA-Model Model-based SA is founded in time series analyses. A very important moment is to forecast/backcast the original series in order to use symmetric filters. If these forecasts are good, the SA revisions of the series will be small and consistent with the variability and uncertainty of each series. Before a forecast can be made, a model for the original series has to be chosen and estimated. In TRS the ARIMA-models are used in a standard way. The specification of the ARIMA-model includes several moments in line with the Box-Jenkins approach. This can be done in many ways. The most important principle for the choice of the model is the likelihood-principle. The model is chosen, which gives the best explanation of the actual outcome, i.e. given the data. The BIC-measure is used, which gives a penalty for overparametrization of the model. The relevance of the BIC-measure lies on the assumption that the error distribution is normal and independent. For that reason, it is very important to continuously check that the underlying assumptions are met. This is done routinely in TRS by

5 5(29) statistical tests 4. A number of parameters have to be estimated. Good quality in SA assumes that the model is significantly estimated and stable over time 5. For all series SA in NA and for IPI, 140 different ARIMA-models have been investigated. The ultimate choice has been made from these models according to the following criteria 6 : Table Criteria used for the selection of the ARIMA-model at Statistics Sweden C1 Maximum likelihood (BIC), C2 Statistical tests of the residuals C3 Autocorrelations of the residuals C4 Graphs of the residuals C5 Significance of the parameter estimates of the ARIMA-model C6 Variability of the SA series C7 Graphs of the SA series In those cases a unique best model according to the criteria C1, C2 and C5 exists, it has been chosen. This is very unusual. Normally, the distribution of BIC among models does not discriminate between models enough for a unique choice of the best model. Among the models showing approximately the same likelihood according to BIC, a further screening take place based on the properties of the residuals and the tests available of the programs. Any significant diagnostic test rejects the model. Graphs of the residuals and also of the SA series and the trend estimates and the changes are used as complementary information eventually showing more complex patterns of the residuals, not revealed by the tests. Models with not statistically significant parameter estimates are normally rejected 7. If two or more models have the same quality rank according to the used criteria, the model showing the lowest variance of the changes of the SA series is chosen 8. We think that many users of SA series, e.g. in economic analysis, do not wholly understand the nature of seasonal adjustment. There is no true seasonal adjusted value. For a particular time there is a distribution of SA values depending on many things as software, statistical method, numerical algorithms and the crucial choice of a model for the series. In order to illustrate the uncertainty of the SA series stemming from different choices of the ARIMA-model, we show graph 2.1 below. It shows the quarterly changes raised to a yearly level of the SA values for GDP based on 50 different ARIMA-models for In the first place we can observe that the choice of the ARIMA-model is important for all times 9 not only at the end of the series. The range is about two per cent Even if we eliminate the most extreme models, there is a remaining model variability of about 1 per cent. If we look at the third quarter of 2001, we have a mean change of about 0.4 per cent with a standard deviation of 0.7 The change according to the official estimate was 0.2 per cent, i.e. in the middle of the distribution 10 of 4 Test of normal distribution, Durbin Watson test of autocorrelation and Q-test of autocorrelation of the residuals. 5 The parameters of the model for the calendar/working-day effects included. 6 At present the distance measure discussed in chapter 4 is also used. 7 Unless there is no acceptable model with significant parameters. The best model in that inferior class is chosen. 8 This consideration of low variability is of great concern for SCB because SCB has started to publish monthly/quarterly changes of SA values raised to a yearly level. Because the noise of the SA series will also be raised to a yearly level, the signal to noise-ratio must be kept at an acceptable level. For instance, a change of GDP of say 15 % at an yearly level is not acceptable. 9 The reason for this is that TRS uses the estimated parameters in the filter used in the decomposition of the series. 10 The mean over models is indicated by a circle in the graph.

6 6(29) changes 11 over all models. For most of the NA series, there has been possible to identify a proper ARIMA-model, which focus of comparisons of the SA series over time. Because of the quite short period, there is a considerable need for statistical surveillance of the specifications Fig. 2.1 SA and working-day adjusted GDP Quarterly changes in per cent at an yearly level 50 ARIMA-models % including the choice of the ARIMA-models.The production system for regular adjustments is designed for instant attention to SA difficulties through the diagnostics. For every series, a SAS-interface produces: D1. Estimated parameters of the ARIMA-model and the standard errors. D2 Statistical tests as discussed in chapter 5.1. Colours are used to call attention to potential problems and deviations from the SA assumptions (red colour). Blue is used to signal no problem and green indicates perfect. D3 Residuals D4 Autocorrelation of the residuals D5 Graphs of the residuals, D6 Outlier effects, value and date D7 Working-day effects, 136 official time series for national accounts were seasonally adjusted in 1999 in regular production. There were 272 infiles of the system, 580 files of output, a lot of things to be kept in surveillance. However, the IT-system is designed for easy maintenance and control running on a server at NA. All series are processed at the same time in batch-mode. At present there are about 500 time series at SCB adjusted by TRAMO/SEATS and about 4000 files to keep track on. A version of an EXCEL-macro for TRAMO/SEATS is also in use at SCB for official statistics. For some statistics the X-11-ARIMA program is still in use but will be replaced in For industrial production index, 140 ARIMA-models have been analysed for all industries in lines with the superpopulation approach. For every industry, the distributions of all unobserved components as well as the diagnostics are part of the production system. In this case, SAS is used to produce graphs in html-format for the seasonal adjusted series, the trend 11 That is also the case for other quarters. 12 The data used for the graph was published in 2001

7 7(29) and the monthly changes. These graphs can be shown by a browser. The IT-setup is shown in the graph 2.2 below. For every industry 13 all output is stored in an EXCEL-sheet and the graphs for all models could be seen e.g., in explorer. What we see in the graph is the classification of the industries in the left and the graphs for all ARIMA-models for manufacture. We can only se some of the html.files in the graph. Fig. 2.2 The IT-interface for the superpopulation approach Here, we have introduced the uncertainty concept stemming from the choice of a particular model. It should be one of the dimensions in a quality measure in seasonal adjustment. Of course, such a measure must be consistent with other quality statements for official statistics, which are briefly discussed in the next chapter. 3. The Quality Concept of Official Statistics in Sweden In this chapter, the official view of the quality concept at SCB is given. 14 Production of statistics is here compared with production of any good in a production process. The output is a product with certain characteristics. The quality of the product is related to the response from the user about the perceived characteristics of the product and related to the concept of 'total quality' as formulated below. (i) (ii) The user shall be in focus Quality refers to all aspects of a product which are of relevance for how well it meets users' needs and expectations. The quality concept has to do with the properties of statistics, use of statistics and the user of statistics. Some statistics may be regarded as statistics with high quality for a particular use/user but of low quality with respect of a different user. The quality concept does not, however, imply on whether the product is of good or bad quality in any absolute sense. A quality concept for a certain product is of importance in quality declarations, in quality improvement work and in the evaluation of productivity of producing statistics. The application of the total quality concept on statistics 'leads' to the following definition. (iii) Quality of statistics refers to all aspects of statistics, which are relevant for how well 13 SNI in Swedish 14 See [2] and [3]

8 8(29) they meet users' needs for information. The quality concept has the following main components, (1) Contents (2) Accuracy (3) Timeliness (4) Coherence and comparability (5) Availability and clarity The main components are broken down into sub-aspects according to table 3.1 below. The production of statistics starts with the construction of a target population of units, e.g., all firms in Sweden producing and selling goods during a specific reference time. From a frame of units, the producer selects a certain number of units, a sample. A questionnaire is designed, which measures the variables of interest, e.g., production of goods (e.g., the value of production). Based on the sample, we use statistical inference to estimate characteristics of the population (statistical measures), e.g., total value of production during the reference time. Accuracy measures how close the estimate from the sample is from the true population value. Different sources of inaccuracy according to table have negative impacts on the overall accuracy. The presentation of accuracy measures deals with how to inform the user of the different sources of inaccuracy. Statistics with high publication frequency, low production time and with punctuality is desirable. Statistics should be comparable over time and over domains of study, e.g., between different industries. Table 3.1 Quality concept for Official Statistics CONTENTS TIMELINESS Statistical target characteristics Frequency - Units and population Production time - Variables Punctuality - Statistical measures - Study domains - Reference times Comprehensiveness COMPARABILITY and COHERENCE ACCURACY Comparability over time Comparability between domains Coherence with other statistics Overall accuracy Sources of inaccuracy - Sampling - Frame coverage - Measurement - Non-response - Data processing - Model assumptions Presentation of accuracy measures AVAILABILITY AND CLARITY Dissemination forms Presentation Documentation Access to micro data Information services

9 9(29) Good quality is also coherence with other statistics, e.g., in international comparisons 15. Availability and clarity deals with how to publish the statistics, in what form, how to access published data, the documentation of the statistical methods and what extra services are supplied by the producer of statistics. Seasonal adjustment is mentioned within the concept comparability over time as follows. Procedures for seasonal adjustments facilitate comparisons, in particular for short-term statistics. The 'implementation' of procedures for certifying quality in statistics has been discussed within a work group for methods and quality issues under the Council for Official Statistics. Their aim has been to clarify the different quality aspect of statistics in terms of criteria to attain what is called 'sufficient quality'. These criteria should only be used for official statistics. The work group has presented three proposals for approval. The three approvals include (1) the clarification of sufficient quality (2) criteria formulation (3) commitments on quality made by the producers The clarification of sufficient quality has been made operational by formulating a large number of criteria 16 organized in three groups, Laws ordinances and regulations that steer the official statistics, Contact with users and Planning Implementation Follow-up. These dimensions of an increased quality concept will be further discussed in chapter 8. For the moment a well informed person in seasonal adjustment and time series analysis will notice that the special problems in seasonal adjustment are not 'there'. The most important issue for the moment is the user of statistics and the respondents. What makes seasonal adjustment special? That is the issue in the next paragraphs. 4. The Characteristics of a Time Series At any time the quality of statistics depends on the units of the populations, measurements of the variables, interactions with respondents, survey design, etc. and also on human subjective decisions. Any change of design and even the people involved in the data generating process will change the output and also the quality of data. This variability in the production process over time produces time series, with varying quality over time. This is particular true for national accounts where all errors are aggregated from different sources. Dramatic changes of the design produce outliers in the time series. Varying data quality of a time series has many unpleasant consequences. In the first place, statistical models are only rough descriptions of nature. Their expected optimal properties are short. Adding new data and/or revising data of a time series often requires a new model for adequate description of the series. Searching for optimality is expensive and may bee not feasible. It leads to locally optimal models but structural changes in terms of models and data leads to difficulties in comparisons over time. Any change of the description of the data in model based SA, e.g. from an ARIMA(110)(010) model to an ARIMA(002)(010) would introduce difficulties in monitoring the unobserved component over time. The most important sources of variation of economic time series are seasonal variation, trading-day effects, working day effects, leap-year effects, economic outliers and design effects. With economic outliers, we focus on special events produced by the economic system, e.g. a strike. By design effects, we think of effects such as changing estimator, changing sample, changing measurement design etc. A new sample may lead to level 15 The measurement of GDP is harmonized among member states in EU because the bill for membership in EU is in proportion to the level of GDP. 16 See Appendix 1.

10 10(29) shifts. 17 The purpose of seasonal adjustment is to estimate these unobserved components and present them in a way, which satisfies the user needs of information. 5. Uncertainty in Seasonal Adjustment To every method of seasonal adjustment there are several kinds of uncertainty about the estimated seasonal adjusted value, the trend, or any unobserved component of the time series. Everyone knows that X-12-ARIMA and TRAMO/SEATS produces different seasonal adjusted values for the same time series even when the same ARIMA-model is used. From a users point of view, that is very depressing because many users have the idea of true seasonal adjusted value. We can estimate the standard error of the irregular component, the standard error of the trend and the standard error of the seasonal adjusted series. Let st be the standard error of the seasonal adjusted series at time t for some series. If the model is true, we can use st to calculate a confidence interval for the seasonal adjusted value for time t and even for the forecasted values. Normally the confidence interval for forecasts include the cases of increase and decrease. Fig. 5.1 below is good example of such kind of uncertainty. The graph shows the trend for the Swedish Activity Index and a 95 per cent confidence interval for the trend. The length of the confidence interval is shown in graph 5.2. The length of the confidence interval is about 1 per cent for and increases for the forecasts to about ten per cent. We can not be sure about the future development, not even the direction in times, where the yearly growth is 2.5 per cent. This measure of uncertainty is very informative if we are certain that the model for calculating the standard errors are correct. If the model is false, the estimated standard error is biased. The true standard error is larger and the true confidence interval will be larger. These graphs are based on the officially published trend estimates. However, the confidence interval is not published. It should be remembered that this kind of uncertainty is in a sense the lower limit of uncertainty. Changing the model for the series will show a somewhat different activity. Now, if we are uncertain about which model nature has used to produce a particular time series, why not investigate all models in a particular class, e.g. ARIMA-models. This approach is outlined in the next section. 130 Fig Activity index. Index 2000=100. Trend. 95 per cent Confidence Interval Source: Statistics Sw eden Data up to and including August This is a common experience at Statistics Sweden.

11 11(29)

12 12(29) Fig Activity index. Length of 95 per cent Confidence Interval for the Trend Source: Statistics Sw eden Data up to and including August A Superpopulation Approach to Seasonal Adjustment Let Ω be a class of models, e.g., the ARIMA-class. 18 Let m be a member of the class, m Ω. sa = Y, t = t, L, t Y, t = t,, t the seasonal adjusted series given Let { } Y be a time series, { } T t t T m, t t L the model m Ω. For every model there are certain set of statistical measures, which are used to classify the quality or properties of the model m 19 for the time series Y T. Let q ( m, Y ), i = 1, L n be n test statistics based on the residuals from the estimated time series i T, using model m. Let Changes BIC = BICm ( YT ) = β m ( YT ); m Ω. SA SA SA SA SA Let Y = { Y Y, Y Y } m Ω δ be the changes of the seasonally adjusted time mt mt mt, L mtt mt, 2 1 T 1 SA μ = t m Ω series for a particular model. Let E ( Y ), t = t, L 1, t estimates of the SA values over the superpopulation of models. Let SA mt T T be the mean of the level δ SA t μ = SA t μ μ SA t 1 SA t 1, t = t, 2 L, t T be the relative changes of the SA values. According to the superpopulation way of thinking, there is a superpopulation of SA levels and changes. Now, we propose that the distribution of levels and changes is more informative than a single member of the population, i.e. a specific model. 18 Ω can also represent a wider class, e.g. 'all methods for seasonal adjustment'. 19 Here we assume that the model is used with a software, whose characteristics determine the estimated unobserved components. In our examples we use TRAMO/SEATS but the approach could use any model-based method.

13 13(29) Proposition 6.1 The whole distribution of level estimates of SA figures and changes in the superpopulation is more informative than any specific model We do not know, which is the true model but we have a measure of its likelihood, the BICmeasure for all models. Therefore, it seems wise to use the whole distribution of data and the likelihood principle by weighting over the whole distribution according to its likelihood or BIC-measure. Let SA β mymt ~ SA = m Ω μ t, t = t1, L, tt (6-1) β m m Ω be the weighted mean level over the superpopulation of models taking notice of the specific likelihood of a particular model. The corresponding relative changes are SA SA ~ ~ SA μ t ~ μ t 1 δ t =, t = t,, t ~ 2 L T (6-2) μ SA t 1 Proposition 6.2 The weighted means of level estimates of SA figures ~ SA μ t and the corresponding SA changes ~ δ t over the superpopulation are more informative than level estimates and changes based on any specific model The superpopulation approach could be applied to every unobserved components of a time series and their changes over time, e.g., the trend. All models contribute to the superpopulation estimates. Diagnostics given by a software could be applied to the whole superpopulation. So instead of looking at a particular model and its diagnostics, we are looking at the whole distribution over those models included. Models with low likelihood are given low impact to the estimate (6-1) and (6-2) and all corresponding measures. The calculation of ~ SA μ t and the changes ~ SA δ t takes more computer time than SA with a specific model. On the other hand, a SA processing system based on the superpopulation principle can be totally automatic. There is no need to revise any model used, because all models are used and their impact on a specific SA value is determined by its likelihood for a particular data set. That is, a scientific approach with no subjective elements. One of the drawbacks relying on a particular model, is that every model has a limited lifetime. Sooner or later, we have to choose another model. When doing so, we must be very concerned about the distance between the 'old picture' and the 'new picture' of what data tells us. Such considerations are very hard to make and very seldom on purely statistical grounds Some Examples of the Superpopulation Approach In this paragraph, an example of the superpopulation approach is given. We will use Swedish GDP, a quarterly time series covering the period 1993:1-2006:2. We have a user-defined regression variable, the number of hours worked for working-day adjustment. The software TRAMO/SEATS has been used. The official graphs of the levels and the quarterly changes 20 Many models could have about the same likelihood, but show quite different dynamics.

14 14(29) are shown in Fig. 7.1 and 7.2 below. According to the official figures, Swedish GDP is in a state of increase by about 5.5 per cent at an yearly level. Now, we will use the superpopulation approach and show the corresponding graphs for the measures (6-1) and (6-2) in Fig. 6.3 and 6.4 below. We have included about 140 ARIMAmodels. The graph 7.3 includes all 140 models. The mean over all models according to (6-1) is shown as the dotted line in the center of the distribution. As can be seen, there is a great dispersion over models. It can also be seen that non-seasonal models are included because of the remaining seasonal for some models in the graph. Now, because these models have low likelihood, they do not affect the BIC-weighted mean so much. If we exclude non-seasonal models, we get the graph 7.4. As can be seen, there is a lot of uncertainty involved. Now, we show the corresponding for the quarterly changes over the whole distribution and for the reduced set of models in Fig. 7.5 and 7.6. If the class of models are reduced to the 'best 15 according to BIC', graph 7.7 appear. The dotted line over the best 15 in BIC-sense is the mean over the best 15 models. The line with label 'All models_bic' is the estimates according to (6-2). 'Best_15_BIC' includes the 15 best models according to the BICmeasure. As can be seen, there are differences at the level of about one per cent between the two curves. The estimated change for 2006:Q2, is about 3.5 for the estimate (6-2) and about 4.5 for the 'Best_15_BIC'. In terms of uncertainty due to the selection of ARIMA-models, we can put the number one percent over the business cycle. The maximum difference is about 1.5 per cent. Fig. 7.1 Swedish Gross domestic product (GDP).Actual and seasonally adjusted values in constant prices. ARIMA(110)(010) Unadjusted, million SEK Seasonally adjusted, million Source: Statistics Sw eden Data up to and including the Second Quarter 2006 Fig. 7.2 Swedish gross domestic product (GDP) Per cent change from previous quarter, annual rate. Constant prices. Seasonally adjusted. ARIMA(110)(010) Source: Statistics Sw eden Data up to and including the Second Quarter 2006

15 15(29) Fig. 7.3 Swedish Gross domestic product (GDP).Seasonally adjusted values in constant prices. 140 ARIMA-models Fig. 7.4 Swedish Gross domestic product (GDP).Seasonally adjusted values in constant prices. Non-seasonal models excluded Fig. 7.5 Swedish Gross domestic product (GDP).Seasonally adjusted values in constant prices. Quarterly changes at an yearly level. 140 ARIMA-models

16 16(29) Fig. 7.6 Swedish Gross domestic product (GDP).Seasonally adjusted values in constant prices. Quarterly changes at an yearly level. Non-seasonal models excluded Fig. 7.7 Swedish Gross domestic product (GDP).Seasonally adjusted values in constant prices. Quarterly changes at an yearly level. All models_bic Best _15_BIC Next, we show some features of uncertainty concerning the estimation of outliers, date, type and level. In the graph 7.3 below, we can see the outliers for July 1997 for the industrial production index (IPI) in manufacture in Sweden given by 140 ARIMA-models. Graph 7.3 shows that the identification of an outlier as well as its magnitude depends on the chosen model. For July 1997, the majority of models do not identify an outlier. An outlier is identified in about 30 models with a value in the range of In graph 7.4, we show the outliers for the whole series with some ARIMA-models. It is obvious that there is uncertainty about the outlier effect stemming from the model. ARIMA(000)(110) produces two level shifts, in 1993 and in ARIMA(000100) brings a series of level shifts, in 1993, 1997 and in In graph 7.5, the working-day effect is shown over models. 21 In this case, we can claim that the reason is due to a bad specification. There is a trend in the series, which should be taken care of. However, in other cases it the different of the outliers are not clearly understood.

17 17(29) Fig. 7.8 Outliers over some ARIMA-models in Swedfish Manufacture in July Models. jul Fig. 7.9 Outliers over some ARIMA-models in Swedfish Manufacture Fig Working-day Effect for IPI in Swedish Manufacture in Dec Models. Dec Model jan-90 jan-91 jan-92 jan-93 jan-94 jan-95 jan-96 jan-97 jan-98 jan-99 jan-00 jan-01 jan-02 jan-03 jan-04 ARIMA(000001) ARIMA(000010) ARIMA(000011) ARIMA(000100) ARIMA(000101) ARIMA(000110) ARIMA(000111) ARIMA(001000) Model

18 18(29) Eliminating working-day effect over 90, gives the distribution of working-day effect among models as shown in graph 7.6 below. Fig Distribution of Working-day Effect in Dec for Swedish IPI in Manufacture At present the superpopulation thinking has not been formalized at SCB. However, it is in use in the way that every chosen model for official statistics are related the properties of the whole distribution of central measures for the components. This is also the case for all diagnostics. We show some examples of graphs in use to see the whole range of diagnostics over the superpopulation below. From the graphs below, we get a fast picture of some diagnostics and user related quality measures over all models. The graph over the quality measure is just a prototype to be further developed in terms of components.

19 19(29) BIC for GDP ARIMA(000001) ARIMA(001001) ARIMA(002001) ARIMA(010001) ARIMA(011001) ARIMA(012001) ARIMA(100001) ARIMA(101001) ARIMA(102001) ARIMA(110001) ARIMA(111001) ARIMA(112001) ARIMA(200001) ARIMA(201001) ARIMA(202001) ARIMA(210001) ARIMA(211001) ARIMA(212001) Standard Deviation of Quarterly Changes for GDP Yearly Level ARIMA(000001) ARIMA(001010) ARIMA(002011) ARIMA(010100) ARIMA(011101) ARIMA(012110) ARIMA(100111) ARIMA(102000) ARIMA(110001) ARIMA(111010) ARIMA(112011) ARIMA(200100) ARIMA(201101) ARIMA(202110) ARIMA(210111) ARIMA(212000) Normality Test for GDP ARIMA(000001) ARIMA(001001) ARIMA(002001) ARIMA(010001) ARIMA(011001) ARIMA(012001) ARIMA(100001) ARIMA(101001) ARIMA(102001) ARIMA(110001) ARIMA(111001) ARIMA(112001) ARIMA(200001) ARIMA(201001) ARIMA(202001) ARIMA(210001) ARIMA(211001) ARIMA(212001) Durbin Watson Test for GDP ARIMA(000001) ARIMA(001001) ARIMA(002001) ARIMA(010001) ARIMA(011001) ARIMA(012001) ARIMA(100001) ARIMA(101001) ARIMA(102001) ARIMA(110001) ARIMA(111001) ARIMA(112001) ARIMA(200001) ARIMA(201001) ARIMA(202001) ARIMA(210001) ARIMA(211001) ARIMA(212001) Ljung & Box T-value for GDP ARIMA(000001) ARIMA(001010) ARIMA(002011) ARIMA(010100) ARIMA(011101) ARIMA(012110) ARIMA(100111) ARIMA(102000) ARIMA(110001) ARIMA(111010) ARIMA(112011) ARIMA(200100) ARIMA(201101) ARIMA(202110) ARIMA(210111) ARIMA(212000) Pierse Q-value ARIMA(000001) ARIMA(001001) ARIMA(002001) ARIMA(010001) ARIMA(011001) ARIMA(012001) ARIMA(100001) ARIMA(101001) ARIMA(102001) ARIMA(110001) ARIMA(111001) ARIMA(112001) ARIMA(200001) ARIMA(201001) ARIMA(202001) ARIMA(210001) ARIMA(211001) ARIMA(212001) The Sign Test for GDP ARIMA(000001) ARIMA(001001) ARIMA(002001) ARIMA(010001) ARIMA(011001) ARIMA(012001) ARIMA(100001) ARIMA(101001) ARIMA(102001) ARIMA(110001) ARIMA(111001) ARIMA(112001) ARIMA(200001) ARIMA(201001) ARIMA(202001) ARIMA(210001) ARIMA(211001) ARIMA(212001) Quality Measure for GDP ARIMA(000001) ARIMA(001001) ARIMA(002001) ARIMA(010001) ARIMA(011001) ARIMA(012001) ARIMA(100001) ARIMA(101001) ARIMA(102001) ARIMA(110001) ARIMA(111001) ARIMA(112001) ARIMA(200001) ARIMA(201001) ARIMA(202001) ARIMA(210001) ARIMA(211001) ARIMA(212001)

20 20(29) 8. On a Quality Measure in Seasonal Adjustment There are many important characteristics of a SA procedure. SA is usually based on some theoretical arguments about the 'sources of seasonality'. There are at least four actors on the scene of SA, a theoretical framework, the person performing SA, a software and a computer and the user of statistics. The theoretical framework could be divided into 'economic theory' as discussed in Hylleberg 22 and statistical theory, which provides the statistical tools for performing SA. It also provides the statistical diagnostic tests used for the statistical validation of the statistical properties of a procedure. The arguments given by Hylleberg and other economists are against SA. Instead seasonality should be included in an economic model and part of the specifications. It is also important to realize that SA depends on properties related to the person performing SA, competence and may bee devotion. There is always an element of say Bayesian subjective thinking in the decisions on SA. At last there is a user, who is concerned with solving a problem using time series data. SA is made to make it easier for the user to understand something. If possible a numeric quality measure will be given in It will be constructed in lines with where wuiqci + QM (8-1) Qui Q w w si ui ui = wsiqsi = User quality component = Scientific quality component = Weights of importance for the user quality component = Weights of importance for the scientific quality component w si + w ui = Guidelines issued by the Council for Official Statistics in Sweden The official guidelines for quality in statistics in Sweden are given in Appendix 1. Those parts which are of relevance for SA are underlined. Translated to SA, the following criteria should be included in a quality concept for seasonal adjustment: Table Guidelines for Quality in Seasonal Adjustment in Sweden Contact with users - Objectives of seasonal adjustment of the statistics are clearly formulated. (O1) - The main users and their main areas of use and future needs are documented.(o2) - The planned characteristics of seasonal adjustment of the statistics are based on a dialogue with the main users.(o3) - Annual follow-up of the quality of the seasonal adjustment is conducted with the main users and documented.(o4) 22 See e.g. Hylleberg S. (2006),

21 21(29) Planning Implementation Follow-up - Relevant EU-regulations for seasonal adjustment are given and complied with.(o5) - The choice of statistical method can be motivated based on scientific principles.(o6) - Quality studies for seasonally adjusted series are done regularly.(o7) 8.2 Statistical Properties of Seasonal Adjustment Model-based SA relies on the assumption that a particular model is correct in the light of data. There are many test-statistics in the diagnostics with TRAMO/SEATS, DEMETRA, or with X- 13-S. These measures should be included in a appropriate way in the quality discussions for a particular seasonal adjustment and also in the yearly quality report as suggested in 8.1. The selected diagnostics are included for the justification of the procedure from scientific requirements. The properties based on the likelihood function, the AIC or BIC or similar measures should also be included. In model-based seasonal adjustment, forecasts are made. It is important that the forecasts errors are small. Forecast errors should be included in measures of quality. 8.3 User related properties of Seasonal Adjustment Scientific justification is of no interest to the user. He/she is interested if seasonal adjusted data can be used in decision making and /or analysis. In particular the comparisons over time is of vital interest. The yearly changes of the original data are also used as a reference measure of change. In this paragraph, we suggest some characteristics of the seasonally adjusted series and the major unobserved components, which could be part of the discussion on quality in seasonal adjustment Smoothness Smoothness has to do with the variability of changes of the seasonally adjusted series and/or the trend. In 1983 Dagum and Morry [0] use smoothness and revisions in criteria for the evaluation of a SA procedure. They conclude that 'for policy-making, smoothness and minimization of revisions are very important and often used criteria'. Smoothness was also mentioned as desirable properties in the recommendations made by Eurostat at the SAM98- Conference in Bucharest.. However, there does not seem to be much support for the smoothness property from the scientific community. At SCB the smoothness property has been used as a complementary property for the discrimination between models, which are acceptable according to statistical criteria, i.e. have roughly the same likelihood in terms of the BICmeasure. This was discussed at a meeting with the scientific council of SCB in There was no general support for the smoothness idea. As Dagum and Morry mentioned, smoothness must be a complementary criteria, because we can control the degree of smoothness we want by choosing a special procedure in SA, e.g. by choosing a special estimator for the seasonal component. Despite there is no general agreement, a smoothness proposition is made. Proposition The user of SA statistics prefer smooth changes of a SA series in comparison with volatile changes of a SA series. 23 The members of the scientific council (Vetenskapliga rådet) are professors in Statistics.

22 22(29) Revisions Revisions in SA are due to revisions of the original data and/or revisions of the models and/or options used in SA. Even if there would be no revision of original data, there will be a revised SA series when a new observation is included in the series, even if we use the same model and the same specifications. Revisions are made to the first k and the last T-k observations, k depending on the convergence properties of the used estimators of the components. When forecasts of the original series are used such as the X-12 and the TRAMO program are used, the revisions of the forecasts will be part of the revision error. If the forecasts of the original series always were perfect, any revision of a SA value would be caused only by revisions of the original data. Users of statistics do not like revisions. Users of SA statistics are familiar with revisions, but hate them. Sometimes they do not understand them providing arguments like 'Why has the SA figure for time T-1 changed so much when the original value for T-1 is unchanged?'. Changing a model for the series will normally cause revisions of the whole SA series. Proposition Users do not like revisions Proposition Revisions in seasonal adjustment should be minimized Proposition Revisions should be part of a yearly quality report for seasonal adjustment Outliers Outliers in time series has been implemented in TRAMO/SEATS and in X-12-ARIMA. There are two types of outliers, outliers stemming from the design of statistics and outliers stemming from the economic system (nature). The knowledge of the magnitude, type and date for outliers are always of importance but for different reasons. The main interest for knowing the magnitude, date and type of outliers from the user is restricted to outliers stemming from the economic system. For instance, the effect of a strike or a consumption tax is of major interest. However, the effect on a time series stemming from the fact that the producer has changed survey design if of no interest for the user. If this is so, the following propositions should be of importance. Proposition Outliers stemming from nature as part of the economic system should be estimated as separate components. Proposition Estimated outliers stemming from nature as part of the economic system should be published. Proposition Outliers in time series stemming from the design or the producer of statistics is of minor interest for the user of statistics. Proposition

23 23(29) Outliers in time series stemming from the design or the producer of statistics should not be a cause to inappropriate time comparisons made by the user. Proposition Outliers in time series stemming from the design or the producer of statistics should be made known to the user in meta-data on statistics. Proposition Outliers in time series stemming from the design or the producer of statistics should be part of a quality control system of the statistics. Outliers are estimated with a specific ARIMA-models. From a theoretical point of view, they are treated as exogenous variables independent of the specification of the ARIMA-model. However, the estimated outlier-effect depends on the choice of a particular model. A different model will make the estimated outlier-effect a bit different. From a superpopulation approach of seasonal adjustment, there will be a distribution of the outlier-effect over the models used. This was discussed in chapter 5. That leads to Proposition Uncertainty about outliers stemming from uncertainty of models should be estimated and published Proposition Outliers should be discussed with the main users Calendar effects The estimation of calendar effects by means of regression models were introduced by the programs TRAMO/SEATS and X-12-ARIMA. Calendar effects are effects from Easter, trading-days, working-days and Leap-year. The following propositions are made: Proposition Calendar-effects should be estimated in seasonal adjustment and published. Proposition Calendar-effects should be discussed with the main users of statistics Yearly Changes of the original series There is a strong tradition to present yearly changes of a time series. It is believed by many users of statistics that taking yearly changes eliminates seasonal variation of a time series, which in most instances is the most important component of a time series. At SCB, yearly changes of GDP and the seasonally adjusted quarterly change raised to an yearly level are shown on SCB:s homepage at asp.

24 24(29) Fig Gross Domestic Product (GDP) Per cent change over four quarters. Constant prices. Calendar adjusted Source: Statistics Sw eden Data up to and including the Second Quarter 2006 Fig Gross domestic product (GDP) Per cent change from previous quarter, annual rate. Constant prices. Seasonally adjusted Source: Statistics Sw eden Data up to and including the Second Quarter 2006 These are two official graphs of the Swedish economy as measured by GDP. The yearly changes of original GDP is from the users point of view a very reliable measure of change. It can easily be understood. However, many users are aware that this measure of change is giving wrong timing of the turning points. The yearly changes of original GDP as shown from fig shows the turning points about a half year too late but give a clear picture of the business cycle. If the graph is shifted half a year to the right at the time scale, it would have the correct dynamics and timing The last presented change, i.e., the change from 2005:2 to 2006:2 about 5 per cent is relevant for 2005:4 with respect to correct dynamics and turning points. The quarterly changes of the seasonally adjusted GDP raised to a yearly level provide correct dynamics to a price of higher variability, i.e., higher uncertainty. From the users point of view, the yearly changes of the original values are considered as a reference reliable measure of change.

25 25(29) Proposition Changes of a seasonally adjusted series at an annual rate should be compared to the yearly changes of the original series with appropriate time shift. We are proposing a distance measure between the yearly changes of the original series and the SA changes at an yearly level. This can be done in many ways, e.g. as follows: where SA D = ( Δ Δ / m t tm 2 t τ ) n, Δ = seasonal adjusted change at a yearly level, = yearly change of the original series with appropriate phase shift τ and n=the number of comparisons made. SA tm Δ t τ D m is the mean deviation between the seasonal adjusted series at a yearly level and the corresponding yearly changes of the original series. A large value means that the changes are unrelated, a small value that they are similar but phase shifted. From a users point of view, D should be small. Therefore, e.g. 1/ D could be used as a weighing factor among models m where 1 / D m = 1. m Ω m In this view, models are evaluated in terms of their closeness to the relevant yearly changes of the original series. 8.4 Uncertainty in Seasonal Adjustment As has been discussed, there are many kinds of uncertainty in seasonal adjustment. Consistent with the guidelines of official statistics, we propose the following criteria for any seasonal adjusted statistics. Proposition Uncertainty in seasonal adjustment stemming from the choice of different models should be measured and made known to the users. The superpopulation approach of seasonal adjustment makes it possible to very clearly show the uncertainty in seasonal adjustment. Any particular model show a particular picture of the development in the economy. We propose that an 'admissible' set of models are used to illustrate this uncertainty in seasonal adjusted series and their changes over time in lines with the illustrations given in chapter 7. From a users point of view, this uncertainty is of most interest. Showing the whole distribution of change for all models or an 'admissible' set of models, as well as a central measure of the distribution. The user should know for sure that the change of a variable has been say 5 per cent and that the uncertainty interval is between 4 and 6 per cent. The superpopulation approach makes it possible to show uncertainty in seasonal adjustment for any unobserved component, i.e., the seasonally adjusted series, the trend

26 26(29) estimates, outliers, trading-day and working-day effects and the leap-year effect. Thus, if proposition is broken down to unobserved components, the following propositions could be made Proposition Uncertainty in the seasonal adjusted series stemming from the choice of different models should be measured and made known to the users. Proposition Uncertainty in the trend estimates stemming from the choice of different models should be measured and made known to the users. Proposition Uncertainty in the identification and estimation of outliers seasonal adjusted series stemming from the choice of different models should be measured and made known to the users. Proposition Uncertainty in the trading-day effects stemming from the choice of different models should be measured and made known to the users. Proposition Uncertainty in the working-day effects stemming from the choice of different models should be measured and made known to the users. Proposition Uncertainty in the Easter-effect stemming from the choice of different models should be measured and made known to the users. Proposition Uncertainty in the leap-year effect effects stemming from the choice of different models should be measured and made known to the users. 8.5 Software and IT Software and IT has a quality dimension by itself but of course of no interest from the user of statistics. However, if there is a bug in the software, it could affect the output from the system, e.g. as an error. The main issue of the conference SAM-98 in Bucharest was software for seasonal adjustment. The decision to develop DEMETRA was made a few years before, despite some doubt from members of the working group of seasonal adjustment organized by Eurostat. Software and IT is still a problem for the design of a SA environment at the producer level. It is about internal software code, interface to run the software, graphics support, etc. It is well known that e.g., the DEMETRA interface often show this icon

27 27(29) Fig Typical DEMETRA-session In terms of system quality, the following propositions are made: Proposition The source-code used for seasonal adjustment should be accessible in open. form. Proposition The software for seasonal adjustment for official statistics should be free from errors (bugs). Proposition Any error found in the source code should be reported to a central office for correction. Proposition Any error found in the source code should be made known to the user of statistics. Proposition There should be an international commitment on the maintenance of the software for seasonal adjustment. Proposition Any software used for seasonal adjustment should be wholly tested with a test report before use for official statistics.