ESSNET USE OF ADMINISTRATIVE AND ACCOUNTS DATA IN BUSINESS STATISTICS

Transcription

1 ESSNET USE OF ADMINISTRATIVE AND ACCOUNTS DATA IN BUSINESS STATISTICS WP4 TIMELINESS OF ADMINISTRATIVE SOURCES FOR MONTHLY AND QUARTERLY ESTIMATES THE USE OF REGRESSION ESTIMATORS FOR ADMIN DATA-BASED STS ESTIMATES (SGA 2011: DELIVERABLE 4.2) Daliute Kavaliausiene 1, Milda Šličutė-Šeštoienė, Pieter Vlag 1 corresponding author: Dept. of Methodology and Social Statistics LT Statistics Lithuania Daliute.Kavaliausiene@stat.gov.lt Telephone: (0370)

2 Contents Contents... 2 Summary Introduction Regression estimators Notation and formulae Description of the National Practice Comparison of Statistical and Administrative data Editing of administrative data Estimation in STS using administrative data of the period t Results Use of regression type estimation techniques in other statistical surveys Conclusions and remars Acnowledgements References Annex

3 Summary Statistics Lithuania overcomes the problem of timeliness by using data of previous periods in regression type estimation techniques. A mixed-model using sample data of the current period and VAT-data of the previous period has been tested. The results obtained using the regression-type estimation techniques are discussed. These techniques require the use of auxiliary data having good correlation with the variable of interest. Regression techniques are also used to compare the variable definitions used by the Survey and the VAT declarations data. Results after editing of the micro-data are also presented. A discussion on the selection of auxiliary variables, outliers treatment and non-randomness of the sample is put forward along with software choices (e.g. the use of SAS and CLAN). After thorough data and time series analysis, it is explained how to use the VAT-turnover of the period t-1 as auxiliary information in regression type estimators. The new methodology has been implemented from 2010 onwards and has delivered a significant decrease in the variance of the estimates. As an example, VAT-data from the State Tax Inspectorate database is successfully used in the estimation of monthly Domestic Trade statistics in the economic activities G45 (Wholesale and retail trade and repair of motor vehicles and motorcycles) and G47 (Retail trade, except of motor vehicles and motorcycles). The incorporation of auxiliary information at the estimation stage in Labour statistics has also significantly improved the quality of the results. Results of different estimators at regional breadown are presented. Statistics Lithuania has shown that the use of VAT-data of previous periods has advantages (e.g. higher quality results, reduction of response burden) but also some disadvantages (e.g. both admin data and surveydata are needed, more data editing is necessary and complex estimators require higher sills and nowledge). Keywords: GREG-estimator, Horvitz-Thompson estimator, (quarterly) Lithuanian Service survey, (monthly) Lithuanian Domestic trade survey, Ratio estimator, VAT-Turnover. 3

4 1. Introduction Statistical production rests on two pillars: statistical surveys and administrative data sources. Wider use of administrative data allows a decrease in the number of statistical surveys, thus reducing the statistical response burden. However, National Statistical Institutions (NSIs) are often faced with the problem that administrative records tend to be unavailable or incomplete at the time when they are needed. Short Term Statistics (STS) are especially sensitive to timeliness issues. Usually, the estimates have to be produced in a very short time. The fact that administrative data are not available at the time they are needed may discourage the substitution of survey data by administrative records. Because administrative records are by definition collected for purposes other than statistical production, NSIs cannot influence the timing issue directly. However the cost-saving effect of using administrative data instead of statistical data collection is especially significant in the case of Short Term Statistics. This encourages strong efforts to solve the timeliness problem by persistent methodological wor and data analysis. Roughly speaing, two general situations could be analysed: 1. A fairly complete administrative dataset of the period t is available in time (with the rest of the data being provided later). 2. No or very few data of the period t are available in time (see Figure 1). In the first case, the natural wish of a statistician would be to complete the dataset to the whole population. Theoretically, many ways of doing this are possible. Various methods of imputation are the most widely used solution. A thorough qualitative data analysis (including historical) should help the statistician to mae a choice. In the second case, it is clear that the statistical survey cannot be directly substituted by administrative data. But, it does not mean that the administrative data cannot be useful for the statistical output. The use of administrative data of the previous periods as auxiliary information in regression type estimators can add effectiveness to the sampling plan, which results in increasing quality parameters of the estimates and reduction of the sample size. General problem - administrative data are not available at the time they are needed Use of incomplete data set Use of data of previous periods Imputation Figure 1. Scope of timeliness problem Regression type estimation technique Nowcasting 4

5 Another option in this case is nowcasting, which is discussed in the ESSnet Admin Data deliverable 4.3 of 2011 by P.Vlag, R. Bier, M. Teneva and E. Toivanen: EXTRAPOLATING ADMIN DATA FOR EARLY ESTIMATION: SOME FINDINGS AND RECOMMENDATIONS FOR THE ESS. The next chapters of this report deal with the use of administrative data as auxilliary information. It is focussed on the situation in Lithuania, where the deadline for providing Value Added Tax (VAT) data to the State Tax Inspectorate (hereinafter referred to as the STI ) is the 25th day after the reference period. The data become available to Statistics Lithuania at the 27th day, but the results of the monthly domestic trade survey have to be ready 22 days after the reference period. For five of the STS indicators, results are published in a monthly news release between 35 and 55 days after the end of the reference period. In practice, VAT-data from the State Tax Inspectorate database are successfully used for estimation of turnover in the quarterly STS-survey on services. 2. Regression estimators Regression estimators occupy a prominent role in recent literature and also in the production of official statistics, but the availability of powerful auxiliary information is crucial to efficient estimation. Regression estimators could also be used as a tool for improving the quality of the results where there is a timeliness problem. For simplicity, only two estimators will be discussed: the classical general regression estimator (GREG) introduced by Särndal, Swensson and Wretman (1992) and the ratio estimator (which is special case of GREG but a much simpler form of it). These two estimators are widely analysed in recent literature and widely used in official statistics. Nevertheless one should not forget that there exist a lot of other estimators incorporating regression model (Synthetic estimator, Empirical best linear unbiased predictor etc.). There is no estimator which is always the best one, but different estimators suit different data. Finding the best estimator for particular data is usually quite a complicated tas Notation and formulae Consider a finite population U={1, 2,...,,...N}, from which a probability sample s ( s U ) is drawn with a given sampling design, p (). That is, p(s) is the probability that s is selected. The inclusion probabilities Pr( s) and Pr( & l s) are assumed to be strictly positive. Let l y be the value of the variable of interest, y, for the -th population element. Let x ( x1, x 2,..., xj,..., xj ) be the auxiliary variable vector of dimension J 1, the value x is assumed to be nown for every element U. The target parameter is the total of the variable of interest, y, Y. The estimators of the total of Y U y, discussed in this paper, have the following form: U y Horvitz Thompson: 1 HT Y ˆ y, s 5

6 Ratio: Yˆ Ratio Yˆ X X, HT HT 1 Yˆ 1 HT HT Xˆ 1 X ˆ 1 Generalised regression (GREG) ˆ GREG ˆ HT ˆ ( ˆ HT Y Y B X X ), J j1 j j j where X i.e. the total of the auxiliary variable x j (derived from a register); j x j U X HT 1 variable x 1; and s 1 x 1 i.e. the Horvitz Thompson estimator of the total of the auxiliary ˆ Bˆ ˆ B ( i.e. the vector of regression 2 1 1,..., B J ) ( xx / ) x y / s s coefficients derived by applying a linear regression model. For simplicity, only the simple (linear) regression model is discussed here, however one should remember that a wide family of GREG estimators could be generated using different assisting models (linear, non-linear, generalised linear, mixed, etc. models) Possible auxiliary information for STS in regression estimators Regression estimation techniques allow the incorporation of administrative data into the survey, thus achieving better accuracy for the results. Many inds of complementary data can be used as auxiliary information in regression type estimators. For example: VAT data VAT is a very important administrative data source for STS surveys, although the data differ from statistical data by coverage and by definitions. Definition problems often can be solved methodologically. If VAT-data arrive too late, which is especially the case for preliminary STS outputs, administrative data for a previous period (t-1) (or (t-12)) can be used as auxiliary information. BusinessRegisterdata It may be worthwhile incorporating some Register variables into the regression model: e.g. size of enterprise (in terms of number of employees or in terms of turnover), economic activity etc. Since GREG and ratio estimators are almost unbiased, the accuracy of the results is quite easy to assess (mean square error can be calculated) Description of the National Practice 3.1. Comparison of Statistical and Administrative data Lithuania s Monthly Domestic Trade sample survey was implemented in Its methodology was based on the Horvitz-Thompson estimator, as no administrative data were available. However, after co-operation agreements were signed with the STI, the State Social Insurance Fund Board 6

7 (hereinafter referred to as the SSIFB ), the Customs Department and the State Enterprise Centre of Registers, these very important data became available for Statistics Lithuania. For example, data on the Annual averaged number of employees and Wages and salaries is available for all units in the STS survey populations. These indicators are no longer being collected from respondents, because the statistics are based fully on administrative data. A very important step forward was the access to VAT declarations. The definition of turnover derived from VAT differs slightly from the definition of Turnover in the statistical survey, which is defined according to the STS regulation and therefore does not include the proceeds of sale of assets. The results of the comparison are provided in the table below. In the remainder of this document we refer to VAT-turnover in case of turnover derived from VAT and simply to turnover when referring to turnover as defined in the STS-regulation. Month of 2011 Survey turnover (millions of litas) VAT turnover (millions of litas) Difference (%) ,2 2026, ,5 2055, ,6 2353, ,6 2424, ,7 2512, ,9 2533, ,8 2449, ,4 2590, ,0 2526, ,7 2541, ,9 2439, ,5 2792,5 3 Table 1. Comparison of VAT turnover data with Survey turnover data Although the VAT-turnover and the statistical data on turnover should be close to identical for each enterprise (when the discrepancies described above are taen into account), there still remain some differences simply due to the fact that they come from two different sources. One of the reasons can be that different individuals prepare VAT declarations and respond to statistical questionnaires. These differences might be difficult to estimate. When producing a monthly indicator, this could constitute a problem, but as these differences are believed to occur at random, one would expect that differences at unit level pull in both directions, thus cancelling each other out. However, VAT-turnover is highly correlated with statistical data on turnover on NACE 4-digit and even more detailed levels. About 75% of the active enterprises (mainly the big ones) are VATpayers. In Lithuania VAT-turnover data derived from the VAT-declarations are available monthly, but not in time for the monthly STS outputs. 7

8 The correlation of VAT-turnover of the period (t-1) derived and turnover from statistical survey is also very high (see table 2). Month Turnover(t)/ VAT(t-1) Turnover (t)/ VAT(t) Turnover (t)/ VAT(t-1) Turnover (t)/ VAT(t) Table 2. Correlation coefficients of VAT-turnover of the period t and period t-1 derived and turnover from statistical survey (totals) Editing of administrative data Good correlation is not the only important condition, because this shows only the linear dependence of the two variables. Let s suppose a model of the linear dependence is analysed: y i 1 2 x i i ; i 1,, m, where y i is the survey variable (turnover) and x i is the auxiliary variable (VAT-turnover). It is to be determined whether the use of auxiliary information can be effective in the estimation domain. The regression model should correspond to the data. Inadequacy can occur for various reasons, one of which is the impact of outliers. It is important to find the outliers and to chec if they have a big impact on the estimates of coefficients of the regression equation. It means that the equation of the regression line with an outlier in the data should be compared with the equation of the regression line after deletion of the outlier. The nature of the outlier should be examined, because it may be an input error or simply an exceptional value. In such cases the outlier can be eliminated, but the outlier cannot be eliminated without any information about its origin, because it could be the case that the model simply does not suit the data. There are several methods to detect outliers. Some of them are realised in SAS. 1. Hat diagonal It shows the distance between and x i : 8

9 ( ) ( ), where (1) The observation y, x ) is to be defined as an outlier, if ( ). Here p = the number of ( i i independent variables. 2. Standardised residual, where h i is defined in (1) above; ; ( ) are forecasted values; ( ) (estimation of the variance ); and ( ), (sum of squares) If STUDENT i > 2, then the point y, x ) can be an outlier. 3. Coo measure ( ) ( ) ( i i (Some authors suggest to apply the rule STUDENT i > 3.) The point y, x ) is an outlier, if D i > F 0.5(2,n-2). If the special tables are not available to find F, ( i i then the rule D i > 1 can be used. Influential observations are those that, according to various criteria, appear to have a large influence on the parameter estimates. Let b(i) be the parameter estimates after deleting the ith observation; s(i) 2 be the variance estimate after deleting the ith observation; X(i) be the X matrix without the ith observation; be the ith value predicted without using the ith observation; be the ith residual; and h i be the ith diagonal of the projection matrix for the predictor space, also called the hat matrix: h i = x i (X'X) -1 x i ' Belsley, Kuh, and Welsch propose a cut-off of 2p/n, where n is the number of observations used to fit the model and p is the number of parameters in the model. Observations with h i values above this cut-off should be investigated. The DFBETAS statistics are the scaled measures of the change in each parameter estimate and are calculated by deleting the ith observation: where (X'X) jj is the (j,j)th element of (X'X) -1. 9

10 In general, large values of DFBETAS indicate observations that are influential in estimating a given parameter. Belsley, Kuh, and Welsch recommend 2 as a general cutoff value to indicate influential observations and as a size-adjusted cutoff. For each parameter estimate, the INFLUENCE option in the SAS procedure proc reg calculates a DFBETA diagnostic for each observation. The DFBETA diagnostic for an observation is the standardized difference in the parameter estimate due to deleting the observation, and it can be used to assess the effect of an individual observation on each estimated parameter of the fitted model. Figure 2 shows how proper data editing helps with the fitting of a linear model to the data. a) b) Figure 2. VAT declarations data a) before editing, b) after editing Although the computer software lie SAS is very powerful, data editing is rather creative procedure and the blind run of the computer program hardly can be a good solution. It is necessary to have a silled specialist. However it also contains a ris: the sudden loss of an employee can have a very negative effect on the quality of the survey, especially in STS Estimation in STS using administrative data of the period t-1 STS in Lithuania are currently prepared on the base of statistical survey. The Horvitz-Thompson estimator was used for a long time to produce turnover estimates. In order to reduce the statistical response burden, the possibilities of decreasing the sample size by using auxiliary information were analysed. It is nown that auxiliary variables (with nown population sum) used in regression type estimators can under certain circumstances be very helpful. Let y and x be the study variable and an auxiliary variable, respectively. Denote the population coefficient of correlation as ρ 2 = s 2 xy/ s 2 y s 2 x, where s 2 y and s 2 x are the population variances of the variables x and y, and s xy is their covariance. 10

11 In the cases of Horvitz-Thompson and regression estimation, the variances of the sample mean can be written down, respectively, as: ( ) and ( ) ( ) Note that the variance of the regression estimator is always less than the variance of the Horvitz- Thompson estimator, provided only the coefficient of correlation of the study variable and auxiliary variables is non-zero. Otherwise, they are the same in the case of a simple random sample and for not too small populations. Simulations with real domestic trade and services data done at Statistics Lithuania show that the use of VAT-turnover data of the period t-1 gives a significant decrease in variance in comparison with the Horvitz-Thompson estimator. The coefficients of variation in Domestic trade and Service surveys are provided in tables 3. and 4. below. As can be seen from the tables, the GREG estimator and the ratio estimator do not differ significantly in terms of accuracy, but the ratio estimator is quite simple and easy to interpret, compared to the GREG estimator. Because of simplicity, the ratio estimator was chosen to produce the estimates of turnover in the real Domestic Trade survey. Year Month Horvitz -Thompson estimator Ratio estimator GREG estimator Table 3. Coefficients of variation for monthly estimates, where VAT-turnover of the month (t-1) is used as an auxiliary variable for Ratio and GREG estimators. 11

12 Year Quarter Horvitz-Thompson estimator Ratio estimator GREG estimator I Q II Q III Q IV Q I Q II Q III Q IV Q I Q II Q III Q IV Q I Q II Q Table 4.. Coefficients of variation for quarterly services estimates, where VAT-turnover of the current quarter is used as an auxiliary variable for the Ratio and GREG estimators. 3.4 Results In order to determine the conditions under which regression type estimates can be used for STS and to highlight the possible shortcomings of the method, some tests on turnover from domestic trade survey and VAT-turnover data were performed. VAT-turnover of the previous period (t-1) and period (t-12) were used as auxiliary information in regression type estimates. The results were revised using VAT-data of the period t and compared with Horvitz-Thompson estimates. The 24 months time series of turnover data was analysed. As could be expected, good correlation between survey variable and the auxiliary variable is ey for good results. For example, coefficients of correlation between survey variable (turnover) and VATturnover data of the previous months in economic activity G45 are provided in the table below: Month Inc.(t)/VAT(t-1) Inc.(t)/VAT(t) Inc.(t)/VAT(t-1) Inc.(t)/VAT(t) Table 5. Coefficients of correlation between the survey variable (turnover) and VAT-turnover As can be seen, the data are better correlated in

13 VAT-turnover The chart below illustrates the differences between the various estimates in the same economic activity during the two years 2010 and The red line represents the true revised results, based on VAT-turnover data of the current month t-1 t h t Figure 3. Turnover in economic activity G45 (Sale of motor vehicles and motorcycles) Note that the estimators using auxiliary information wor much better in (It is mainly due to the experience of the application of the method and better data editing sills of the employees). The same can be said about the sampling error (CV): t-1 t h Figure 4. Relative standard error in economic activity G45 The Figure 4. demonstrates the relative standard error. The ratio estimator with VAT-turnover of the period (t-1) as an auxilliary variable provides lower relative standard errors than the Horvitz- Thompson estimator (blue and green columns). It allowed reducing the sample size in G45 by 13 percent. If the administrative data of the current period were available in time, the sampling errors would be lower than in case with (t-1) (red columns). 13

14 3.5. Use of regression type estimation techniques in other statistical surveys The Lithuanian Labour statistics The Lithuanian Quarterly Survey on Earnings (QSE) is a nationally regulated survey, carried out since The ey variables of the survey are as follows: number of employees, labour cost elements, hours wored and paid. Statistical information is published on a quarterly basis. A simulation study was conducted using the real data of the QSE in order to examine whether the incorporation of auxiliary information at the estimation stage could improve the quality of the results at the regional level. In total, 60 regions exist in Lithuania. Although the regional breadown was of high importance, there was no possibility to incorporate it into stratification procedures, and as a result the Horvitz Thompson estimator did not produce estimates of sufficient quality. Two variables were analysed: the number of full-time employees and the number of part-time employees. Only one auxiliary variable was chosen for further analysis the number of insured persons. The number of full-time employees is highly correlated with the auxiliary variable, while the correlation of the number of part-time employees with the auxiliary variable is quite poor. Auxiliary variables are derived from Sodra database which is under the SSIFB of Lithuania. The coefficients of correlation between the study variables and the auxiliary variable were calculated in each region. In the table below, the minimum, mean, median and maximum values of the coefficients of correlation between the study variables and the auxiliary variable are presented. Coefficient of correlation No. of full-time employees No. of part-time employees Minimum Mean Median Maximum Table 6: Minimum, mean, median and maximum values of the coefficients of correlation of the study variables with the auxiliary variable in regions Relative Root Mean Square Error (RRMSE) is examined as a quality measure of estimators. RRMSEs averaged with respect to the classifications of domains into groups according to the expected sample size are presented in the Tables overleaf: 14

15 Domain sample size Number of domains HT Ratio GREG Total and > Table 7: Average of the relative root mean square error (RRMSE) for the number of full-time employees in domains of different sample sizes (the coefficient of correlation equals 0.996) Domain sample size Number of domains HT Ratio GREG Total and > Table 8: Average of the relative root mean square error (RRMSE) for the number of part-time employees in domains of a different sample size (the coefficient of correlation equals 0.479) As can be seen from the tables above, the incorporation of auxiliary information at the stage of estimation significantly improved the quality of the results in each domain size group. This improvement is extremely significant in the case of a strong correlation between the study variable and the auxiliary variable (see Table 7). In the case of a weaer correlation (see Table 8), this improvement is not so high, but it is still significant. The RRMSE for the HT estimator is higher than for the rest of the estimators. For all estimators, RRMSEs decrease as the domain sample size increases. The analysed estimators are unbiased or almost unbiased, so bias is not discussed in this paper. In the case of a strong correlation between the study variables and auxiliary variables (see Table 7), the GREG estimator produced better results than the ratio estimator. In the smallest domains (with sample size from 1 to 10), the GREG estimator only slightly outperforms the ratio estimator, but as the domains get bigger, the difference between the ratio and the GREG estimator increases. Note that for large domains the GREG estimator is more accurate than the ratio estimator. In the case of a weaer correlation between the study variables and auxiliary variables (see Table 8), the GREG and ratio estimators produced almost the same quality of results. The RRMSE for the GREG estimator is quite similar to that for the ratio estimator in each domain size group. 15

16 Even though the GREG estimator produced in general slightly more accurate estimates compared to the ratio estimator, the ratio estimator was chosen to implement in practice because it is quite simple and easy to interpret, compared to the GREG estimator. The incorporation of auxiliary information at the stage of estimation significantly improved the quality of the results at the regional level. Results of different estimators for each region are presented in the Annex The Italian Labour statistics The Italian Statistical Institute is aimed at producing quarterly estimates on employment for the STS Regulation using incomplete database of Social Security. These administrative data are used in combination with survey data on Large Enterprises. During the last ten years, the Social Security source has been subjected to several administrative changes that have produced relevant effects on the availability of preliminary administrative data, affecting the choice of methodology for the early estimates. Until 2004 less than 70% of administrative data were available on time for production of estimates on employment for STS. Since spring 2004, when after change in legislation over 95% of the reporters became available for the short term deadlines, the preliminary estimation method was radically simplified. Before April 2004 the non-complete administrative data was considered a non-random sample consisting of a big and increasing set of self-selected reporters. This sample showed a good coverage in terms of main target population characteristics and included a fair number of new births. In this information context, in the absence of a design-based framewor, the preliminary estimation was set up using a calibration weighting procedure based on a model-assisted approach. In order to face the challenge to accurately measure the dynamic of employment due to change in the population of enterprises, the method tried to exploit the information contained in the administrative register and cope with its over-coverage problem. For more detailed description of the Italian practice refer to the ESSnet Admin Data deliverable 4.7 of 2010 by C. Baldi and D. Tuzi: THE STS EMPLOYMENT ESTIMATES IN ITALY BASED ON ADMINISTRATIVE DATA A study about the use of GREG-estimators in the Netherlands Statistics Netherlands investigated the use of regression and ratio estimators for monthly turnover estimates in 2005 (Aelen et al., 2005). The aim of this study was to find out whether the sample size of the survey of current month (t) can be reduced by using the GREG estimator instead of the Horwitz-Thompson estimator. VAT-turnover of the previous month is used as auxiliary information when using the GREG-estimator. The main reasons to investigate the use of the GREG estimators for STS-turnover estimates were quite similar to the motivations of Statistics Lithuania. More precisely; 16

17 The availability of VAT is too late to produce monthly estimates with VAT of current period. Hence, a survey for monthly turnover estimates cannot be completely abolished. As an alternative, it was investigated whether the survey can be reduced by 30 % if VATturnover of previous periods can be used as auxiliary variable in the GREG-estimator. Aelen et al. (2005) tested their method on the retail trade. In the Netherlands the retail trade is subdivided in 50 sub-activities. The use of a GREG-estimator by itself does not lead to a higher precision, if such a detailed stratification is used for weighting (see Table 9). The explanation is that the GREG-estimator wors better with bigger samples, because detecting the relationship between VAT-turnover of the previous period and survey turnover of current period is difficult, if the sample size becomes too small. This leads to less precision. Therefore, one of the recommendations of Aelen et al. (2005) was that high stratification levels used for weighting are required if the use of a GREG- estimator is considered (for example NACE 3-digit without application of size classes). Other recommendations included: As the GREG-estimator requires a minimum amount of data (points), it is recommended to impute non-respondents on the survey. These imputations should be based either on a timeseries model or on Y t /Y t-1 growth rates of available respondent, because estimators are based on relations between current and previous period. The survey design can be optimised (implicitly the survey size can be further reduced), by determining the population variance ˆ ( t ) g, S on turnover growth rates over the last 24 months in stead of determining ˆ ( t ) g, S on turnover levels over the last 24 months when using a Neyman allocation to determine the survey design (Cochran, 1977). N g S ˆ, g, ng, n G K N Sˆ g1 1 g, g, Despite these recommendations for improvement, Aelen et al. (2005) concluded that it is difficult to obtain a 30 % survey reduction by using the GREG-estimator instead of the Horwitz-Thompson estimator, if a similar (detailed) publication level with similar output quality should be maintained. The performance of ratio estimators depends on the relationship between VAT-turnover of the previous period and survey turnover of the current period for enterprises with a certain activity. However, results of individual enterprises may deviate from this general relationship due to Differences in definition between VAT-turnover and statistical survey turnover, Possible differences between VAT-units and survey units, Errors in one of the data sources, A growth rate for t-1; t which deviates from the average. As the reasons for deviations may differ (and affect the result), data analyses are quite elaborative to decide whether data-editing should be applied, or observations should be excluded from weighting by outlier assignment. One reason for not applying regression estimators for STS on a large scale in 17

18 the Netherlands was, therefore, the costs of data analyses. The other reason was that the GREG estimator is quite complex and might be considered as a blac box, which is a ris in statistical production. root mean square error (RRMSE) Retail trade total Subactivities (= lower stratification levels) Mean Minimum Maximum Current survey size and current survey design GREG-est. (VAT t-1 ) 0,0035 0, , ,13053 Horwitz-Thompson est. 0,0037 0, , ,17625 Current survey size and optimized survey design GREG-est. (VAT t-1 ) 0,0019 0, , ,09085 Horwitz-Thompson est. 0,0037 0, , ,17625 Reduced survey size (70 % compared to current situation) and optimized survey design GREG-est. (VAT t-1 ) 0,0025 0, , ,11436 Horwitz-Thompson est. 0,0029 0, , ,13632 Table 9: Average of the relative root mean square error (RRMSE) when using a GREG-estimator and Horwitz Thompson estimator for a) the current survey, b) a survey having the same size as the current survey but with a optimised allocation (using turnover growth rate instead of turnover level as population variance) and c) a survey having the 70 % of the current size but with an optimised allocation (using turnover growth rate instead of turnover level as population variance) Study about the use of administrative data from previous period in the UK In the UK, enterprises are expected to report VAT to Her Majesty s Revenue and Customs (HMRC) according to one of 16 schedules. These are monthly (one schedule), quarterly (3 schedules, each with a different set of starting months) and annual (12 schedules, each with a different starting month). Approximately 10% of enterprises report monthly, and 0.2% report using an annual schedule. Parin (2010) tested several methods of converting the quarterly VAT-data into monthly series, but none were found to be superior to simply apportioning the quarterly figures equally into each month covered by the quarter. Using VAT-data reported to an annual schedule has not been explored because the suitability of annual data to produce useful monthly series for short-term statistics is highly questionable, and because so few enterprises report to an annual schedule (Orchard et al., 2011). 18

19 HMRC receives 100% of VAT returns within 188 days of the end of the reference period. Returns accounting for 94% of turnover are received within 40 days of the end of the reference period. Parin (2010) tested various methods of forecasting these complete mature VAT-data, and recommended the use of the Holt-Winters method. The results of this forecasting method are compared with estimation methods using mature VAT-data of previous periods, in combination with survey results of current period. More precisely, following estimators were used to resolve the timeliness problem of late VAT-data for STS-estimates: Yˆ Yˆ t VAT, tx * n i1 n i1 y y survey, t VAT, tx with Y ˆ population estimate of the complete VAT-dataset of previous period t-x VAT, tx y survey, t survey units for current period t yvat, tx VAT-turnover of units in previous period t-x, corresponding with survey units in current period t Common point between ratio and GREG-estimators and the estimator applied by UK is that both combine the survey results of the current period with VAT-turnover data of the previous period. Therefore, it is not surprising that the justification for exploring this approach is similar to the justification for using ratio or GREG-estimators in other countries; viz. reduction of the survey size. For the same reason, it is not surprising that in line with the observations of Statistics Netherland and Statistics Lithuania the UK noted that data analyses and data editing are elaborative when combining survey result of current period with VAT-turnover data from previous periods. To partially alleviate this drawbac, two alternatives for the applied estimators were tested, which are described below. Alternative I: rescaling from reduced sample This method assumes that the sample size for enterprises below the employment threshold is reduced, but not to zero. The sample reduction was carried out by stratum and tested on sample sizes of 5%, 10%, 25%, 50% and 75% of the existing stratum sample sizes, subject to each stratum retaining a minimum sample size of 5. This allows the cleaned VAT-data to be rescaled by a factor comparing the reduced sample turnovers to the VAT-data. The factors were calculated in 3 ways: the median of turnover ratios; the ratio of turnover totals; and the trimmed mean of turnover ratios. These factors were calculated for each combination of stratum (for those strata below the employment threshold) and month. This method was only applied to mature VAT-data from six month before current period. Alternative II: alternative rescaling from reduced sample Alternative II is the same as alternative I, except that the cleaned mature VAT-data is multiplied by two factors. The first factor compares the reduced sample turnovers from the mature data reference period to the mature data (using either the median of turnover ratios, the ratio of turnover totals or 19

20 the trimmed mean of turnover ratios). The second factor is the ratio of the reduced sample turnover estimate for the survey period to the reduced sample turnover estimate for the mature data reference period. The results have been evaluated by using the criteria that the root mean square difference (RMSD) between alternative I, alternative II and final estimates based on complete VAT-data should be less than 5.0 percentage points for 1-month growths (over a period of 24 months). Note that a root mean squared difference of no more than 5.0 percentage points, may at first sight seem large, but it should be regarded in the context of the standard errors of the existing survey at division level. Based on the definition of performing well, alternative I outperforms alternative II, which performs poorly nearly everywhere. For alternative I, the median of turnover ratios generally provides the best factor for rescaling the VAT-data. In general these alternatives for the estimator perform better than simply forecasting of complete VAT-data to the current methods. The project has therefore demonstrated that VAT-data of the previous period can be used to partially reduce surveys for measuring short term (1-month) output growth for some divisions. The downside is the need for intensive data cleaning and data analyses. Therefore, it is a matter of debate whether the forecasting approach is preferred over combining admin data of previous periods with survey results for the smallest enterprises which have in general a small contribution to the total. This is discussed further in the ESSnet AdminData s deliverable 4.3 of Conclusions and remars Many NSIs are faced with an increasing demand for statistical information and at the same time with the imperative to reduce the reporting burden on businesses. The political interest in administrative burden, in general, and statistical response burden, in particular, has never been higher than at the present time. In this situation, statisticians have to consider seriously every possible source of information (first of all the administrative data) to increase the efficiency of the wor. The example of Lithuanian Domestic Trade survey shows that the administrative data can be useful even in the case when they are available too late to produce the Short-Term Statistics estimates. After thorough data and time series analysis, it was decided to use the VAT-turnover of the previous period (t-1) as auxiliary information in regression type estimators. The new methodology has been implemented from 2010 onwards and has provided a significant decrease in the variance of the estimates. Similar testing was carried out at Statistics Netherlands and ONS. It did not result in applying regression estimators for STS on a large scale. Their experience shows that even the well nown method, which wors well in one country, was not chosen in another. It depends on particular circumstances (for example costs, availability of the specialists, on the fact how well the auxiliary information describes the survey variable, e.t.c.). That s why the testing and thorough analysis of the results is extremely important in every single case. 20

21 Considerations stemming out of practical use of regression type estimation technique: 1. GREG estimators wor by taing advantage of various relationships in the data and involve a statistical model to describe these relationships. If the assumed model describes the finite population well, the GREG estimator can potentially bring about a large variance reduction when used in place of the Horvitz-Thompson estimator. The auxiliary data are incorporated in the estimation procedure by an appropriate model and availability of powerful auxiliary information is essential for the estimators considered. 2. Specification of vector of auxiliary information is a challenging tas requiring good modelling sills. Even with only one auxiliary variable, different vectors of auxiliary information can be constructed. Nevertheless one should not forget that GREG is design-based model-assisted and that means that it relies on sample design, and the model is used only as an assisting tool. That means that GREG estimator retains good properties even if the model specification is not exact. 3. The Ratio estimator is quite simple and easy to understand, while the GREG estimator is more complex. Nowadays there is a lot of different software, and it is easy to apply the software and calculate the estimates, but one should avoid running the software without being able to interpret the results, i.e. the statistician needs to have deep understanding of the algorithm and has to be able to analyze the results because otherwise it could lead to unreliable statistics. 4. Often information is needed in domains. Some domains can have too small or even zero sample size. In that case GREG or Ratio estimator is not suitable and small area techniques should be applied. Pros and cons of the use of administrative data in general 1. A mixed-model using sample data and VAT-data of previous period can be a solution of the timeliness problem. The main advantage is that, it allows increasing the accuracy of the estimates and reducing the sample size. 2. Having a sample, even a small one, allows to perceive important changes in the economic cycle and seasonality. No one method could grasp such changes without having data for reference period. 3. One more advantage is also important: a mixed-model (survey + administrative data) maes statisticians less dependent on the owners of administrative data sources in comparison with fully administrative data-based statistics. Some disadvantages of using the administrative data should be mentioned too: 1. The cost saving effect of the use of administrative data instead of statistical data collection seems to be obvious. However, practical experience shows that the idea of having administrative data as a cost-free source for statistical production can be an illusion. Administrative data usually require additional cleaning and possibly matching with other data (statistical or administrative) as well as testing for quality of derived estimates. This can be a challenge for Short-Term Statistics, where the estimates have to be produced in a very short time. 21

22 2. Not all economic entities report VAT monthly, which is very important for monthly surveys. Moreover, the degree of coverage is not nown at the planning stage of the survey. Thus, the survey has to be designed with caution, considering the coverage issues of administrative data. 3. Calculation of more complex estimates requires higher qualification and sills of compilers of statistics. Some remars concerning administrative data editing: 1. Practical wor with administrative records in all participating countries showed that there was a need to invest more resources into data editing and imputation procedures. As regards the Short- Term Statistics, more attention has to be paid to the development of corresponding software because the time devoted to data editing is extremely short. A deep nowledge of an administrative institution, including its primary purpose and the way data are collected and processed can provide a valuable insight into possible sources of errors. 2. It is important to eep in mind that administrative information can be shifted in cases when administrative sources show gains or losses to individuals or businesses. For example, enterprises hardly ever report higher turnover to the taxation institution than to the statistical department. For this reason the turnover reported to statistics should be equal or higher than that derived from VAT data. If an opposite situation is noticed during data editing, it can be presupposed that, either statistical data contains an error or VAT-data does not correspond to statistical data by its definition (for example, an enterprise is reducing its activity and sells its estate, for which VAT was paid). A thorough nowledge of a source will enable to better assess these records. 5. Acnowledgements The authors would lie to acnowledge the advices and discussions of all the countries participating within ESSnet WP4 for they improved the quality of the report. This wor is supported by the Wor Pacage 4: Timeliness of Administrative Sources for Monthly and Quarterly Estimates within the programme ESSnet: Use of Administrative and Accounts Data in Business Statistics, Grant number: SGA References Aelen F, van den Brael J, Ouwehand, J, Methodologie Kortetermijnstatistieen in Golven, internal report Statistics Netherlands, 2005 Bracstone G J (1987). Issues in the use of Administrative Records for Statistical Purposes, Survey Methodology, June, Vol. 13, No. 1, pp Cochran, W G. (1977). Sampling Techniques, 3 rd edition, Wiley and Sons, New Yor. Eurostat (2002). Business Register Recommendations Manual, Chapter 20: The Use of Administrative Sources. December. Hedlin D, Falvey H, Chambers R & Koic P (2001). Does the Model Matter for GREG Estimation? A Business Survey Example. Journal of Official Statistics, Vol. 17, No. 4, pp

23 Hoffmann E (1995). We must use administrative data for official statistics but how should we use them?, Statistical Journal of the United Nations ECE 12(1995) Krapavicaite D & Pliusas A (2005). Imčių teorijios pagrindai. Vilnius: Technia. Orchard C, Moore K, Langford A (2012). National practices of the use of administrative and accounts data in UK Short-term Business Statistics, Deliverable 4.2 of ESSnetADminData SGA2010, Parin N (2010). Interpolation and Extrapolation from Value Added Tax Returns, ONS internal paper, available from the author on request Särndal C, Swensson B & Wretman J (1992). Model Assisted Survey Sampling. New Yor: Springer-Verlag. Sova M G, Broad P J & Orchard C B (2012). On the Potential Use of Administrative VAT-data for Estimating Short-term Output Growth in the UK, Proceedings of Worshop of Baltic-Nordic- Urainian Networ on Survey Statistics, Statistics Finland (2004). Use of Register and Administrative Data Sources for Statistical Purposes, Best Practices of Statistics Finland, Handboos 45, Helsini. 23

24 ANNEX 1 Minimum, maximum and median values of different estimators for number of full-time units by region Expected HT GREG RATIO Region True Coeff of sample ID Value correlation Min Max Median RRMSE, RRMSE, Min Max Median Min Max Median RRMSE, size in % in % in % Total