European Commission. Replacing judgement by statistics: New Consumer Confidence Indicators

Transcription

1 European Commission Directorate General Economic and Financial Affairs Replacing judgement by statistics: New Consumer Confidence Indicators Andreas Reuter Business and consumer surveys and short-term forecast (ECFIN A3.2) EU Workshop on Recent Developments in Business and Consumer Surveys, 14 November 2016

2 Structure (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official EA Consumer Confidence Indicator (CCI): (a) graphical inspection (b) ability to track reference series (c) ability to forecast reference series (iv) Extension: new indicators based on subsets of survey questions (v) Conclusions 2

3 (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical (i) Motivation for design of new indicators current EA CCI: uses 4 out of 12 consumer survey questions arithmetic mean of the four questions' balance series advantages: easy calculation >>easy communication >>easy interpretability changes in indicator attributable to changes in individual questions disadvantages: no statistical foundation (ad-hoc) >>risk of sub-optimal performance (in tracking + forecasting reference series) 3

4 (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical (ii) Construction methods of the new indicators input variables: "rich" set of time-series, namely: balance series of all 11 * cons. survey questions series for 10 EA countries stretching sufficiently far back (to 1985) ( 11 questions * 10 countries = 110 time-series) reference series: EA private consumption growth (y-o-y) * question on appropriateness of saving discarded (insufficient harmonisation) 4

5 (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical aggregation technique: straight-forward solution would be: OLS regression of ref. series on all survey series (algorithm determining questions' weights) OLS estimator fails if number of expl. variables too large for sample size inflated variance of estimated parameters (=inaccurate estimates) if predictors are (near) collinear likelihood of collinearity increasing with number of variables need of genuine "data-reduction" methods to generate confidence indicator Principal Component Analysis (PCA): summarises information in (limited number of) "factors" "factor" reflects tendency shared by several (or all) series "factors" are uncorrelated first "factor" summarises largest share of variables' co-movement Partial Least Squares (PLS): like PCA, but co-variance of input series with reference series is considered Ridge Regression (RR): regularised regression (i.e. imposes thresholds on values of coefficients) works even if more regressors than observations first "factor" = confidence indicator fitted values = confidence 5 indicator

6 (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical some technical information: confidence indicator constructed in pseudo real-time, i.e. to calculate value for January 2015, only data released until January 2015 may be used >>practical implementation via blocking approach survey series expressed as first differences (given their non-stationarity) confidence indicator constructed for period to ensure that indicator quality does not vary over time, we use a rolling in-sample window of 36 quarters for the calculation 6

7 (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical (iii) Comparison of new indicators & official EA Consumer Confidence Indicator (CCI) (a) graphical inspection indicators very similar! 65 current CCI PCA-based PLS-based RR-based (b) tracking performance: correlation with EA priv. cons. growth (y-o-y) coincident: 1-Q-lead: current CCI: PCA-based: PLS-based: RR-based: no big differences in tracking performance!

8 (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical (c) ability to forecast reference series expansions & contractions in EA private consumption questions: Do new cof. indicators contain information additional to that contained in current CCI? If yes, is supplementary info complementary to timely released hard data? set-up: for every cof. indicator, run two probit-models predicting recession probability: restricted model: predictors are cof. indicator (+ constant) augmented model: predictors are cof. indicator (+ timely available hard data) short-term int. rates out-of-sample period: 2005q2 to 2015q1 EuroStoxx 50 pseudo-real time set-up with assumption that EA HICP forecast of q t is conducted at end of M3 of q t

9 (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical How to visualise the results? predicted recession probabilities rarely (exactly) 0 or 1 >>standard solution: adopting cut-off value beyond which predicted probabilities are interpreted as recessions downside: choice of cut-off value has bearing on which model performs best solution: visual inspection of receiver operating characteristic (ROC) curves: for different cut-off values (0 to 1), plot: false positives rate (x-axis) vs. true positives rate (y-axis) good models tend to lie above 45-degree line (= higher true positives than false positives rate)

10 RR-based: PLS-based: PCAbased: current CCI: (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical restr. models: aug. models: conclusions: models tend to be above 45-degrees line all cof. indicators contain forecastrelevant info, no matter which cut-off value is used current CCI carries least forecast info hard-data improves models current-cci model NOT inferiour to other models statistical test if new cof.-models are better than current-cci-model: >>restricted models: new cof's make better forecasts >>augmented models: new cof's do NOT make better forecasts new cof's have (a bit) more forecastingrelevant info but: new info largely covered by timelyreleased hard data No added value of new cof's in realistic forecasting scenarios!

11 (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical inspection (b) ability to track ref. series (c) ability to forecast ref. series (v) Extension: New indicators based on subsets of survey questions (vi) Conclusions (iv) Extension: new indicators (PCA-/PLS-/RR-based) based on subsets of survey questions: (a) motivation: new indicators not producing superiour forecasts in realistic forecast scenario (i.e. if hard data are included) stat. efficiency decreases if many series with little info reg. target are combined Can we produce new cof. indicators which are truly complementary to hard data when using less survey questions? choice of question subsets based on: objective characteristics with likely bearing on complementarity with hard data expectation questions: logic: particularly beneficial for forecasting capture dimension largely absent from hard data assumption: expectation-based new cof. indicators outperform current CCI superiour performance also if hard data are included micro questions reg. household's situation: logic: micro-question measures dimension largely absent from hard data assumption: unclear if outperforming current CCI, but added value likely to persist when hard data are added (more complementarity)

12 (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical inspection (b) ability to track ref. series (c) ability to forecast ref. series (v) Extension: New indicators based on subsets of survey questions (vi) Conclusions (b) findings: new cof. indicators based on expectation questions: in restricted model: outperforming current CCI in (hard data) augmented model: statistically not better than current CCI new cof. indicators based on "micro"-questions: in augmented model: statistically better than current CCI however: in restricted model: no better performance than current CCI 1. Using only "micro" questions does not produce a better cof. indicator as such. 2. But as assumed - better complementarity of "micro" questions with hard data!!!

13 (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical inspection (b) ability to track ref. series (c) ability to forecast ref. series (v) Extension: New indicators based on subsets of survey questions (vi) Conclusions (c) visual inspection of "micro"-question-based cof. indicators: current CCI PCA-based PLS-based RR-based observations: no big differences between PCA-/PLS-/RR-based indicators new cof. indicators in lockstep with current CCI, except for last 2 crises: current CCI has lowest trough in financial crisis micro-based new cof. indicators have lowest trough in sovereign debt crisis Larger response of "micro"-based cof. indicators to sovereign debt crisis makes sense, considering most austerity measures took place in 2013/14.

14 (i) Motivation for design of new indicators (ii) Construction methods of new indicators (iii) Comparison of new indicators & official CCI: (a) graphical inspection (b) ability to track ref. series (c) ability to forecast ref. series (v) Extension: New indicators based on subsets of survey questions (vi) Conclusions (v) Conclusions current CCI has potential shortcomings: ad-hoc aggregation method just 4 input questions not tailored to target-series by design we addressed all shortcomings: PCA/PLS/RR 110 time-series (PLS/RR): tailored to target-series results: only slight improvements in tracking private consumption growth only slight improvements in forecasting expansions/contractions in private consumption + all improvements fading in realistic forecasting scenario (i.e. when timely hard data are included) interpretation: more complicated cof. indicators no compelling alternative to current CCI more generally: researchers interested in forecasting private consumption dynamics should focus on "micro" questions from the consumer survey 14