31 August Energy Consumption and the Effects of Energy Efficiency Measures Based on Analysis of NEED Data DECC

Transcription

1 31 August 2011 Energy Consumption and the Effects of Energy Efficiency Measures Based on Analysis of NEED Data DECC

2 Project Team Daniel Radov Dr Orjan Sandewall Martina Lindovska Samuel Brown Dr Kenneth Train Dr Gregory Leonard NERA Economic Consulting 15 Stratford Place London W1C 1BE United Kingdom Tel: Fax:

3 Contents Contents Executive Summary i 1. Introduction Introduction to NEED Dataset Project Background 1 2. Methodological Overview Conditional Demand Models & Treatment Effects Household Characteristics and Energy Demand 7 3. NEED: A Description of the Dataset Data and Data Sources Basic Descriptive Statistics Simple Univariate Analysis of Treatment Effects In-year Analysis Across-Year Analysis: Simple Difference in Differences Relationship between Future Treatment and Energy Use Summary Panel Analysis of Treatment Effects Panel Specifications Confidence intervals Other Sensitivities Summary and Concluding Remarks Non-Parametric Analysis Overview Sample and Treatment Density Consumption Treatment Effects Relationship Between Household Characteristics and Energy Consumption Overview Motivation of Specifications Considered Demand Results Two-Stage Estimation Alternative Specifications Other variants 99 NERA Economic Consulting

4 Contents 7.6. Summary Summary and Conclusions Treatment Effects Derived from NEED Data Modelling Household Energy Consumption Suggestions for Further Research: Enhancing the NEED database 108 Appendix A. Treatment Density 112 Appendix B. Treatment Effects, Non-Parametric Estimates 116 Appendix C. Results for Non-MECE Panel Estimations 120 Appendix D. Explanatory Power of Household Characteristic Data 125 Appendix E. Comparison of VOA Sub-Samples 129 Appendix F. Investigation of Electric Heating 133 NERA Economic Consulting

5 List of Tables List of Tables Table 3.1 Numbers of Selected Measures Installed in NEED (thousands), Table 5.1 Panel Regression Results for Gas Consumption in Logarithms, Without Income Interaction Effects 58 Table 5.2 Panel Regression Results for Gas Consumption in Levels, Without Income Interaction Effects 59 Table 5.3 Panel Regression Results for Gas Consumption in Logarithms, Including Income Interaction Effects 67 Table 5.4 Panel Regression Results for Gas Consumption in Levels, Including Income Interaction Effects 68 Table 7.1 Results of Two-Stage Demand Estimation Regression (Levels) 93 Table 7.2 Alternative Specifications OLS and Random Effects vs. Two-Stage 98 NERA Economic Consulting

6 List of Figures List of Figures Figure 3.1 Biased Estimates of Treatment Effect when Treatment is Not Observed 16 Figure 3.2 Share of Properties with Gas Consumption ( ) 18 Figure 3.3 Gas Consumption 19 Figure 3.4 Electricity Consumption, Figure 3.5 Electricity Consumption, Excluding Gas Users 22 Figure 3.6 Average Gas Consumption, 2007, by Type of Household 24 Figure 3.7 Distribution of Measures, by Household Characteristics 30 Figure 3.8 Energy Prices and Consumption, Figure 4.1 Univariate Results, Treatment Effects Figure 4.2 Simple Treatment Effect, by Household Characteristics (from Experian), Figure 4.3 Difference in Differences Treatment Effects, , Full Sample 42 Figure 4.4 Treatment Effects - Impact on Gas Consumption, Figure 4.5 Treatment Effects: Cavity Wall Insulation 47 Figure 4.6 Comparison of Pre-Treatment and Never Treated Gas Consumption, Figure 4.7 Treatment Effects, "Simple" Estimates, Figure 5.1 Confidence Intervals for Treatment Effects (kwh) 64 Figure 5.2 Confidence Intervals for Treatment Effects (%) 65 Figure 5.3 Relationship of Income to Treatment Effect, Full Sample ( ) 69 Figure 5.4 Relationship of Income to Treatment Effect, Three-Bed Semis ( ) 69 Figure 5.5 Summary Panel Treatment Effects Absolute Savings (kwh) 71 Figure 5.6 Summary Panel Treatment Effects Relative Savings (%) 72 Figure 6.1 Definition of Bins for Non-Parametric Analysis 73 Figure 6.2 Density of Population, 2007 (number of observations) 74 Figure 6.3 Density of CWI Treatments (2007) 75 Figure 6.4 Gas Consumption by Bin, 2007 (kwh) 76 Figure 6.5 Treatment Effect of Cavity Wall Insulation Only, by Bin, 2007 (kwh) 77 Figure 6.6 Treatment Effect of Cavity Wall Insulation Only, by Bin, 2007, Excluding Bins with <100 Treatments (kwh) 78 Figure 6.7 Relative Treatment Effect of Cavity Wall Insulation Only, by Bin, 2007 (%) 79 Figure 6.8 Energy Savings by Measure(s), Population Average, (kwh) 80 Figure 6.9 Alternative Presentation of Non-Parametric Treatment Effects, Overall Population, , (kwh) 81 Figure 6.10 Energy Savings by Measure(s), 3-Bed Semis (Experian data), (kwh) 83 Figure 6.11 Alternative Presentation of Non-Parametric Treatment Effects, 3-Bed Semis (Experian data), , (kwh) 84 Figure 6.12 Energy Savings by Measure(s), Priority Group, (kwh) 85 Figure 6.13 Alternative Presentation of Non-Parametric Treatment Effects, Priority Group, , (kwh) 86 Figure 6.14 Energy Savings by Measure(s), Higher Income Households, (kwh) 87 Figure 6.15 Alternative Presentation of Non-Parametric Treatment Effects, Higher Income Households, , (kwh) 88 Figure 8.1 Summary Treatment Effects, Full Sample 103 Figure 8.2 Summary Treatment Effects, 3-Bed Semis 105 NERA Economic Consulting

7 Executive Summary Executive Summary Introduction and Background The Department of Energy and Climate Change (DECC) has recently developed the National Energy Efficiency Data-framework ( NEED ), a large database containing detailed information about household energy consumption and household characteristics. This dataset has been developed to make possible in-depth analysis of household energy use to inform and evaluate policy analysis. 1 The dataset is large (approximately four million households energy consumption over five years) and rich (with information about gas and electricity use, building form, household location at output-area level, energy efficiency measures installed, and household demographic information), although there are various questions about the data that we discuss in the main report. This report uses the NEED dataset to estimate the impacts of a range of energy-efficiency measures on household energy consumption, and to estimate the influence of household characteristics on energy consumption, using a range of econometric techniques that take advantage of the panel nature of the dataset. After summarising and discussing the features of the data, we consider a variety of ways to estimate the impacts of energy efficiency measures, and then present models of overall energy demand. Our analysis is relevant to a wide range of past, present, and future government policies and regulations, including the EEC, CERT, Warm Front, Renewable Heat Incentive, Building Regulations, and the Green Deal. It will help contribute to a more robust evidence base for assessing the impacts of energy saving measures, and in understanding household energy demand more broadly. The Effects of Energy Efficiency Measures We apply a variety of approaches to estimate the effect of three energy efficiency measures that have played an important part in the UK s recent efforts to reduce household energy consumption: loft insulation, cavity wall insulation, and efficient new gas boilers. We also analyse the impacts of combinations of these three measures. We find that the energy efficiency measures analysed deliver savings significantly below the values used in recent DECC policy analysis, such as the Impact Assessment for the CERT Extension. 2 The two main methods for estimating treatment effects used in this study are fixed effects panel and non-parametric analyses. The fixed effects panel regression is a standard approach to estimating treatment effects, whereas the non-parametric approach, which is based on a simpler year-by-year comparison of consumption levels in treated and non- 1 2 In June DECC published a Report on the development of the data-framework and initial analysis. DECC s report covers some of the areas covered in this current report, although much of the analysis in the current report was undertaken prior to publication of DECC s report. The two reports were produced independently and apply different techniques. The results are generally consistent with each other, although our preferred approaches to estimating treatment effects make use of more of the dataset, and yield lower estimates of energy savings. NERA Economic Consulting i

8 Executive Summary treated households, controls better for the fact that efficiency measures may affect different types of households in different ways. 3 A summary of our estimates of the impact on gas consumption of the efficiency measures analysed here is shown below. The Figure ES.1 compares the estimates from the current analysis to theoretical values developed by DECC to estimate the savings from the extension of the Carbon Emissions Reduction Target ( CERT ), which are based primarily on engineering estimates of the effectiveness of the measures. 4 Figure ES.1 Summary Treatment Effects, Full Sample 4,000 3,500 3,000 2,500 2,000 1,500 1, Std PG Std PG Std PG Treatment effect (kwh) CERT NEED CERT NEED CERT NEED. NEED. NEED. NEED. NEED CWI Loft Condensing boilercwi + Loft Loft + C. Boiler CERT Estimate, standard CERT Estimate, Super-PG NP, Full Panel, Full Panel, PG CWI + C. Boiler All Measures Notes: 1. Panel results are not year-specific so the treatment effect estimates are shown as horizontal lines. 2. CERT values are based on the theoretical savings estimates of measures used in producing the Carbon Emissions Reduction Target extension impact assessment. CERT Std or "CERT Standard" estimates are for non-priority Group households and for Priority Group households; "Super-PG" refers to CERT Super Priority Group households (which have the lowest incomes). 3. NP, Full" is the non-parametric analysis using the entire NEED database. "Panel, Full" refers to the entire sample / population, whereas "Panel, PG" refers to the sub-sample of households classified as Priority Group, based on NEED estimated annual household income below 20,000. DECC s estimates based on the CERT Extension (which are available only for the measures on their own, not in combination) are shown as the blue bars, and the results of our analysis 3 4 In addition to these two main approaches, we tested a number of others. We also considered specifications based on the logarithm of energy consumption instead of consumption itself. Using the logarithm gives estimates of the treatment effect that can be interpreted in terms of the relative reduction in energy use, rather than the absolute reduction. These theoretical CERT savings differ from those used in the current Standard Assessment Procedure ( SAP ). NERA Economic Consulting ii

9 Executive Summary are shown as horizontal lines (for panel model estimates) and as points (for the nonparametric analyses in each year). Across all the estimation methods that we used, the relative magnitudes of energy saving across different measures confirmed that cavity wall insulation has the greatest impact, followed by condensing boilers, and then loft insulation. This is generally consistent with DECC s own estimated savings, although the CERT Extension savings for condensing boilers for Super Priority Group households depart from this general rule. In all cases, the estimated energy savings based on NEED data are well below the levels derived from the theoretical CERT savings. Cavity wall insulation: The panel and non-parametric results are clustered around 1,500 kwh for the full NEED sample. The CERT estimate for the non-priority Group is double that. The difference is similar for the Priority Group (which we expect to have a lower treatment effect than the full population, both because its base consumption level is lower, and because we expect them to be more prone to comfort-taking 5 ): the CERT savings estimate is around 2,100 kwh, or nearly 80 percent higher than our estimate of 1,200 kwh. Loft insulation: The results from our two preferred econometric approaches using NEED are somewhat less consistent with each other: the panel model suggests savings on the order of 450 kwh, whereas the non-parametric approach suggests savings of just 100 kwh over all years. Again, these results are significantly below DECC s CERT estimates, which are 1,000 kwh for the Super Priority Group and 1,500 kwh for others. As we discuss in the report, some of this difference may be due to the fact that a significant proportion of UK installations of loft insulation are not captured by the NEED database, because these installations are done by householders themselves or by smaller independent contractors and therefore not recorded by the Energy Saving Trust, which maintains the underlying HEED database. 6 We estimate, however, that these missing observations referred to as hidden treatments could bias our panel estimates down by percent. This would suggest an adjusted treatment effect closer to 600 kwh still well below the CERT Impact Assessment value. Condensing boilers: The NEED-based estimates for the full sample are clustered between 1,200-1,300 kwh from , with the non-parametric estimates lower in 2004 and As with the other efficiency measures, these estimates are much lower than the CERT value, which is 2,600 kwh irrespective of Priority Group status. Again, even if we attempted to adjust for potential hidden treatment bias in the installation of condensing boilers, the impact of condensing boilers would be well below the CERT estimate. 5 6 Comfort-taking is also often referred to as the rebound-effect, and reflects the fact that households will often increase the internal temperature of their homes (or more generally the level of energy services consumed) after adopting more efficient technologies, because the marginal cost of internal temperature and other energy services declines the more efficiently energy is used. DECC assumes 40 percent comfort taking for the Super Priority Group and 15 percent for others for insulation measures, but not for boilers. DECC s June 2011 report on NEED presents one estimate of the impact these hidden treatments have on estimates of the savings of observed measures; we discuss our own estimates of their impact in the main report, below. NERA Economic Consulting iii

10 Executive Summary Combinations of measures: We do not have any explicit corresponding technical savings estimates based on DECC assumptions. The results from our two preferred methods are generally consistent with each other in terms of the relative magnitudes of the results (although for the combination of loft insulation and condensing boilers the non-parametric results become less consistent, particularly for the years , due to the relatively small number of observations of households with combinations of treatments). We find that estimates of the relative impact of energy efficiency measures (using model specifications based on logs of energy consumption, rather than consumption levels) give similar results, although the estimates are somewhat less statistically significant. Changes over time: We also find a significant underlying time trend showing reduced energy consumption that is not explained by the observed adoption of energy efficiency measures. These trends reflect both higher prices over the period as well as other energy efficiency improvements that are not observed including the uptake of hidden loft insulation and condensing boilers. The effect of other energy efficiency improvements accounts for more of the declining trend than does price: changes in price account for average annual reductions of around 100 kwh over the period, whereas the residual time trend is closer to a reduction of 500 kwh per year. Heterogeneity of impacts: We find that the treatment effects themselves differ by household type. The non-parametric analysis provides treatment effect estimates for 216 different categories of household, although there are many more categories that could be created using the data set. 7 Households that adopt energy efficiency measures also differ significantly from the overall population for example, they tend to be poorer because of the targeting of CERT. One of the primary motivations of the non-parametric analysis is to correct the estimate of the average treatment effect to reflect some of these differences. 8 As more households adopt energy efficiency measures, it will become increasingly important to take into account the differences between the average household and those that have not yet taken up measures, because they may have characteristics very different from the average. Modelling Household Energy Consumption We estimated models of household energy demand using a range of specifications, and including or excluding different variables to test their ability to explain differences in consumption. For this analysis we supplemented the NEED dataset with regional price data, as well as information about the type of electricity meter installed in the household (which may be related to whether or not the dwelling is heated primarily using electricity). Our preferred approach uses a two-stage estimation that makes use of the fixed-effects panel estimates of treatment effects in the first stage, and then estimates the effects of household characteristics in the second phase. This makes full use of the panel nature of the data where 7 8 The categories were selected to analyse the impacts of a wide range of combinations of different household characteristics while still retaining a manageable number of distinct household types. Even though the number of treatments within each household category are not always large enough to draw robust conclusions about the treatment effect results for individual household categories, the approach provides a more accurate way of calculating the overall average treatment effect across the entire population. NERA Economic Consulting iv

11 Executive Summary it can be used, and then considers the impact of household characteristics which do not vary over time in the dataset. 9 Using this approach we are able to explain between percent of the variation in gas consumption among households. It would be possible to increase this value using more complicated model specification with more variables, although this would mean using a less transparent model that would make it more difficult to develop an intuitive understanding of the underlying relationships. These results compare well to other attempts to estimate gas consumption (see, for example, Meier & Rehdanz (2010), who achieve R-squared levels of around 20 percent for their full sample, for energy or gas expenditure, using a significantly smaller panel dataset that contains much more detailed household information). As for the analysis of treatment effects, for our consumption analysis we assessed models whose dependent variable is the raw energy consumption level as well as the natural logarithm of energy consumption. In general, the log specifications do not fit the consumption data as well as those where the dependent variable is simply the energy consumption. This is somewhat counterintuitive, because we might expect that, for example, consumption would increase not only with property size and with income separately and independently, but that there might also be some interaction between them. The log specification allows for such interaction (because the terms in the regression equation are all multiplied by each other, rather than being added together). But it appears that the multiplicative interaction represented by the logarithmic consumption equation is not a very good representation of reality. Building characteristics: The single most important determinant of household gas consumption is floor area. A typical property might have annual gas consumption on the order of 17,000 kwh, and a floor area of around 90 m 2. Our model suggests that floor area alone accounts for consumption of around 10,000 kwh annually, implying that floor area explains well over half of the overall demand. The constant term accounts for around 7,500 kwh of consumption, with other household characteristics modifying the value in both directions. Consumption also increases with the number of rooms (even controlling for floor area). As expected, older properties have higher consumption than newer properties and the effect is most pronounced with the newest properties built since the 1980s (which have average consumption more than 4,000 kwh below that of otherwise similar properties). Finally, the geographic location of properties also has quite a large impact on expected consumption, controlling for other household characteristics with properties in the North showing higher consumption than similar properties in the South, with the exception of London, which also has higher consumption levels. Demographic characteristics: Demographic characteristics also have a significant impact, although the underlying demographic data are of uncertain quality for various reasons, so results based on them should be understood to reflect correlations in the data, rather than any causal relationship. Controlling for all other variables, for every increase in income of 9 As we discuss in the main report, the NEED dataset contains household characteristics corresponding to only a single year, even though its consumption information spans five years. NERA Economic Consulting v

12 Executive Summary 10,000, gas consumption is on average more than 450 kwh higher (or 380 kwh for the NEED Priority Group). 10 Consumption increases with the number of adults in a household, by around 750 kwh for Priority Group households, or up to around 875 kwh for the full sample. Having a female head of household is associated with a slightly higher consumption (around 300 kwh for the full sample, 500 kwh for the Priority Group) than a male head of household, which may reflect occupancy patterns among such households, but may also be an artefact of how the data were developed. Children are also associated with higher levels of consumption, but only kwh. Households in rented accommodation tend to use less gas than owner occupiers, even accounting for other factors. Finally, the age of the head of household does appear to have some effect on the consumption of the household, although these effects are among the weakest that we estimate. Significantly, Priority Group households with older residents (above age 56) appear to have the most significantly lower consumption relative to typical consumption levels (500 to 800 kwh lower). It is worth emphasizing that the demographic data are expected to be among the least reliable in NEED, and this measurement error will reduce the apparent magnitude of these factors in the estimation. We have been able to analyse the impact that more reliable demographic information has on the estimates, and we find that when more reliable data are used, it does tend to increase the magnitudes of the estimates. 11 Electricity demand: Unfortunately, the model is less successful explaining variation in electricity demand than variation in gas demand. It is likely that an important reason for this is that we do not have a reliable way of identifying households that are electrically heated. In addition to the estimates of individual coefficients, we also investigated the value of the separate datasets that have been used to create NEED. We find that the data from Experian perform very similarly to data from the Valuation Office ncy when the VOA data are limited to the type of information available in the Experian dataset such as number of bedrooms, building form, or building age. However, when the full range of VOA variables is used, the VOA data are able to explain significantly more of the variation in the data than the Experian dataset is able to explain on its own. Suggestions for Further Research The current work is only the first step in what we believe could be a very fruitful set of analyses of the NEED dataset in its current and future incarnations. We summarise here a wide range of possible future research areas that could be explored. Some of these relate to areas where the NEED database could be enhanced, augmented, or improved to increase its power to provide meaningful input to policy questions. Others relate to further econometric analysis. And others would take the data and results and apply them to other policy questions It is important to note that this finding controls for the other variables, which may not be entirely appropriate or meaningful: in the real world, demographic factors such as income have a very significant influence on where people live and the size of their homes. If one were developing a comprehensive model of consumption that endogenised dwelling choice as well as energy consumption, therefore, it would not be appropriate to vary income controlling for dwelling size. In such a model, the coefficient on income would most likely be much larger. The more reliable observations are actually a subset of the full NEED dataset, which we identify as more reliable using other variables in the dataset. NERA Economic Consulting vi

13 Executive Summary Reporting year: Gas consumption is reported for the gas year, whereas electricity consumption is reported for the calendar year. Although this is not a major concern in understanding treatment effects, it does complicate any analysis that seeks to relate electricity consumption to gas consumption. Weather correction: Gas and electricity data also differ in that gas consumption is weather-corrected, whereas electricity consumption is not. They cannot therefore be compared directly, and their relationship (e.g. as substitutes, or as complements in a cold winter) cannot be investigated very well. Including additional data to make them comparable would be very useful. Adding weather and weather-corrected and uncorrected consumption information (ideally at the regional level) would help to understand actual demand levels better, improve estimates of price impacts, and improve the understanding of the relationship of gas consumption to electricity consumption. It could also help to explain differences in rates of uptake. Although these are not focuses of the current report, they are relevant and may be the focus of future work. Building fabric: NEED does not include information about the nature of a property s building fabric. For example, it does not include information about whether a property has cavity wall or solid wall construction, which makes it difficult to use it to get a clear picture of how much potential remains for the appropriate insulation technologies. It also complicates the definition of the relevant control group for different treatments. The EST s HEED database does appear to contain information related to building fabric, but we do not know how accurate this information is. Hidden treatments: The share of properties with hidden treatments that are not reflected in NEED should be established. This could be done through a survey of a representative sample of NEED households. Even without such primary data gathering, it would be possible to improve the dataset by supplementing it with annual information collected by Ofgem about the total aggregated national penetration of efficiency measures due to energy efficiency policies. This would make it possible to account for more of the currently unexplained annual reductions in average energy demand. Quality and value of household characteristic data: The quality of some of the data currently used in NEED is uncertain. At a minimum, it seems important to gather household demographic data that actually pertains to the energy consumption years of interest (current the data are for 2010 only, but even so, they do appear to have some explanatory power). Additional data filtering: There is more that could be done to improve the dataset to eliminate outliers and suspect values: for example, analyzing the variance within households and considering whether households with very large fluctuations should be included and if so, under what conditions. Improved energy price variables. It would be possible to improve upon the price variable, for example, by incorporating information about the share of different contracts prevalent in particular regions, and using this to develop weighted average prices, or even to interact prices for different contracts with different income groups. NERA Economic Consulting vii

14 Executive Summary Use of engineering values: the current work has estimated treatment effects and demand using relatively simple binary variables indicating the presence or absence of an energy efficiency measure. These binary variables obscure variation in the more fundamental physical attributes of buildings (such as U-values, the thickness of loft insulation, or area of external walls), which could be used instead. Variables representing these physical attributes could be developed (with varying degrees of approximation and associated effort). Such estimates could be useful if DECC s aim was to try to test the assumptions in physical models, for example, or even to investigate the extent to which building regulations were actually being adhered to. Use of detailed geographical information: It would be possible to use the detailed geographical information to develop additional variables, based on other existing datasets, that would supplement or even replace the Experian demographic data currently in use. Analysis of agreement between underlying NEED data sources: The VOA and Experian data sets overlap for a number of building characteristics, and it seems worthwhile to test the agreement between the datasets. Analysis of incremental benefits of variables / datasets: In Appendix D we present the incremental benefits of adding progressively more detailed household characteristic data to the econometric model. There is more that could be done here, but it would first be important to understand what options DECC may be considering in future uses of NEED. Identification and analysis of electric heating: NEED does not currently contain information that makes it possible to identify which properties are electrically heated. This makes it far more difficult to model electricity demand, and also (to a lesser extent) complicates the estimation of the gas demand model. There are a variety of ways that electric heating could be investigated in more depth, for example: Making greater use of the detailed geographical data to develop a variable that would indicate whether a dwelling was in an area where there was no (or limited) gas grid. Stratifying no-gas households into electricity consumption bands and estimating demand models separately for each band, to try to identify the influence of household characteristics differ in ways that could be explained by the presence or absence of electric heating. Interacting meter profile with other variables. Analysis of additional treatment effects: There are other measures of significant interest to DECC that we expect will be considered in subsequent work, including solid wall insulation, solar thermal technologies, heat pumps, etc. Similarly, there are also other treatment effects that could be investigated using the existing data, such as the impact of glazing measures, or the additional electricity consumption associated with condensing boilers (which use non-negligible amounts of electricity to run the condenser). Finally, it has not been possible to analyse treatment effects in flats because the measures data for flats are unreliable. When the data are improved the analysis could be extended to them. Analysis of treatment effects for additional sub-populations: We present results for the treatment effects of measures and combinations of measures for the entire population, NERA Economic Consulting viii

15 Executive Summary and for a large number of sub-populations. It would be possible to provide estimates for a wider range of sub-populations using either the simple non-parametric approach or the panel fixed effects approach. It would also be possible to estimate the fixed effects panel model restricted to installations of measures in different years, to investigate potential changes over time. More complicated representation of energy demand: In specifying the demand model, there is a trade-off between adding a large number of variables to explain more of the variation in the sample, on the one hand, and maintaining a parsimonious, meaningful, and understandable representation of the demand relationship, on the other. It would be very difficult to interpret the coefficients of models with a large number of variables (including interactions between categories), however. Nevertheless, depending on DECC s priorities it may be worthwhile to develop demand specifications that would more precisely differentiate between the characteristics of different households, and achieve a better fit while remaining parsimonious enough to be understandable and useful for explanation and policy analysis. Development of household simulation tool: Based on the current work (or refinements described above) it would be possible to develop a tool that would take individual household input characteristics (or sets of characteristics) and calculate the estimated demand and/or estimated treatment effects for the household. Aggregate demand simulation model: The underlying demand equations developed here could relatively easily be built up into a simulation model that would calculate the household energy demand in England under different assumptions about the development of the housing stock including energy efficiency measures and other underlying household demographic characteristics such as income and family size. This would be a potentially very useful tool that could perform some of the functions currently envisaged for the new UK national household energy demand model. Analysis of the determinants of uptake: Our analysis demonstrates that the rate of uptake and of penetration of measures across different types of households is not constant, and that households that take up measures are different from those that do not even when all observed characteristics are accounted for. It would be possible to use the NEED dataset to develop uptake models of varying degrees of complexity. The most significant difficulty with using NEED or the augmented dataset for this purpose is that it does not currently contain cost information about the relevant measures. As a start it would be possible to populate variables for the costs of different measures in different regions and for different types of dwelling. It would however require considerable analysis to develop a methodologically sound way of using these synthetic data to draw conclusions about uptake, although it seems likely that such analysis would provide useful insights that could be used for policy design. Analysis of regional variation: The dataset has very detailed geographical information that could be used to a much greater extent. This could provide input to local authorities or others who may be interested in understanding more about energy consumption in areas they are responsible for. NERA Economic Consulting ix

16 Executive Summary NERA Economic Consulting x

17 Introduction 1. Introduction 1.1. Introduction to NEED Dataset The UK Department for Energy and Climate Change has recently developed the National Energy Efficiency Data-framework (NEED) dataset to help improve its understanding of, and ability to analyse, household energy consumption. NEED contains detailed energy consumption data for a subset of approximately four million households in England, or roughly 18 percent of English households, along with information about the physical characteristics of the dwelling, data about the demographic characteristics of the residents, and information about specific energy efficiency measures of interest to DECC. The expectation is that NEED will be expanded and improved over time, to yield a rich dataset that can be used for DECC s research and policy analysis Project Background The project for which this report is written has two related aims. The first is to estimate the impact of specific household energy efficiency measures on household energy consumption, using the NEED dataset. The second is to estimate a more general model of household energy demand, using the characteristics data contained in NEED. The results of the first analysis information on the effectiveness of energy efficiency measures in different types of household will be used to help DECC and the UK Government as it prepares the details of the new Green Deal policy to promote energy efficiency and other measures to reduce energy use and carbon dioxide emissions by households. The estimates developed from this analysis will be compared to DECC s existing estimates of how effective the relevant energy efficiency measures are, to better assess the impacts of the Green Deal, and to understand the financial implications for consumers of measures that are now, and that will in the future, be the focus of the policy. The measures that DECC has identified as of particular interest for this initial analysis of the NEED dataset include: Cavity wall insulation ( CWI ) Loft insulation CWI + loft insulation CWI + new (condensing) boiler New (condensing) boiler + replacement of electric heating with wet heating system. In addition to estimates of the average energy saving benefits of these measures (and combinations) across the population, DECC has also expressed interest in distinguishing the impacts on specific categories of households. These include: Households in the Priority Group (assumed to be lower income households) and outside it; NERA Economic Consulting 1

18 Introduction Income categories more generally; Different household tenures including: Properties that are owner-occupied, Properties that are privately rented, Council housing or housing association properties. DECC has also expressed an interest in whether it is possible from the NEED data to identify any time-dependence of the treatment effects for example, reduced impacts in treated households over time, or changes in the treatment effects of households taking up the measures in successive years. The results of the second analysis estimating a model of household energy demand will be used in a variety of areas of DECC s policy work. The remainder of this report is organised as follows. Chapter 2 presents a brief overview of the econometric literature on estimating energy demand and treatment effects. Chapter 3 reviews the NEED dataset in detail, and presents other information about energy consumption and prices in the UK, and the energy efficiency measures and policies relevant to the study. Chapters 4-6 present alternative approaches to estimating the effectiveness of the energy efficiency measures identified by DECC for examination here. Chapter 4 presents simple estimates of treatment effects based on the overall sample and investigates whether the effect appears to vary with individual household characteristic variables. Chapter 5 presents panel regression estimates of the treatment effects. Chapter 6 presents a highly disaggregated nonparametric approach to estimating the treatment effects. Chapter 7 presents the results of our analysis of household energy consumption using the NEED dataset. Chapter 8 concludes, providing summary and discussion, and comparing our results to DECC s own estimates (independent of the data contained in NEED) of the treatment effects for a selection of the measures of interest. The final chapter also provides suggestions for improving and analysing further the NEED dataset. NERA Economic Consulting 2

19 Methodological Overview 2. Methodological Overview This section provides a brief overview of potential ways to overcome or mitigate key empirical issues relating to the estimation of energy demand. We also discuss our estimation approach and its motivation. In this interim report, we focus on estimating the expected energy saving from a selection of key energy efficiency measures. Ultimately, however, we also want to characterise the quantity of energy consumed by UK households as a function of energy prices and household characteristics (including the efficiency measures) that affect energy demand. An important consideration for the full demand model will be that price data appear to be very limited. To the extent that price data are available, they are aggregated over a range of retail tariffs and over the year. This presents problems for estimating price effects that will need to be addressed for full demand estimates, and that may prevent meaningful elasticity estimation Conditional Demand Models & Treatment Effects Key empirical issues In estimating the relationship of energy demand to efficiency measures and household characteristics, we draw on the literature of so-called conditional demand studies. These studies in turn draw on the treatment effects or programme evaluation literature, which speaks to the question of what is the expected change in energy consumption for a household randomly selected from the population that implements measure X. In this type of investigation, there are two primary methodological concerns). First, the likelihood of being treated (i.e. of adopting a given efficiency measure) may depend upon observable or unobservable factors which are also directly related to household energy consumption. If we observe, for example, that dwellings in the North of England have higher uptake of energy-efficiency measures but also higher energy consumption, a simple comparison that pooled all households in all regions could lead one to conclude that the measure in question increased rather than decreased consumption. A similar example for an unobservable factor would be if environmentally conscious households are more likely to install measures and also have lower energy consumption overall, which would tend to conflate the effect of the measure with the effect of the environmental consciousness. Second, the effect of treatments may vary across households, e.g. by observable factors like income and property size which also affect energy consumption directly, but also by factors which are not observed yet are related to energy consumption. An example of such an unobserved factor is where highly energy inefficient buildings may be more likely to have measures installed and also to experience a stronger effect from an installed measure. In the latter example, the observed effect of the installed measure would tend to be an overestimate of the effect we can expect among the as yet untreated households (i.e., the most susceptible households may also be the earliest adopters, on average). NERA Economic Consulting 3

20 Methodological Overview Potential underlying mechanisms for allocating measures across households are often categorised under the following three headings: random assignment, selection based on observed characteristics, and selection based on unobserved characteristics. Random assignment is rarely observed in practice it would imply that households adopt efficiency measure independently of the type of building they occupy, how long they expect to remain in the property, their pre-treatment level of energy consumption, etc. If, instead, we assume that selection occurs but is based on observed characteristics, this is equivalent to saying that we know or are able to control, in some way, for all of the characteristics that make a household likely to decide to adopt an efficiency measure and that also influence energy consumption. Finally, if we assume that selection is based on unobserved characteristics that also influence energy consumption, we in effect say that the available household characteristic data are not sufficiently rich to enable us to control directly for factors which are confounded with treatment effects, so we must employ estimators that take this into account Selection based on observed factors If all factors which affect consumption both directly and via the selection of treatment effects are observable, then we can incorporate household characteristics into regression models of the effect of adopting treatments upon household energy consumption by conditioning on these characteristics in various ways. The idea is that it may be more credible to claim that households decisions to adopt energy efficiency measures are unrelated to pre-existing energy consumption levels or to expected reductions in energy consumption levels conditional upon the observed characteristics of the households. Two prominent (and complementary) types of regression models that attempt to use household-specific information to address the selection problem include panel regression models and models using propensity score methods. In this interim report, we present results from two econometric approaches to estimating treatment effects under the assumption of selection based on observed factors. Panel regressions / Difference in Differences Weighted average (cross-sectional) estimates One can also consider results based on basic linear regressions, which we discuss after summarising the above two approaches. We employ multiple approaches because it enables us to check whether the estimated treatment effects are robust to the different estimators that we employ. In effect, using multiple approaches acts as a test of how sensitive the results are to changes in the underlying assumptions. In addition, the basic regression approach provides a useful bridge to models that attempt to estimate a full demand specification, rather than only the treatment effects Panel regressions The panel approach utilises only changes over time within each individual household to estimate treatment effects. Like difference-in-differences approaches, panel techniques can NERA Economic Consulting 4

21 Methodological Overview control for time-invariant unobserved characteristics of a household that affect both uptake and energy use. In practice this is achieved by assuming that household consumption is the sum of a fixed household-specific factor that does not change over time ( α i, where the subscript i as before denotes a particular household), the effect of the treatments, and (in some specifications) year-specific effects. q it = α + τ + D δ + ε i t it it Note that since the data on household characteristics from Experian does not vary over time, it does not contribute any additional explanatory power over and above that contained in the household-specific fixed effect, and so those variables are not used in the panel regressions. Under the assumption that the effect of treatment does not vary between treated and nontreated, and that the timing of the uptake is not correlated with other factors affecting energy consumption (such as a change in household demographics from one year to another), this approach is capable of producing unbiased estimates of the treatment effects. However, panel approaches (or difference-in-differences) cannot control for unobserved characteristics that could influence the way or the amount that the treatment actually affects the individual household. In other words, it can control for heterogeneity within the population, but not for heterogeneity of treatment effects themselves Non-parametric approach / weighted average treatment effects Another approach that is sometimes employed in these studies is the so-called propensity score approach. This involves estimating the probability that a household will adopt a given efficiency measure conditional on that household s characteristics, dividing the population of households into pairs or groups with similar propensity scores, calculating treatment effects by propensity score groupings, and then calculating a weighted average treatment effect using weights derived from the distribution of propensity scores in the overall population. The nature of the treatments that are of interest in this study is that we have multiple treatments whose effects are of interest both on their own and in combination. These treatments may be adopted sequentially over time (in any order). This makes an appropriate model of propensity complicated to develop, as it would ideally account for uptake of treatments conditional on having taken up another treatment in previous years, as well as conditional on household characteristics. For the present analysis, therefore, we have not pursued the propensity score approach explicitly. Instead, we achieve a similar end that is, ensuring that observationally similar households are compared to each other to estimate treatment effects through a nonparametric approach. This approach, which is described more fully in Section 6, in effect involves: Computing average energy consumption within bins of observationally similar households; For each bin, computing the estimated treatment effect in a given year (for each treatment) as the difference in average consumption by the treated and untreated groups; NERA Economic Consulting 5