A Randomized Control Trial To Evaluate The Power of Information

Transcription

1 A Randomized Control Trial To Evaluate The Power of Information Carrie K. Armel June A. Flora Sébastien Houde Anant Sudarshan Annika Todd Stanford University Precourt Energy Efficiency Center 29 th USAEE/IAEE Calgary October 14 th 2010

2 Overview This talk is about quantifying the benefits of a particular type of technology that could be used to reduce energy usage.

3 Overview This talk is about quantifying the benefits of a particular type of technology that could be used to reduce energy usage. The benefits are not due to the technology alone but humans using the technology.

4 Overview This talk is about quantifying the benefits of a particular type of technology that could be used to reduce energy usage. The benefits are not due to the technology alone but humans using the technology. This command a particular approach to perform the evaluation.

5 Opportunities on the Demand Side The Problem Electricity consumers have little information pertaining to their consumption. Information is coarse and complex. Electricity consumption is not salient. Consumers have biased beliefs (underestimate costs?). Some Solutions Provide better information and feedback about electricity usage. Use technology to implement behavioral change interventions.

6 The Technology A hardware device records real-time household electricity consumption. A web application displays electricity information on households personal computer(s). Figure: Web Application: Google PowerMeter

7 Research Question What is the behavioral potential of this particular feedback technology?

8 Definition Behavioral Potential -The behavioral potential of a feedback technology is the opportunity for reductions in energy consumption through behavioral changes induced by the technology.

9 Research Strategy We used a field experiment to test the technology. The experimental design consists of an encouragement design. We used the instrumental variable framework to derive an internally valid estimate of the behavioral potential of the technology and its heterogeneity.

10 Experimental Design

11 Experimental Design: Planet Earth Viewed From...

12 Experimental Design Experiment implemented in collaboration with Google. Experiment restricted to Google households. At least one household member is employed at Google.

13 Experimental Design: Restricted Population

14 Experimental Design: Experimental Population All US Google employees were invited to participate in a study involving a feedback technology. Employees voluntarily enrolled. Prospective participants had to meet eligibility criteria.

15 Experimental Design: Experimental Population

16 Experimental Design: Randomization Eligible participants were randomly allocated to a control and treatment groups. All participants received a hardware device and had to perform the same steps to install the system. After collection of 1 week of baseline data, participants learned whether they were part of a control group (no access to Powermeter) or treatment group. Experimental procedure ensured that attrition did not occur because of assignment to experimental conditions.

17 Experimental Design: Randomization

18 Experimental Design: Treated Households Participants assigned to the treatment group decided to use or not to use the technology. Participants in the control group could not use the technology.

19 Experimental Design: Treated Households

20 Validity of the Experiment

21 Validity of the Experiment Table: Summary Statistics: Treatment and Control Groups Treatment Control T-C Pop CA Pop US Electricity Consumption Baseline mean (kwh/day) Demographics Household size % with children (0-18 y) % income <100, % engineer % in CA % owner Housing Characteristics % single family detached % < 1,000 sq. ft % heating with electricity Psychographic Factors % democrat or leaning % donation to environment % turn off lights if not needed % turn off power strips Nb of Households

22 Empirical Strategy We are interested in measuring the reductions in electricity usage for households that actually used the technology (the treated households). The decision to use the technology is most likely correlated with households characteristics that we cannot fully observe. We have an endogeneity problem. Assignment to the treatment and control groups is a valid instrumental variable. 1 Assignment is correlated with whether or not a household used the technology. 2 Assignment satisfies the exclusion restriction. 3 Assignment is uncorrelated with household characteristics because of randomization.

23 Quantities of Interest 1 Local Average Treatment Effect on the Treated Households. 2 Local Average Treatment Effect on the Treated Households at Different Time Periods. 3 IV Quantile Treatment Effects. 4 Heterogeneous Treatment Effects.

24 Quantity of Interest I: LATE on the Treated

25 Quantity of Interest I: LATE on the Treated Table: Local Average Treatment Effects (LATE) on the Treated Households Estimators (1) (2) (3) Y=log(daily kwh) Compliance Rate Use Technology (s.e.) (.0143) (.0084) (.0132) Demographics X Housing Characteristics X Psychographics X Daily Weather X X Week Fixed Effects X Household Fixed Effects X R Nb of Observations 35,222 40,219 35,222 Significance levels : : 10% : 5% : 1%

26 Quantity of Interest II: ATE Different Periods Table: LATE, Different Periods of the Week LATE week day LATE week evening LATE weekend LATE night Y=log(kwh) No Covariate Use Technology (s.e.) (.0168) (.0136) (.0269) (.0141) R Nb of Observations 25,164 25,146 10,044 35,218 Weather and Household Fixed Effects Use Technology (s.e.) (.0157) (.0124) (.0178) (.0100) R Nb of Observations 28,785 28,750 11,418 40,208 Significance levels : : 10% : 5% : 1%

27 Quantity of Interest III: IV Quantile Treatment Effects We can uncover the change in the distribution of electricity consumption due to the treatment. Note that this a different quantity than the estimate of the distribution of treatment effects.

28 Quantity of Interest III: IV Quantile Treatment Effects Table: IV Quantile Treatment Effects QTE.25 QTE.5 QTE.75 Y=log(daily kwh) Median No Covariate Use Technology (s.e.) (.0214) (.0121) (.0116) Nb of Observations 35,222 Long Regression: All Covariates Use Technology (s.e.) (.0302) (.0166) (.0151) Nb of Observations 35,222 Significance levels : : 10% : 5% : 1%

29 Quantity of Interest IV: Heterogeneous Treatment Effects Table: Heterogeneous Treatment Effects on the Treaded, by Quartiles Quartile 1 Quartile 2 Quartile 3 Quartile 4 Y=log(Daily kwh) Weather and Household Fixed Effects Compliance Rate Use Technology (s.e.) (.0220) (.0200) (.0141) (.0132) R Nb of Observations 8,301 9,516 11,168 11,234

30 Conclusions

31 Design: Results: Our experimental design consists of an encouragement design. Using the instrumental variable framework, we can uncover an internally valid estimate of the treatment effect on the treated. External validity is also possible with such design...but do not send the invitation to all potential users! Average treatment effect of the order of -3 to -5% Largest reductions in baseload consumption. Significant heterogeneity: many households experienced small reductions and some have very large reductions. Households with the larger baseline consumption decreased their consumption the most.

32 Acknowledgments We would like to thank without implications the Google Powermeter development team: Ryan Falor, Luis Arbulu, Ed Lu and Omar Khan. Our advisors and colleagues: Jim Sweeney, John Weyant, Jon Levin and Kenneth Gillingham. Funding from the Department of Energy, ARPA-E grant, and a doctoral scholarship from the SSHRC.