MO Power to the Customer: An Evaluation of a Dual Fuel Home Energy Reports Program

Transcription

1 MO Power o he Cusomer: An Evaluaon of a Dual Fuel Home Energy Repors Program Ken Agnew, Dr. Mn Nu, Paulo Tanmoo, Dr. Mram Goldberg, KEMA, Madson, WI Bobb Wlhelm, Puge Sound Energy, Bellevue, WA ABSTRACT Energy savngs programs feaurng behavoral elemens are n vogue. Home energy repors represen a relavely pure form of behavoral based energy conservaon program -- here s no use of prcng and here s no echnology provdng auomaed conrol or real-me dsplays. Parcpang households receve malngs whch provde energy comparsons o her neghbors, personal comparsons o her own energy consumpon, and argeed energy savng ps. Ths paper evaluaes 2 monhs of savngs resuls from a Home Energy Repors Program sponsored by Puge Sound Energy. Sarng n November 28, roughly 4, households n he uly s errory began recevng he repors from Opower. Ths program was he second program mplemened by Opower and was he frs o be offered o households ha used boh naural gas and elecrcy. The program was desgned as a randomzed conrolled ral. The mpac evaluaon of he program used acual monh bllng daa produced usng daly AMI daa. The paper examnes annual and monhly elecrc and naural gas savngs whn he frs 2 monhs. The analyss also eases ou he effecs of quarerly repors compared wh he ypcal monhly schedule. Fnally, he paper provdes he resuls of an arbuon analyss o deermne wheher energy savngs came from behavor change or from he nsallaon of oher uly-sponsored (and rebaed) energy effcency programs. Inroducon In an effor o beer undersand he energy savngs poenal of resdenal behavor-based conservaon programs, Puge Sound Energy (PSE) launched a plo program wh OPOWER o measure he mpacs of Home Energy Repors (HER) on energy consumpon for sngle famly, dual fuel households. Launched n November 28, he program provdes a monhly or quarerly Home Energy Repor o nearly 4, households n PSE s combned gas and elecrc servce errory. In 21, PSE hred KEMA o esmae he mpac of he HER on he energy consumpon of recpens. KEMA esmaed savngs by conduc a bllng analyss of he randomly assgned recpen and conrol groups of cusomers who were seleced from a pool of poenal parcpans of he HER Program. The analyss developed annual and monhly elecrc and naural gas savngs whn he frs 2 monhs. The analyss also compared he effecs of quarerly repors wh he ypcal monhly schedule of malngs. Fnally, he analyss ncluded an arbuon analyss o deermne wheher energy savngs came from behavor change or from he nsallaon of oher uly-sponsored (and rebaed) energy effcency programs.

2 Program Background The HER program provdes a monhly or quarerly HER o nearly 4, households n PSE s combned gas and elecrc servce errory. The repors are desgned o provde recpens wh feedback on her household energy use, how her usage compares wh ha of neghborng homes, and cusom ps on how a household can reduce her energy consumpon. PSE s program s desgned wh wo delvery mechansms: a monhly repor and a quarerly repor. Parcpang households are assgned o eher he monhly group or he quarerly group; never o boh groups. Each monh, hree quarers of he households n he reamen group receve a HER whch provdes deals on her household energy usage and ps on usng energy wsely. Once every quarer, he remanng households n he reamen group receve a repor wh he same nformaon provded o he monhly recpens. Expermenal Desgn Background HER Programs have boh he opporuny and necessy of workng whn an expermenal desgn framework. In hs respec HER Programs sand n conras o mos oher energy effcency programs. HER programs requre a parcularly robus, unbased evaluaon resul gven he relavely small magnude of expeced savngs n percenage erms. In urn, he HER program offerng s amenable o mplemenaon n an expermenal desgn framework unlke mos oher knds of programs. Before he PSE HER program launched, a group of 83,811 sngle famly homes, locaed n PSE s combned gas and elecrc servce errory, were seleced o parcpae n he expermen, eher as a recpen or conrol group parcpan. Households were seleced o parcpae on he expermen based on he followng crera: dual fuel (usng boh naural gas and elecrcy provded by PSE), sngle famly resdenal home, uses more han 8 MBu of energy per year, home does no ulze a Solar PV sysem, home address mus be avalable wh parcel daa from he couny assessor, home has as a bll hsory ha sars on or before January 1, 27, home mus have 1 smlar szed homes (neghbors) whn a wo mle radus, and he home mus have auomac daly meer reads. Afer selecon of he expermen households was complee, 39,755 homes were randomly assgned o parcpae n he recpen group and he remanng homes serve as a conrol group. Of he seleced recpen homes, 9,949 (25%) were randomly seleced o receve repors on a quarerly bass, whle he remanng 29,86 (75%) homes are parcpang as monhly repor recpens. The random assgnmen of monhly and quarerly repors allows boh PSE and OPOWER o es he effec of repor frequency on energy savngs. Mehodologcal Approach The evaluaon of PSE s HER program used wo analyss approaches. The frs approach was a basc dfference-of-dfferences approach. The dfference-of-dfferences approach leverages he expermenal desgn of he HER program o produce a smple, nuve esmae of program mpacs. For he second analyss approach, KEMA used a pooled, fxed effecs model. The pooled model explcly accouns for boh ndvdual household baseload and sysemac me perod effecs. By conrollng for ndvdual effecs, monhly effecs and weaher, he pooled approach reduces he sandard error of he mpac esmae. Of equal mporance, he model makes possble o normalze he esmaes of program savngs o reflec ypcal weaher condons. The fnal aggregae mpac esmaes are produced wh hs model.

3 Dfference-of-Dfferences The dfference-of-dfferences approach s he mos drec and smple way of leveragng he expermenal desgn of he HER program. The approach compares he dfference n reamen group average consumpon beween pre- and pos-repor perod wh he same dfference for he conrol group. The reamen group pre-pos dfference capures all changes beween he wo perods ncludng hose relaed o recevng he repors. The conrol group capures all changes wh he excepon of hose relaed o he repor. The random selecon of he reamen and conrol groups ensures ha, on average, he conrol group wll appropraely reflec he non-repor relaed changes experenced by reamen and conrol group alke beween he pre-and pos-repor perods. Removng he non-repor dfferences from he reamen dfference, as represened by he conrol group dfference, produces an unbased esmae of he repor s solaed effec on consumpon. Accordng o he expermenal desgn, pre-repor dfferences should be non-exsen whle posrepor dfferences reflec he effec of he repors. Eher knd of observed dfference beween parcpans and non-parcpans s unlkely o be evenly dsrbued across he year. In a dfference of dfferences framework, s deal f pre- and pos-repor perods cover he same 12 monh ncremens for boh parcpans and non-parcpans. In hs case, he pre-perod adjusmen due o random dfferences n he groups pror o malng of repors reflecs he dfference beween he wo groups across a full year. Smlarly, he pos perod, 12-monh ncremens wll represen annual esmaes of savngs. Imporanly, despe he presence of he conrol group, dfference-of-dfferences mpac esmaes reflec he observed weaher durng he analyss perod. The sandard pracce of reporng program savngs on a ypcal year bass s no possble wh he dfferences of dfferences approach. The average consumpon of energy for he reamen group n he pre-repor perod s calculaed wh he equaon: 1 ETrm Pr e = E ntrm Trm Average, 12 monh energy consumpon n he pre-repor perod for he ETrm Pr e = reamen group; n Trm = Coun of households n he reamen group; 12 monh energy consumpon for household ; E = Usng hs equaon srucure, average energy consumpon s calculaed for boh reamen and conrol groups n boh he pre- and pos-repor perods. The dfference of dfferences s hen produced wh he followng equaon. ΔE = ( E E ) ( E E ) Trm Pr e TrmPos Con Pr e ConPpos The dfference-of-dfferences approach can be appled on a monhly or seasonal bass. As long as me perods are balanced n he pre- and pos-repor perods he savngs esmae wll be conssen for ha me perod.

4 Pooled Model The prmary mpac esmaes for hs evaluaon were based on a pooled regresson approach o modelng he monhly consumpon daa. We esmaed mulple specfcaons whn a fxed effecs framework. The basc fxed effecs regresson approach models all household monhly consumpon as a combnaon of a household-specfc baseload, average heang and coolng rends and monhly meseres effec. The model specfcaon s srucured o effecvely replcae, n he regresson framework, he dfference-of-dfferences approach dscussed above. The relevan oupu of he model s he modeled pre-pos dfference n he reamen group, ne of he pre-pos dfference of he conrol group. The pooled, fxed effecs approach mproves on he dfference-of-dfferences approach by allowng fnal savngs esmaes ha are derved wh normalzed weaher. Ths means savngs esmaes represen expeced savngs n a ypcal weaher year. The pooled approach also conrols for beween household varably reducng he sandard errors on he mpac esmaes. Fnally, he pooled approach can be run so as o produce heeroscedascy-robus esmaes of savngs sandard error. The pooled model approach produces resuls wh precson beer han en percen a a 95 percen confdence level. Basc Pooled Specfcaon The prmary mpac esmaes for hs evaluaon are based on a pooled approach o modelng he monhly consumpon daa. We esmaed mulple specfcaons whn a fxed effecs framework. The basc fxed effec specfcaon has he followng equaon: E = μ + λ + ε H1 C C1 H T 2 H 2 C 2 H C T T P 3 H 3 C3 C H P P T P 4 H 4 C 4 C H T P T P Where E = Energy consumpon per day durng monh for cusomer ; H or C = Average heang or coolng degree-days a he base emperaure τ H or τ C durng monh, based on daly average emperaures, for cusomer s meer readng perod; T = One for households n he reamen group, Oherwse; P = One for me perods n he pos-repor perod, Oherwse; μ = Household-specfc base consumpon esmae for cusomer ; λ = Monh-specfc me perod effec for monh ; β = Coeffcens, deermned by he regresson; ε = Regresson resdual. The household fxed effec, μ n hs model, capures he unque household level baseload consumpon. The me perod fxed effec, λ n hs model, capures sysemac monhly effecs shared by all households. These fxed effecs conrol for correlaon of resdual errors over me for an ndvdual cusomer, and across cusomers for a gven me perod.

5 The expermenal desgn s a smple 2x2 desgn: reamen and conrol by pre- and pos-repor. In he regresson framework, he 2x2 desgn s represened by a base level (nercep), a reamen effec, a pos-repor effec and a combned reamen and pos-repor effec. Esmaed ogeher, he nercep represens average conrol group consumpon n he pre-repor perod, he reamen effec capures he dfference beween reamen and conrol across all me perods, he pos-repor effec capures he dfference beween pre-and pos-repor perods for boh reamen and conrol groups, and The combned reamen and pos-repor effec capures he margnal effec n he pos-repor perod for he reamen group alone. Ths basc srucure apples o all weaher and characersc effecs ncluded n he model. In he pooled model specfcaon provded above, he reamen effec, he pos-repor effec and he combned reamen and pos-repor effec are clearly evden n he base, heang and coolng porons of he model (T, P, and T P, respecvely). All four combnaons are evden n he neracons wh heang and coolng degree days. For he non-weaher effecs, he base level s represened by he household effec μ whch s a unque nercep for each household. Includng degree-day erms s mporan o conrol for weaher effecs over me, and o make rackng of reamen effecs over me more meanngful. Though he presence of he conrol group conrols for weaher effecs beween he pre- and pos-repor perod, he esmaed savngs from he posrepor perod s sll a funcon of he weaher durng he evaluaon perod. As wh any energy effcency mpac esmae, he savngs should be pu on a ypcal year bass, so ha savngs do no reflec consumpons paern from an evaluaon meframe defned by aypcal weaher. Model Choce A varey of model specfcaons have been dscussed n he HER program evaluaon leraure. Whn he pooled, fxed effec model framework 1, hs model specfcaon ncludes he bes aspecs of hose models. Oher research has proposed specfcaons wh logged dependen varables wh and whou fxed effecs and/or weaher varables. A hs pon, he pooled, fxed effecs approach pursued here reflecs he bes of he approaches comng ou of he energy evaluaon conex. Absorbed Pooled Specfcaon For an analyss wh a sample he sze of he PSE HER program, a fxed effec model srans he lms of mos sascal compuer programs. The household fxed effec eners he model as a marx of dummy varables wh dmensons equal o he number of cusomer n boh he reamen and conrol groups. The nverson of a marx of hs magnude s compuaonally challengng. Forunaely, for he lnear fxed effecs model, here s a way o ransform he daa ha removes he marx of dummy varables from he compuaon. Smply pu, he same model s run wh all varables de-meaned -- ha s, all varables are replaced by her dfference from he household level mean of ha varable. The deals of hs approach are no dscussed furher here bu are a key praccal consderaon for he developmen of hese models. 1 Ths would nclude Summ Blue s 29 Impac Evaluaon of OPOWER SMUD Plo Sudy wh her bes model, he dfferenced lnear effecs model.

6 Model Resuls Savngs are esmaed by fng he absorbed pooled model parameers assocaed wh he combned reamen (T ) and he pos (P ) effecs. For esmaes of normalzed savngs, he degree days should be f o degree days calculaed usng normal emperaures and he opmal degree day based deermned for he model: Δ E = ( 365 ˆ β + ˆ β H + ˆ β 4 H 4 C 4 ( C Where: Δ E = Esmaed annual energy savngs; H ( Normal annual heang degree-days a he opmal heang base = emperaure; C ( = ˆ β, 4 ˆ, ˆ β H 4 β C 4 = Normal annual coolng degree-days a he opmal coolng base emperaure; Esmaed parameers from he absorbed, pooled model; Program Resuls Monhly Dfferences Fgures 1 and 2 provde he monhly reamen mnus conrol group consumpon dfferences for 39 monhs ncludng 15 monhs n he pre- malng perod and 24 monhs n he pos perod. In he premalng perod for boh fuels (o he lef of he vercal lne n boh fgures), he resuls are all or mosly below zero ndcang slghly lower levels of consumpon among reamen parcpans even pror o he malngs. These monh dfferences are no sascally dfferen han zero and would also be removed n he dfference of dfference framework. The pos perod shows conssen sascally sgnfcan reducons n consumpon hrough he full wo year pos perod. These resuls are presened as a vsual reference of he program resuls 2. 2 Dfference of dfference resuls were calculaed separaely for he full wo-year pos perod afer he submsson of he repor on whch hs paper s based. The remanng resuls reflec he 2 monh pos perod addressed n he orgnal repor.

7 kwh Savngs Lower CI Upper CI Jul 7 Sep 7 Nov 7 Jan 8 Mar 8 May 8 Jul 8 Sep 8 Nov 8 Jan 9 Mar 9 May 9 Jul 9 Sep 9 Nov 9 Jan 1 Mar 1 May 1 Jul 1 Sep 1 Fgure 1. Average Household Monhly Elecrc Dfferences, Conrol vs Treamen Therms Savngs Lower CI Upper CI Jul 7 Sep 7 Nov 7 Jan 8 Mar 8 May 8 Jul 8 Sep 8 Nov 8 Jan 9 Mar 9 May 9 Jul 9 Sep 9 Nov 9 Jan 1 Mar 1 May 1 Jul 1 Sep 1 Fgure 2. Average Household Monhly Gas Dfferences, Conrol vs Treamen

8 Full Program Annual Savngs Evaluaon Resuls ndcae a posve and ncreasng elecrc and gas savngs for households whch parcpaed n he Home Energy Repors Program. We repor ypcal year, average savngs for hree dfferen meframes. The frs 12 monhs of he program, November, 28 Ocober, 29, All 2 monhs he program has been n exsence, November, 28 June, 21, and The las 12 monhs of he program, July, 29 June, 21. For boh elecrc and gas he average savngs ncrease as he meframe ncludes more recen daa. Fgure 3 provdes a vsual represenaon of he elecrc and gas savngs for he hree dfferen meframes Annual kwh 15 1 Frs 12 Monhs Annual Therms All 2 Monhs 4 Frs 12 Monhs 5 Las 12 Monhs All 2 Monhs 2 Las 12 Monhs All Treamen Households All Treamen Households Fgure 3. Full Program Elecrc and Gas Average Annual Savngs, Three Tmeframes Table 1 shows he frs year resuls and her assocaed 95 percen confdence nervals n abular form. Frs year savngs are esmaed a kwh and 11.3 herms per household for household recevng he repors. The able also repors average savngs n percenage erms wh respec o prerepor consumpon and oal savngs for he 31,618 household for whom we can esmae savngs. Table 1. Frs Year Average Savngs (November, 28 Ocober, 29) Consumpon kwh/therms +/ * Percen +/ * 1 Therms +/ * Elecrc (kwh) %.12% 6, Gas (Therms) %.15% *95 percen confdence level

9 Table 2 shows ha he average annual savngs over he 2 monhs of pos-repor daa avalable average savngs are esmaed a 24.2 kwh and 12.8 herms. Table 2. All Monh Average Savngs (November, 28 June, 21) Consumpon kwh/therms +/ * Percen +/ * 1 Therms +/ * Elecrc (kwh) %.11% 6, Gas (Therms) %.13% *95 percen confdence level Fnally, Table provdes he average annuals savngs for he mos recen 12 monhs for whch daa are avalable. Averages savngs are esmaed a kwh and 14. herms. Table 3. Las Year Average Savngs (July, 29 June, 21) Consumpon kwh/therms +/ * Percen +/ * 1 Therms +/ * Elecrc (kwh) %.12% 7, Gas (Therms) %.14% *95 percen confdence level The resuls provded n hese ables are model-based resuls ha have been weaher normalzed. They reflec expeced annual savngs n ypcal weaher year for he average household n he conrol and reamen samples. Program Annual Savngs, Monhly vs Quarerly Malngs The PSE HER program esed wo dfferen malng schedules for he Home Energy Repors. A subse of 25 percen of he reamen group receved a repor every hree monhs, whle he remander receved repors on a monhly bass 3. Fgure 4 provdes a comparson of monhly and quarerly elecrc and gas savngs for he hree dfferen meframes. Boh bar graphs show ha quarerly repors generaed lower savngs han he monhly repors. The dfference s sascally sgnfcan for all meframes for elecrc savngs a 95 percen confdence. For gas, he dfferences n he laer me frames are sgnfcan a he 9 percen confdence level. 3 The average annual savngs repored above represen he overall savngs for he program gven hs mx of monhly and quarerly repors.

10 Annual kwh Frs 12 Monhs All 2 Monhs Las 12 Monhs Annual Therms Frs 12 Monhs All 2 Monhs Las 12 Monhs Monhly Quarerly Monhly Quarerly Fgure 4. Monhly vs Quarerly Elecrc and Gas Average Annual Savngs, Three Tmeframes There s an apparen dfference beween elecrc and gas savngs for households recevng he quarerly repors. Gas savngs were lower for household recevng quarerly repors bu hey ncreased hrough he meframes n a manner smlar o he monhly repors. To he conrary, elecrc savngs for households recevng quarerly repors was effecvely fla hrough he weny monhs. Whou survey daa from program parcpans s no possble o know for sure wha s drvng hs dfference. Gas savngs are prmarly relaed o reducons n heang consumpon. I s possble ha parcpans are lowerng her hermosa se pons and ha hese changes are more lkely o be reaned over he hree monh perod beween repors. To he exen ha elecrc savngs are based on behavoral changes lke urnng lghs and oher plug loads off, s possble ha such behavors are no reaned as effecvely over he hree monh perod beween repors. Table 4, Table 5, and Table 6 provde he household annual savngs for monhly repors versus quarerly repors n abular form. Table 4. Frs Year Average Savngs (November, 28 Ocober, 29) Consumpon kwh/therms +/ * Percen +/ * 1 Therms +/ * Elecrc (kwh) Gas (Therms) Monhly Monhly % 1.24%.14%.17% 6, Quarerly Quarerly %.99%.23%.29% 4, *95 percen confdence level Table 5. All Monh Average Savngs (November, 28 June, 21) Consumpon kwh/therms +/ * Percen +/ * 1 Therms +/ * Elecrc (kwh) Gas (Therms) Monhly Monhly % 1.42%.13%.15% 7, Quarerly Quarerly % 1.9%.21%.26% 4, *95 percen confdence level

11 Table 6. Las Year Average Savngs (July, 29 June, 21) Consumpon kwh/therms +/ * Percen +/ * 1 Therms +/ * Elecrc (kwh) Gas (Therms) Monhly Monhly % 1.56%.14%.17% 7, Quarerly Quarerly % 1.21%.24%.28% 4, *95 percen confdence level Poenal for Double Counng Prelmnary examnaon of rackng daa from oher PSE Energy Effcency programs offered evdence ha he Home Energy Repors have ncreased gas rebae program parcpaon among reamen group households. We examned PSE Energy Effcency programs rackng daa from November 28 o he presen and denfed program parcpans from boh he reamen and conrol group. Comparng he parcpaon levels beween he reamen and conrol groups, gas rebae program nsallaons ncreased for he reamen group. The gas savngs due o ncreased program parcpaon represen approxmaely en percen of he overall HER program gas savngs. If hese savngs were clamed by boh he HER program and he rebae programs, hs would amoun o double counng for ha poron of he HER program gas savngs. Conclusons Ths paper summarzes he evaluaon of he Puge Sound Energy (PSE) Home Energy Repors (HER) plo program. The resuls of he evaluaon help us beer undersand he energy savngs poenal of resdenal behavor-based conservaon programs, The evaluaon confrmed savngs of beween one and wo percen dependng for he wo fuels and he dfferen 12 monh perods. Conssen resuls were calculaed usng dfferen mehodologcal approaches. The resuls repored here are weaher normalzed savngs esmaes from pooled, fxed effecs models. The evaluaon also confrmed ha monhly repors generae more savngs han he quarerly repors. The dfference for gas savngs s less dramac n boh magnude and rend. The dfference for elecrc savngs s larger han for gas for he frs welve monhs and he dfference almos doubles for he las welve monhs. Ths ndcaes ha more frequen malngs are parcularly mporan when seekng elecrc savngs. The evaluaon dd ndenfy he poenal for double counng of gas savngs. The HER program reamen group ncreased her parcpaon n gas rebae programs relave o he conrol group. Ths ncreased parcpaon amouned o savngs equalng approxmaely 1 percen of oal HER program gas savngs. References Navgan Consulng. Impac Evaluaon of OPOWER SMUD Plo Sudy. 29. Prepared for OPOWER (wh subsequen versons now avalable).