Evaluatin f a Lw-Incme Energy Efficiency Prgram Jacqueline Berger, APPRISE ABSTRACT The NJ Cmfrt Partners Prgram prvides n-cst energy efficiency services t lwincme utility custmers thrughut the state. This prgram is a gd example f a lngstanding lw-incme efficiency prgram that has evlved and imprved ver the past decade by wrking t incrprate state f the art weatherizatin prcedures and techniques and evaluatin recmmendatins. The prgram is unique because electric and gas utilities acrss the state have cllabrated t prvide jint service delivery t eligible custmers. A cmprehensive evaluatin f this prgram is assessing hw prgram prcedures have been implemented and the effectiveness f prgram treatments. The research includes analysis f prgram data, usage and payment impact analysis, custmer surveys, n-site bservatin f service delivery, and pst-cmpletin inspectins. The research will als include an analysis f hw prgram prtcls fr energy savings relate t the actual savings achieved in the prgram. Because the evaluatin includes n-site research, infrmatin n the quality and cmprehensiveness f treatments will be factred int this analysis. Recmmendatins will be als be made fr mdifying the prtcls t prvide fr mre accurate predictins f energy savings. Objectives The bjectives f the study were t dcument the fllwing: The extent t which the prgram achieves its gals. Opprtunities available fr increased effectiveness r reduced csts. Specific prgram changes t accmplish these imprvements. Study Overview The evaluatin wrk scpe had five key tpic areas with several subtasks. 1. Data Tracking System This research identified the prgram s infrmatin needs; reviewed the cntent and quality f infrmatin in the current tracking system; assessed the sufficiency, accuracy, and efficiency f the system; determined hw the system is used by prgram partners t effectively manage the prgram; and develped recmmendatins fr enhancements t and use f the system.. Mving frward, the research will verify prtcl savings calculatins that are cntained in the database. 2. Prcess The prcess research included materials review, needs assessment, utility interviews, cntractr interviews, n-site bservatins, and inspectins f cmpleted jbs. 2-13
Prcedures and Materials Review: We reviewed and assessed the Cmfrt Partners prcedures manual, and additinal materials including staff training, marketing, and reprting. Needs Assessment: We analyzed American Cmmunity Survey data t prvide infrmatin n the characteristics and needs f NJ husehlds that are eligible fr the prgram. Utility Interviews: We cnducted in-depth interviews with managers and staff at the six participating utilities t develp a cmplete understanding f prgram design and implementatin, reviewed utility decisin making fr cnsistency, and identified barriers t prgram effectiveness. Cntractr Interviews: We cnducted in-depth interviews with managers and staff at the five prime service delivery cntractrs. The interviews included discussin f staff experience, training and certificatin prcedures, perfrmance tracking, and recmmendatins fr prgram imprvement. We spent a day n site at each prime service delivery cntractr t assess prgram management and peratins. On-site Observatins: We cnducted n-site bservatins f audits, measure installatin, and final inspectins using a detailed data cllectin system that assessed whether key steps and tests were cnducted, and rated the quality and cmprehensiveness f the services. Inspectins f Cmpleted Jbs: We are cnducting inspectins f cmpleted wrk t prvide a statistically reliable analysis f jb quality and cmprehensiveness. 3. Affrdability The research assessed custmers views n the prgram s impact n affrdability and will measure this impact thrugh the custmer billing analysis. Custmer Survey: We cnducted a survey with prgram participants t assess prgram understanding, impact, and satisfactin. Affrdability and Payment Impact Analysis: We are analyzing custmer billing and payment data t estimate the prgram s impact n energy bills, subsidies, energy burden, and energy bill payment. 4. Impact The impact research includes analysis f the data btained frm the prgram s data tracking system, cmputatin f savings estimates, estimates f the impacts f the prgram thrugh analysis f custmers energy usage data, and analysis f the realizatin f expected savings. Cmfrt Partners Prgram Data Analysis: We analyzed the tracking system data t characterize the prgram, including custmer and husing characteristics, measure installatin penetratin and csts, and inclusin f health and safety measures. Engineering Impact Estimates: We are using data n measures installed t estimate the expected impacts f service delivery. Usage Impact Analysis: We btained energy usage data frm the six utilities and are cnducting weather nrmalized, cmparisn grup adjusted analysis f the energy impacts f the prgram n natural gas and electricity cnsumptin. Savings Realizatin Rates: We will cmpare predicted t actual savings and cmpute the savings realizatin rates. 2-14
5. Energy Saving Prtcls We will review and verify the apprpriateness f existing energy saving prtcls, cmpare thse estimates t ur engineering impact estimates, and recmmend changes that culd imprve the accuracy f the savings estimates based n findings frm the impact analysis. Prgram Resurces The NJ Cmfrt Partners Prgram has develped and refined tls and guidelines fr managing and implementing the prgram. This sectin fcuses n the data tracking system and the prgram prcedures. The fllwing research activities were cnducted t prvide an understanding f the cntent and capabilities f the data tracking system, hw the system is used, and planned and desired system enhancements. 1. Tracking System Data Analysis We dwnladed the data in the tracking system, reviewed the included data fields included, and analyzed thse data. 2. User Access t Tracking System We reviewed the infrmatin, capabilities, and reprts available t utilities, cntractrs, and the inspectr. 3. Utility Interviews We discussed hw utilities use the tracking system, reprts that are utilized t manage the prgram, and hw the system can be imprved. 4. Cntractr Interviews We discussed hw the Cmfrt Partners tracking system is used, use f additinal internal data systems and why thse systems are needed, and recmmendatins fr imprvements t the CP tracking system. 5. Cntractr On-Sites We bserved hw cntractrs use the prgram tracking system and their wn tracking systems, and btained cpies f reprts used t manage the prgram. The Cmfrt Partners Tracking System prvides imprtant data t manage and implement the prgram, t evaluate the prgram, and t determine hw the prgram can be imprved. Data that are available t prgram managers and cntractrs are much mre cmprehensive than have been seen in many ther prgrams. Recmmendatins fr imprving the system t prvide fr mre efficient prgram management and peratins, and t allw fr a mre cmprehensive evaluatin are as fllws. 1. Management Many canned reprts in the CP tracking system allw the utilities t btain a list f custmers wh meet certain criteria. Additinal reprts that prvide summary statistics culd prvide useful infrmatin t help manage the prgram. Fr example, the fllwing types f reprts, by cntractr and utility ver specified time perids, may be useful fr utilities and/r cntractrs. Number f jbs audited, installed, and cmpleted Percent f jbs deferred r partial due t service delivery barriers Average jb cst Percent f jbs that have certain key measures installed Average measure cst fr key measures installed Fr the shrter term, the utilities and cntractrs shuld develp a list f reprts that wuld be helpful fr prgram management. Fr the lnger term, the wrking grup 2-15
shuld additinally cnsider develping a system t allw utilities t perfrm n-line queries r t dwnlad the data s that they culd create custmized reprts that best meet their needs. Hwever, ne challenge is that because the data are usually fr mre than ne utility, the utilities must agree t make thse data available t bth utilities invlved in a jb. 2. Data Quality The type and amunt f quality cntrl cnducted n data entered int the system varied by cntractr. While tw cntractrs had frmal data validatin checks and balances, the three ther cntractrs did nt. In ur use f the data, we fund inaccuracies in the accunt numbers. We recmmend that the utility managers require that cntractrs develp and submit a data quality cntrl plan and that the tracking system include, as planned, additinal data quality checks. 3. Evaluatin Data Previus research has dcumented the ptential and actual health and safety benefits that result frm energy efficiency services. Sme f these impacts can be best dcumented using data that are cllected n the audit paperwrk, but that are nt currently included in the tracking system data fields. Adding a few fields t the database wuld allw fr analysis f the prevalence f these types f issues and hw frequently they are reslved by the prgram. Cmfrt Partners culd then dcument the health and safety impacts f the prgram. We recmmend that the fllwing additinal fields are added t the tracking system. Ambient CO pre and pst Flue CO pre and pst Gas Leak detected Additinal data items that wuld be useful in the evaluatin are described belw. Inspectin Type The database allws fr assessment f the percent f inspectins that were dne and the pass rate and prblems fund. Hwever, the system des nt indicate the type f inspectin that was dne. It wuld be useful fr the evaluatin t have a better understanding f the cmprehensiveness f inspectins that were undertaken. This is a planned enhancement t the system. Measure Cding It wuld be useful t have the system cde measures as t whether r nt they are included in the seasnal spending allwance. This wuld allw fr a clean cmparisn f the amunt spent n seasnal measures and the seasnal allwance that was calculated. This New Jersey Cmfrt Partners Prgram is well ahead f many ther utility efficiency prgrams, as it has prvided detailed prcedures and specificatins fr their cntractrs t implement. This includes prcedures that have been updated as the results f findings frm the previus cmprehensive prgram evaluatin. Many ther utility efficiency prgrams rely n their implementers t fllw prcedures that are prvided thrugh WAP r ther standards, rather than prviding their wn prgram guidelines. This prcedures review included recmmendatins fr imprving the rganizatin f the manual, and imprving and clarifying sme f the technical infrmatin, prcedures, and materials described. Hwever, a full set f priritized recmmendatins cannt be furnished until after the impact evaluatin is cmpleted. The impact evaluatin will prvide up-t-date 2-16
findings n the energy-saving impacts f the measure installatin prcedures cntained in the manual, and may result in sme specific recmmendatins fr revising installatin guidelines where measures are r are nt fund t be cst-effective. The New Jersey Cmfrt Partners prgram fllws a cntinual imprvement prcess with frequent updates t the prgram prcedures and the manual. The prcess has the benefit f prviding state f the art techniques and implementing imprtant findings frm the field. Hwever, it has als resulted in a manual that is patched tgether in places, and sme cnfusin fr cntractrs. Managers shuld cnsider a regular prcess fr updates and adptin f thse updates, s that implementers can have a better understanding f prgram expectatins. Eligible and Served Custmers This Needs Assessment prvides a prfile f New Jersey s lw-incme husehlds using data frm the 2009-2011 American Cmmunity Survey (ACS). These data prvide infrmatin n the demgraphic characteristics, energy assistance needs, and efficiency service needs f lwincme husehlds thrughut the state and by cunty. We fcused n husehlds with incme at r belw 225 percent f the pverty level, the incme-eligibility standard fr the prgram. The data represent the state in 2011. Several key facts abut this ppulatin are imprtant fr analyzing prgram need and eligibility fr Cmfrt Partners services, and hw this varies thrughut the diverse state. 26 percent f husehlds in New Jersey had incme at r belw 225 percent f the pverty level. Abut 62 percent f the lw-incme husehlds had husing and energy bill characteristics that made them eligible fr NJ Cmfrt Partners. 85 percent f these husehlds heated with natural gas r electricity. Abut 78 percent lived in a husing type that is eligible (single family r multi-family buildings with n mre than 14 units). 90 percent paid directly fr gas r electric. Sme demgraphic characteristics can make it mre difficult t serve segments f the ppulatin. Only 39 percent wned their hmes, and renters can be mre difficult t serve, as landlrd permissin must be btained. 39 percent did nt speak English in the hme and may have the need fr service prvided in anther language if a family r friend interpreter is nt available. Many f these husehlds had usage at a level that indicates a need fr energy efficiency services. 50 percent f thse wh did nt heat with electricity were estimated t have annual electric usage ver 8,000 kwh. 24 percent f thse wh heated with electricity were estimated t have annual electric usage ver 16,000 kwh. 55 percent f thse with gas heat were estimated t have annual gas usage abve 1,200 ccf. Anther key finding frm this analysis is the extent f diversity acrss the state. Pverty level: While 26 percent f husehlds were incme-eligible fr NJ CP in the state as a whle, the percent eligible varied frm 14 percent t 36 percent in varius 2-17
cunties. Five cunties had less than 20 percent incme-eligible and five cunties had 30 percent r mre incme-eligible fr NJ CP. Main Heating Surce: The majrity f husehlds in mst cunties used natural gas as the main heating fuel, but this was nt the case in fur cunties. Hme Type: The majrity f lw-incme husehlds in 14 cunties lived in single family hmes, and the majrity f lw-incme husehlds in six cunties lived in multi-family buildings. Language: The percent f lw-incme husehlds wh spke English at hme ranged frm 32 percent in ne cunty t 87 percent in anther. The percent f lw-incme husehlds wh spke Spanish at hme ranged frm five t 52 percent. Eligibility fr Cmfrt Partners: We estimated that 62 percent f lw-incme husehlds in New Jersey were eligible fr Cmfrt Partners because they had gas r electric heat, paid their utility bill directly, and lived in an eligible husing type, but this percentage varied frm 30 percent t 79 percent in varius cunties. These factrs shuld be reviewed when thinking abut the challenges, pssibilities, and strategies fr the NJ Cmfrt Partners Prgram in different parts f the state. The NJ Cmfrt Partners tracking database prvides rich infrmatin t examine the ppulatin f husehlds served by the prgram, their hme and jb characteristics, prgram spending and measures, and inspectin results. These data add greatly t the understanding f wh the prgram is able t serve and hw these services are delivered. Demgraphics: Mst f the husehlds served by Cmfrt Partners, abut 70 percent, had at least ne vulnerable husehld member wh was a child, elderly, r disabled. The majrity f husehlds served had extremely lw annual husehld incme, f less than $20,000. Only five percent had incme f mre than $40,000. While abut 35 percent had emplyment incme, abut 25 percent had retirement incme, 13 percent received public assistance, 11 percent received disability, and seven percent received unemplyment incme. While the Needs Assessment fund that 59 percent f incme-eligible husehlds rented their hmes, the tracking database analysis shwed that nly 30 percent f thse served were renters. Hwever, sme lw-incme renters wuld nt be eligible because they lived in buildings with ver 14 units r did nt pay directly fr electricity r gas. EIC and Nrtheast Energy were mre successful than the ther cntractrs at serving renters. Hme Characteristics: Abut 70 percent f the hmes treated were single family and mst f the rest were multi-family hmes. Rw hmes were abut ten percent f thse served, and mbile hmes were nly abut three percent. Abut 90 percent f treated hmes had natural gas heat and abut ten percent had electric heat. Supplemental heating was used in a large percentage, 38 percent, f hmes. Almst all f the supplemental heating was electric heating. Pre-Treatment Usage: Pre-treatment energy usage has been fund t be highly related t the amunt and percent f pre-treatment energy usage saved. The NJ Cmfrt Partners prgram benefits the efficiency f administratin by prviding jint delivery f electric and gas service, and als makes the prgram mre cnvenient fr custmers. Hwever, the linkage can limit the ability f an individual utility t serve nly its highest usage 2-18
custmers. This results frm the fact that if the custmer is targeted fr service by the gas utility, the electric utility will serve the custmer regardless f the usage level. A gd benchmark fr an effective gas prgram is t serve custmers with usage f 1,200 ccf r higher annually. The analysis shwed that mean pre-treatment usage fr custmers with gas heat and gas water heat was 1,030 ccf, but that 25 percent had usage f 1,200 r higher. This cmpared t 55 percent seen in the eligible ppulatin analysis. Electric baselad usage prgrams are ften fund t be effective when usage is 8,000 kwh r mre annually. Custmers with gas heat and ht water averaged 7,837 pretreatment electric usage and the median was 7,200 kwh. Electric heating prgrams frequently define high use as 16,000 kwh r mre annually. Hmes with electric heat and ht water averaged 13,815 in pre-treatment use, and 25 percent had usage f mre than 16,215. This cmpares well t the 24 percent fund t have usage ver 16,000 kwh in the eligibility analysis. Testing Results: One third f the jbs did nt have a pre-treatment blwer dr test and 52 percent did nt have a pst-treatment blwer dr test. Cntractrs reprted that they were nt able t d blwer dr tests due t health and safety issues, such as mld, asbests, vermiculite insulatin, use f breathing apparatuses r ther medical equipment, and custmer refusals. The percentage missing either a pre- r pst-treatment blwer dr test varied widely by cntractr. When examining the half f the hmes that had pre- and pst-treatment blwer dr test results, the analysis shws that 19 percent had a reductin f 1,000 CFM50 r mre, and eight percent had a reductin f 1,500 CFM50 r mre. The mean reductin was abut 600 CFM50. Refrigeratrs were mnitred in abut 80 percent f the treated hmes. This als varied greatly by cntractr, frm 56 t 90 percent f the hmes treated. The metering results shwed that abut 56 percent f the metered refrigeratrs had usage ver 1,000 kwh, mst f which wuld be eligible fr replacement under the prgram. Measures: Measure installatin was examined bth by prtcl savings categry and by detailed measure grup. The prtcl savings categry analysis shwed that 79 percent had CFL s installed, 65 percent had air sealing, and 61 percent had ht water measures. Other categries where abut ne third r mre f the jbs had measures were HVAC, refrigeratrs, thermstats, duct sealing, and insulatin. The percentages varied widely by cntractr. The prgram database prvided a detailed list f 406 different measures. We cmbined these int measure categries fr the purpses f analysis. The mst cmmn measures, with penetratin f ver 85 percent, were the audit, energy educatin, and cmbustin testing. Health and safety measures were prvided in 78 percent f hmes, air sealing was dne in 60 percent, and attic insulatin in 27 percent. While 98 percent f jbs with gas heat and gas water heat had a health and safety measure, 61 percent f thse with electric heat and electric ht water had a health and safety measure. The infrmatin will be incrprated int the usage impact analysis t assess measurelevel impacts and prgram targeting, and t make recmmendatins fr refining the prgram prcedures. 2-19
We cnducted 977 telephne interviews with custmers wh participated in Cmfrt Partners and had their installatins cmpleted apprximately ne year prir t the survey. Key findings are summarized belw. Respndent Characteristics The custmer survey cllected infrmatin nt available in the prgram tracking database. This infrmatin demnstrates that the prgram is serving a grup f custmers wh have need fr assistance. 15 percent f custmers served by Cmfrt Partners had ne r mre veterans in the husehld. 47 percent reprted that they received retirement incme in the past year, 27 percent received cash assistance, and 22 percent received emplyment incme. In additin, 40 percent received nn-cash assistance. 37 percent reprted that smene in the husehld had been unemplyed and lking fr wrk in the past year. Reasns fr Participatin The mst cmmn surce f infrmatin (27 percent) fr the prgram was a friend r relative. 61 percent reprted that the main reasn they wanted t participate in Cmfrt Partners was t reduce their energy bills. Understanding f Energy Bill While 43 percent f ne cntractr s custmers reprted that the prvider reviewed the energy bills, 24 t 35 percent f the ther cntractrs custmers reprted this. 48 percent reprted that the prvider explained hw energy use is measured. One cntractr s custmers were less likely t reprt that they received this infrmatin. 78 percent reprted that they felt they had a gd understanding f hw t review their energy bill. Actin Plan and Actins Taken 54 percent reprted that the service prvider furnished a written plan f actins t save energy. This ranged frm 44 percent fr ne cntractr t 58 percent. 27 percent reprted that the service prvider tld them hw much mney they culd expect t save by taking the actins n their plan. 60 reprted that they had taken energy-saving actins, and when asked what they did, 48 percent reprted at least ne actin. Custmers f tw cntractrs were mre likely t reprt that they reduced their heating, ht water, and air cnditining usage than the thers. Prgram Measures Mst custmers, 92 t 95 percent, reprted that they were very r smewhat satisfied when asked specifically abut insulatin, air sealing, and heating system wrk. Hme Cmfrt 50 percent said that the winter temperature in their hme imprved and 39 percent said that their summer temperature had imprved. Satisfactin Mst custmers prvided high ratings fr all aspects f the prgram. 84 percent stated that the prgram was very r smewhat imprtant in helping the custmer t meet his r her needs. Recmmendatins 2-20
Custmer need The survey fund custmers were likely t be veterans, receive cash and nn-cash assistance, r t have been unemplyed in the past year. Recmmendatin The Cmfrt Partners Prgram shuld assert that ne imprtant prgram benefit is that they are helping custmers in the state wh are very much in need f assistance. Prgram infrmatin surce Friends and relatives The mst cmmn surce f infrmatin fr the prgram was a friend r relative. Recmmendatin The prevalence f wrd f muth marketing is an imprtant reasn t ensure that participants understand the prgram purpse, benefits, and custmer rle. Energy educatin Educatin prvided The survey fund that tw cntractrs were mre effective in prviding energy educatin and their custmers were mre likely t reprt that they tk energy-saving actins. Recmmendatin The utilities shuld prvide additinal guidance t the ther cntractrs n prviding energy educatin t custmers. Custmer actins Mtivatin fr participatin The majrity f custmers, 61 percent, reprted that the main reasn they wanted t participate in Cmfrt Partners was t reduce their energy bills. Mnetary savings frm energy usage behavir change Twenty-seven percent reprted that the service prvider tld them hw much mney they culd expect t save by taking the actins n their plan. Recmmendatin Prviders shuld be trained t furnish educatin n ptential dllar savings frm energy actins, as custmers are mst interested in reducing their energy bills. Prgram Satisfactin Mst rated highly Mst custmers said they were very r smewhat satisfied with key aspects f the prgram and the prviders. Recmmendatin There is rm fr imprvement in the percent f custmers wh say they are very r smewhat satisfied with the prgram and prviders shuld wrk n imprved custmer cmmunicatin. Quality f Service Delivery We cnducted 18 weeks f n-site bservatin f service delivery and bserved 81 audits, 36 days f measure installatin, and 12 final inspectins. Observatins were stratified by gas utility, electric utility and cntractr. A detailed data cllectin system was develped t assess whether key steps were cmpleted and tests were cnducted. The quality and cmprehensiveness f services was als assessed. The infrmatin was cllected by prviding bservers with frms and a database in which t recrd the infrmatin. The frms listed the expected steps fr each stage f the prcess the audit, the measure installatin, and the final inspectin. Rating scales will tally the number and percent f steps cmpleted crrectly. 2-21
The bservatins will quantitatively assess the quality f wrk dne and will prvide detailed data n where the prgram is succeeding, where prcedures culd be refined, and where additinal training is needed. We cnducted 288 inspectins f cmpleted jbs. The inspectins prvide a statistically reliable analysis f the quality and cmprehensiveness f Cmfrt Partners jbs. The apprach is described belw. 1. Sample We recruited custmers frm a sample custmers wh cmpleted the telephne survey and expressed willingness fr an inspectin. The survey sample was selected t be representative f the Cmfrt Partners ppulatin. Thrugh the inspectin perid, we reviewed inspectin cmpletin statistics by electric utility, gas utility, cntractr, and cunty t ensure that the inspectins were representative f the prgram s custmers. This apprach had the fllwing advantages. At the end f the survey, we infrmed custmers abut the inspectin, the $50 incentive that is ffered t inspectin participants, and assessed interest in participatin in the inspectin. The survey cnfirmed that the custmers are in the same hme where services were delivered. Custmers wh were nt willing t participate in the survey were unlikely t agree t an inspectin, s the survey prvided an initial screening fr willingness t participate. The survey included custmers wh had Cmfrt Partners service delivery cmpleted apprximately ne year ag. By inspecting these hmes, we were able t determine if the measures held up fr a year after service delivery. We can link the results frm the survey and the inspectins tgether t prvide increased infrmatin frm the study. 2. Recruitment We cntacted, screened, and recruited a sample f husehlds fr a 3 t 4 hur n-site inspectin. We paid clients a $50 incentive in recgnitin f the time and effrt required t keep the inspectin appintment. 3. Visit Prtcl Our inspectin prtcl had the fllwing elements. Data Retrieval We extracted all relevant service delivery data fr each sampled hme, including pre and pst diagnstics, installed measures, and csts. On-Site Inspectin We sent a senir technician and a technician assistant (all with BPI certificatin) t cnduct the inspectin that included the fllwing. Diagnstic testing Inspectin f all installed measures fr final quality and cmpleteness Identificatin f any missed pprtunities Discussin with the custmer f health, cmfrt, and safety issues, as well as any client-related factrs that may have led t exclusin f certain measures. Pst-Inspectin Analysis The data fr each hme will be analyzed in terms f measure selectin, installatin quality, and health and safety issues. Reprting We will cnduct an investment weighted analysis f the quality and cmpleteness f the wrk dne in the prgram. The reprt will furnish infrmatin n the 2-22
effectiveness f the prgram in addressing the needs f each system in the hme and an verall assessment f installed measures. Key indicatrs in terms f Measures will include the fllwing. Percent f Spending Apprpriate (i.e., cnsistent with prgram guidelines) and Gd Quality Percent f Spending Apprpriate but Pr Quality Percent f Spending Inapprpriate Cst f Measures Apprpriate but nt Installed This research will furnish a cmprehensive understanding f what is dne in the field, and why savings gals are r are nt achieved. Mrever, this infrmatin will help prgram managers t refine prgram prtcls in ways that increase the installatin levels fr apprpriate measures, reduce installatin f subptimal measures, and fcus attentin n the key areas fr imprving installatin quality. Prgram Impacts We btained energy usage data frm the six utilities and cnducted weather nrmalized, cmparisn grup adjusted analysis f the energy impacts f the prgram n natural gas and electricity cnsumptin. The cmparisn grup was cmprised f later prgram participants. Billing data analysis methds can be bradly gruped int tw categries huse-byhuse savings analysis and pled analysis. Huse-by-huse analysis: PRISM is an example f the huse-by-huse analysis, where energy usage fr each hme is analyzed fr perids befre and after treatment. Grss savings is calculated fr each hme as the difference between pre and pst treatment weather-adjusted usage. Net savings is calculated by adjusting grss savings by the average change in weather-adjusted usage fr cmparisn hmes. In additinal t PRISM, we utilized a degree day analysis that allws fr a greater percentage f cases t be included. Pled analysis is cnducted using a regressin mdel, where savings are nt estimated fr each hme, but instead the mdel directly estimates the prgram savings as a parameter f the regressin mdel. Fr the target analysis perid, because f the unusual weather patterns, it was particularly imprtant t have mre than ne analytic technique. During the winter f 2011-2012, New Jersey experienced apprximately 20 percent fewer heating degree days than the nrmal, and than the prir winter f 2010-2011. Mrever, the distributin f degree days by mnth was unusual and is likely t have resulted in idisyncratic usage that can have an impact n the precisin f grss and net energy savings mdels. Findings frm the usage impact analysis will be imprtant in making recmmendatins fr prgram treatments. The recmmendatins will address the fllwing. Allwable measures Appliance replacement criteria Spending levels 2-23
We will als cmpare these findings t savings frm ther lw-incme energy efficiency prgrams arund the cuntry, and cmpare health and safety spending t prgrams arund the cuntry. We will cmpare predicted t actual savings and cmpute the savings realizatin rates. We analyzed custmer billing and payment data t estimate the prgram s impact n energy bills, energy burden, and energy bill payment. The fllwing areas were addressed in this analysis. Energy bills: We cmpared pre and pst-treatment bills fr prgram participants. We disaggregated the reductin in bills int the segment that is a reduced custmer payment and the segment that is a reduced USF subsidy. Energy burden: We calculated energy burden and cmpared pst-prgram energy burden t pre-prgram energy burden. USF subsidy: The NJ USF prgram prvides mnthly credits t electric and gas custmers t reduce their electric and gas energy burdens t six percent f husehld incme. We cmpared pre and pst-treatment USF credits fr prgram participants. We estimated the amunt by which the prgram reduced the ratepayer burden by reducing participants usage and bills. We cmpared the percent f custmers wh reach the maximum USF subsidy in the pre and pst treatment perids. Payments: We analyzed the number and amunt f cash and LIHEAP payments made by participants in the year prir t service delivery and cmpared these payments t payments made in the year fllwing service delivery. Cverage rates: We analyzed the bill cverage rate f these payments. Arrears: We analyzed the mean level f arrears and distributin f arrears fr prgram participants ne year after service delivery. We cmpared these arrearages t preprgram levels. Findings and Recmmendatins The NJ Cmfrt Partners Prgram is underging a cmprehensive evaluatin. The research tasks are cmplimentary and tgether will infrm a priritized list f recmmendatins t help the prgram meet its gals in the mst effective manner. Initial research has prvided sme preliminary recmmendatins, but the frthcming findings frm n-site wrk and impact analysis will be critical in understanding whether and hw prgram investments can be mre effective. References APPRISE. 2002. New Jersey Cmfrt Partners Cmprehensiveness Reprt. Princetn, NJ. http://www.appriseinc.rg/reprts/njcp%20cmprehensiveness.pdf. APPRISE. 2006. PPL Electric Utilities Winter Relief Assistance Prgram Evaluatin Reprt. Princetn, NJ. http://www.appriseinc.rg/reprts/final%20ppl%20wrap%20evaluatin %20Reprt.pdf. 2-24