Modeled vs. Actual Energy Savings

Transcription

1 Modeled vs. Actual Energy Savings for Energy Upgrade California Home Retrofits PIER Project PIR Working Paper 3.1 Final Review Draft Rebecca Brown, BKi 7/27/12

2 Table of Contents 1.0 Introduction Overview Summary of Findings Study methods Calculating realized energy savings using utility bills Modeling upgrades in EnergyPro Results Overall and average savings Comparing net realized savings to predicted savings Variability and asset versus operational savings Baseload as a percent of total energy Energy savings and climate zone and home age Interpretation Principal findings Occupant behavior Quality of upgrade installation Additional energy savings measures Inadequacies of identifying base load and seasonal use Energy simulation software issues Conclusions Appendix July 2012 i

3 Modeled vs. Actual Energy Savings for Energy Upgrade California Home Retrofits 1.0 Introduction 1.1. Overview This working paper is part of BKi s 1 PIER (Public Interest Energy Research) project PIR for the California Energy Commission. That project, Technology and Strategies for AB32 Compliance in the Existing Homes Sector, seeks to assess the state s home energy retrofit program effectiveness and compare its results with AB32 2 targets for CO 2 emission reductions. The project is also intended to develop and demonstrate advanced technical approaches and innovative delivery strategies for home retrofit programs to meet the AB32-related California Public Utilities Commission (CPUC) energy savings goals and suggest refinements in present practices that may be needed to reach those goals. This report focuses on a BKi analysis of a sample of home energy retrofit projects to identify the relationship between their reported simulation model-based energy savings predictions and the actual changes in energy use after the retrofit. The accuracy of the reported savings is important in meeting the CPUC s need for reliable estimates of the retrofit program s overall savings delivery, which in turn is a key factor in determining the program s cost-effectiveness and avenues for improvement. This report s findings do not consider every possible source of the model versus realized energy savings differences, but clearly indicates a need for further study and reconciliation Summary of Findings Using a statewide sample of 51 home retrofits in the Energy Upgrade California program with a year of both pre- and post-retrofit billing data, the analysis indicated that the simulation modeling tended to overestimate actual net savings realized over the year following the retrofits. On average for the homes studied, the predicted energy savings were found to exceed the realized savings by nearly half; while the energy models averaged an electric-and-gas BTU savings of about 30% for these sample homes, actual utility bill data showed a reduction of 20%. Both the actual average savings and the model/realized savings differences were substantially higher for natural gas than for electricity. This is most likely due to the retrofit 1 BKi was formerly known as Bevilacqua-Knight, Inc. 2 Assembly Bill 32 is California s 2008 Global Climate Solutions Act, administered by the Air Resources Board in collaboration with other agencies including the Public Utilities Commission. July

4 program s emphasis on water heating and space conditioning largely gas rather than other primarily electricity loads such as lighting, appliances, and other plug loads. Also noteworthy is the high degree of house-to-house variation in the model vs. realized savings differences; the correlation coefficient between the two measures was only in the range of 0.05, indicating virtually no linear relationship. The modeled results were higher than the bill records for most homes, but in some cases that relationship was reversed. These results indicate that the program s reliance on the asset-based savings estimates of the EnergyPro model does not yield an accurate estimate of actual operational savings. 2.0 Study methods 2.1. Calculating realized energy savings using utility bills In order to evaluate the energy savings achieved from a typical retrofit performed as part of Energy Upgrade California, we obtained 12 months (slightly fewer in a few cases) of preupgrade and post-upgrade utility bill data for 51 homes participating in the program throughout the state (in Title 24 climate zones 2, 3, 4, 9, 10, 11, and 12) from October 2010 to July Modeled and billing data were obtained from the utility provider or the contractors data submittals, with the consent of the homeowners. For most homes in the study, a full year of both before and after-retrofit data was used. For the few homes for which a full year of post-upgrade bill data was not available, we compared analogous months from the before and after conditions, for example, February through November pre-upgrade, and February through November post-upgrade. Additionally, in order to calculate average annual usage estimates for homes with fewer than 12 months of data, the kwh and therms usage rates for those homes were scaled up according to the number of months available. For example, if only 10 months of data was available, total kwh and therms were multiplied by 1.2 (12/10) to estimate total annual use. Because home performance is influenced by local weather fluctuations, we adjusted seasonal energy use for pre-upgrade utility bill data to account for variations in heating and cooling uses (in degree days) before and after the upgrade was performed. To isolate the proportion of annual kilowatt-hour and therm usage attributable to seasonal energy demand (the energy use required to cool or heat the home), we estimated and excluded the base load (year-round) energy use in the home for each fuel type using the common billdisaggregation approach: For all months of available pre-upgrade bill data, the total energy usages for the two months with the smallest demand were determined and averaged. This average was identified as the base load for that fuel type. Any energy use exceeding that base load each month was considered seasonal (heat/cool)energy use. To adjust for weather variation before and after the home retrofit, the total number of local heating degree days for natural gas use (and cooling degree days for electricity use) was summed for the pre-upgrade and post-upgrade months for which bill data were available. The seasonal pre-upgrade natural gas use (or electricity use) was multiplied by the proportion of post-upgrade to pre-upgrade heating degree days (or cooling degree days). Because seasonal energy use calculations assume the home is being heated and cooled, this adjustment was not performed for electricity use in homes with no air conditioning system. July

5 Seasonal adjustments for all-electric homes were performed slightly differently. Because these homes use the same fuel type for both heating and cooling, applying only CDD to electricity use would omit the effect of heating demands on the utility bills. Instead, a histogram of the year s month-by-month electricity use, and the two lowest months between the humps of summer cooling and winter heating were identified. CDD were applied to only the summer cooling hump between the two low months, while HDD were applied to only the winter heating months. One of the primary challenges of this research was obtaining utility bill data from utility providers. Because of our close relationship with Sacramento Municipal Utility District, we were able to acquire more and more thorough results for SMUD customers electricity use than for other customers. For this reason, all-electric homes are disproportionately represented in our sample of upgrades (8 out of 51 homes, or about 16%). Because only about 12% of homes in California are all-electric, the overall and average actual and modeled energy use results shown below were weighted according to statewide proportions Modeling upgrades in EnergyPro The California home retrofit programs all used the EnergyPro energy simulation model, which had been developed by the California Energy Commission for use in creating energy rating scores for existing homes. The energy savings predictions were based on EnergyPro models created for each individual home. These models were based on the pre- and postupgrade home conditions reported by the contractor to the program administrator in the Job Reporting Template as part of participation in the Energy Upgrade California program. All models were re-calculated using the most recent version of EnergyPro available at the time of analysis (5.1.7). 3.0 Results 3.1. Overall and average savings According to the weather-adjusted utility bill data, the overall energy reduction for the 51 sample homes was 20% (average 19% savings, 17.0 MMBTU annually per home). Total electricity savings was 10% (average 9% savings or 1,117 kwh annually per home), while natural gas savings were 26% (average 23% savings, 132 therms annually per home). The overall savings do not precisely match average per-home savings. Overall savings were calculated by combining the BTU savings from the weather-adjusted kwh and therms for the full sample of homes, and comparing the gross pre-upgrade and post-upgrade results. This produced an overall BTU reduction of 20%. However, when BTU savings were calculated separately for each individual home, and the percent savings per home was averaged, the average BTU savings was slightly less, at 19%. This indicates that homes with larger percent savings also tended to be larger energy users to begin with. July

6 Base load versus seasonal (HVAC) savings were also calculated. Based on the study s determination of base load versus seasonal energy use, overall base load energy use declined 13% (8% for electricity, 21% for natural gas), while the remaining (i.e., seasonal) energy use declined 27% (16% for electricity, 29% for natural gas). The base load savings can be attributed primarily to water heater upgrades, although other factors such as behavioral changes could also be involved. Similarly, in 11 of the 48 sample homes in which a cooling system was not added, overall electricity use actually increased in the year following the retrofit. This result suggests that additional complicating factors, other than the effect of the home performance retrofit, were affecting pre- vs. post-upgrade energy use in these homes Comparing net realized savings to predicted savings On average, realized percent energy savings were exceeded by predicted modeled savings, by a ratio of about 1.5 to 1. While the energy models showed a total BTU savings of about 30% for these sample homes, actual utility bill data showed a reduction of only 20%. Among the total pool of sampled homes and using the base load energy calculation methods described above, EnergyPro underestimated base load BTU reductions (3% modeled versus 13% realized), but overestimated seasonal reductions (47% modeled versus 27% realized). Discrepancies between energy use predicted by the model and realized in utility bills were also manifested in the total kwh and therms usage and savings, in addition to percent savings. Average modeled pre-upgrade energy use was MMBTU annually per home, while utility bills showed per-home average annual energy use of 84.8 MMBTU. Total annual energy savings predicted by the model were 36.6 MMBTU per home, more than double the average 17.0 annual MMBTU savings manifested on utility bills. A scatter plot of modeled savings versus utility bill savings reveals a very weak correlation between these two results. The r-squared value is 0.06, indicating no statistically reliable predictive power from the model. Similarly weak results are shown when examining kwh and therms savings results separately, as documented in the study s Excel data summary in the Appendix to this report. July

7 Overall, 22% of homes enjoyed bill-documented utility bill savings that exceeded EnergyPro s predicted savings, while 78% had realized savings below modeled savings. Only about a quarter (27%) of homes had realized savings within 10 percentage points of the EnergyPro savings prediction, and for about half as many homes (14%) the utility bill savings were within 5 percentage points of the model Variability and asset versus operational savings There was high variability in realized energy savings among the 51 sample homes. While the average per-home BTU savings was 19%, the standard deviation was 17%, indicating wide variation in actual savings resulting from energy upgrades. This actual variability is due to genuine differences among scopes of work and home conditions as well as possible occupant behavioral changes, and further research is needed to separate such causes. In contrast, the average BTU savings predicted in EnergyPro modeling was much higher at 29%, with a standard deviation of only 9%. This may suggest inaccuracies in the model s estimates of asset-based savings (those attributable only to the physical changes in home performance due to the retrofit, which was the original purpose of the Energy Commission in its development of the model) but can also be in part attributable to changes in other conditions such as home occupancy, further home modifications, and occupants behavioral changes. While prior studies elsewhere have suggested that such operational effects tend to be small, this issue remains for further research Baseload as a percent of total energy Using the definition of baseload and using the calculations described above, the utility bill analysis indicated that 49% of average pre-retrofit BTU usage was attributable to baseload (73% of kwh, 33% of therms), including water heating a major energy use. EnergyPro modeling yielded somewhat different findings, calculating that only 40% of total BTU use July

8 and 52% of electricity use is associated with those baseload end-uses. 35% of modeled therms usage was baseload, which is similar to the utility bill analysis findings. While the calculation methods used in the utility bill analysis show baseload as a higher percent of total use than does EnergyPro, both sources underestimate the role of baseload compared to statewide usage patterns. According to the Energy Commission s RASS ((Residential Appliance Saturation Survey, 2009), 72% of average residential BTU usage in California comes from activities other than space conditioning. This includes 90% of electricity use and 61% of natural gas use. Water heating improvements are often included in home retrofits, although other non-hvac uses are largely excluded from the retrofit programs. The statewide RASS data indicates that HVAC and water heating together account for only about 50% of total BTU, in contrast with the study s finding of about that same proportion without water heating, suggesting that the study s homes may not be typical. The major deficiencies of the month-by-month analysis are that it assumes that no cooling is used in the winter and that no heating is used in the summer (which may not be true for all users, particularly in coastal climates) and that it assumes no seasonal variation in enduses other than space conditioning. As described below, higher temporal resolution of utility bill results (through submetering or tools such as smartmeter data) would allow us to refine and improve our baseload calculations to better reflect actual usage Energy savings and climate zone and home age Though clearer relationships may become apparent when more utility bill data are available and analyzed, the samples of homes examined so far have not revealed definitive trends relating climate zone. Average realized BTU savings for homes in Bay Area climates (CZ 2, 3, 4) were 18%, compared to 21% for the LA Basin (CZ 9, 10) and 18% for the Central Valley (CZ 11, 12). Sample sizes for these groups were small (n = 11, 8, and 32, respectively), so no statistically significant relationships can be derived from these results. Home age seemed to have a larger impact on utility bill savings, though again because sample sizes are small, any perceived relationships should be approached with caution. Homes built before 1978 had an average BTU savings of 22% (n = 34), homes built between 1978 and 1987 averaged 12% savings (n = 12), and homes built after 1987 averaged only 8% savings (n = 5). It is tempting to suggest that these results offer at least some support of the assumption that deeper energy savings are possible in older rather than newer homes, but the very small sample sizes by cohort do not allow a conclusion. 4.0 Interpretation 4.1. Principal findings Two main conclusions can be drawn from these results: First, preliminary analysis suggests that predicted energy savings derived from the energy simulation model used in California may overestimate both the relative and absolute net savings achieved from real-world July

9 home energy upgrades. Predicted savings reported by IOUs to the CPUC may warrant readjustment if realized savings do not meet energy simulation model estimations. A second conclusion is that more data collection and analysis, including examination of smartmeter results and interviews with contractors and customers, is needed to better understand why predicted and actual savings seem to differ so much. Based on these results, in addition to model refinements such as calibration to pre-retrofit energy use data, program policy and design changes may be needed. Such changes, including incentivizing energy efficient behavior after the upgrade, can be designed to help align estimated and realized energy savings. There are multiple possible factors contributing to the discrepancy between estimated savings shown in energy simulation models and actual savings manifested on utility bills. In the remainder of this section we discuss some of those potential sources, and how factors that may reduce actual savings can be addressed in program policy and design Occupant behavior Perhaps the most significant and difficult challenge in isolating the variables affecting energy use patterns in an individual home is occupant behavior. The number of occupants may change over time, or a family may purchase a single large appliance that greatly increases their electricity use. According to a recent smartmeter data analysis, one or two end-use sources (e.g., heated towel rack, game console left in idle mode, decorative fountain pump and lighting) can increase a home s electricity use by as much as 20% or more. 3 Another behavioral phenomenon affecting energy use is takeback, the conversion of some of an energy efficiency savings to comfort or convenience. Occupants may take advantage of the savings achieved from a home performance upgrade by heating the house to a higher temperature (or cooling it to a lower temperature) or being more liberal with their use of lighting and plug load sources. Conversely, because our utility bill analysis indicates that base load energy use declined as much as or more than seasonal use, some part of overall energy savings may be attributable to changes in occupant behavior rather than only effects of the energy upgrade notably water heating savings, a part of baseload. Occupants may have taken personal steps (e.g., turning off excess lights and appliances, washing clothes in cold water) to contribute to energy savings, in addition to investing in a home performance upgrade. These behavioral issues can be addressed either indirectly or directly. An indirect way to motivate occupants to maintain or adopt energy efficient habits after an upgrade has been performed would be to provide monthly reports to residents showing them whether their energy use increased or decreased since the same month of the previous year. Automated Smart Meter utility bill disaggregation analysis could provide gross estimates of how the home energy is being used, and recommend behavioral adjustments to maximize savings for each end-use category. A more direct approach would involve having the rebate or incentive be based on realized, rather than modeled, energy savings, or a combination of 3 High Energy Audits (2012), Home Energy Disaggregation Using Smart Meter Data. July

10 the two. Both approaches require that occupants be empowered through access to data and education about behavioral steps they can use to reduce energy use Quality of upgrade installation Energy simulation models assume that the upgrade measures being implemented are being installed according to manufacturers specifications and in such a way that maximizes energy efficiency value. In real homes, however, this ideal may be difficult to reach. Insulation can be poorly installed, air sealing might be too limited, ducts improperly sized or located, and HVAC equipment inadequately commissioned. Quality installation is essential for optimizing the energy savings achieved from a home energy upgrade. For example, about a 5% gap in insulation coverage in the attic or walls can reduce the overall insulation effectiveness by 30%; hot or cold air readily finds the gaps, even if almost all the insulation is properly installed. 4 This issue should be addressed with an effective upgrade quality control program, coupled with a training and internal quality assurance system requirement for contractors and their personnel Additional energy savings measures Because only program-eligible retrofit measures are reported to Energy Upgrade California, we would not be notified of any non-home performance installations implemented in the home, such as passive plug load reduction measures (e.g., lighting timers, Smart Strips), non-permanent appliance change-outs (e.g., dishwasher or washing machine upgrades), or adding on-site energy generation to a home. Indeed, the EnergyPro model as used does not allow such improvements to enter the energy savings prediction. Of the 51 homes sampled, it was confirmed that two home upgrades included a photovoltaic installation; though this was not directly reported to the program, the installation occurred concurrently with the home performance upgrade, and the result of the installation manifested as negative kilowatt-hour draws from May to July after the upgrade. This improperly increases the apparent energy bill savings attributable to the energy efficiency measures, rendering the bill analysis invalid for a comparison with the model results Inadequacies of identifying base load and seasonal use As noted earlier, one of the surprising results from this utility bill analysis was that base load energy use was reduced as much as or more than seasonal energy use, despite the fact that home performance measures (except water heater improvements) should affect only seasonal use. While this result could be attributable to one of the factors described above, another possibility is that our simplified method for determining base load versus seasonal use is inadequate to properly disaggregate energy use associated with heating and cooling compared to other sources. 4 CBPCA Home Performance with ENERGY STAR contractor training series curriculum (2009) July

11 One of the limitations of the available data is that it is available only by month or billing cycle. This large temporal resolution masks much of the intra-month variation in electricity and natural gas use that could provide better clues about how energy is being used in a home. SmartMeter data that reveal energy use patterns by day, hour, or minute can be used with simple occupant surveys to more accurately identify what proportion of energy use is derived from space heating, cooling, water heating, lighting, large appliances, or stand-by losses Energy simulation software issues Finally, discrepancies between actual and modeled energy use and savings may represent deficiencies in the energy modeling software itself. In kwh and therms, EnergyPro overestimates annual energy savings by a ratio of 2 to 1. This may reflect some limitations built into the software for its original energy asset rating purpose. For example, EnergyPro does not reconcile its estimates of energy use with the actual utility bills. As with other energy simulation models, without such a true-up it tends to overestimate total energy pre-retrofit use. In EnergyPro s case, in this sample of homes that BTU overestimate averaged nearly 50%. Thus any subsequently predicted kwh and therm savings, when translated to annual BTU savings units, is similarly magnified by 50%. The results of our utility bill analysis also shows that, in addition to overestimating annual kwh and therms savings in terms of energy units, it also overestimates the percent savings achieved from a typical upgrade by 1.5 to 1. One possible source of this difference is that EnergyPro was found to significantly underestimate the percent of household energy use attributable to baseload (defined as energy use not associated with space heating or space cooling). Thus, when upgrades that reduce space conditioning needs are implemented, EnergyPro applies those effects to an unrealistically high percentage of the home s energy use, which in turn inflates the overall savings achieved from that upgrade (both in percent savings and total energy units). Larger-sample data and more detailed analysis are needed to better understand the relationship between modeled and actual energy savings resulting from whole-house upgrades. To help ensure accurate savings predictions, energy simulations used in state programs should be calibrated against actual pre-upgrade utility bill results. Preferably the models would incorporate smartmeter data analysis that can better estimate the size of different energy end-uses within the home being examined. The simulation software, in turn, could then scale the effects of upgrade measures according to the individual home s energy use patterns. If space heating and space cooling account for only 30% of a home s energy use, for example, the predicted savings achieved from an upgrade affecting only those uses couldn t exceed 30%. These recommendations apply whether state programs implement energy simulation modeling for each individual participating home or in aggregate based on a sample of participating homes. EnergyPro is designed to be an asset rating software and cannot reasonably capture the full range of possible behaviors (and changes in behavior) exhibited by real occupants. However, from a statewide program implementation standpoint, it matters less who is to blame for the discrepancies between modeled and realized energy savings (be it the software developers, the contractors for performing low-quality work, or the customers for July

12 behaving in unpredictable ways); instead, it matters more that California s energy planners be able to accurately predict (on an aggregate basis) the magnitude of energy savings that can and will be achieved from implementing energy upgrades. Statewide residential energy programs could be designed to accommodate the spectrum of actual behavior, rather than basing predicted energy savings exclusively on models that assume uniform behavior, both among homes and within homes before and after the upgrade is performed. This could be accomplished by eliminating modeling entirely (in favor of a point-based system or incentivizing actual bill results), or by adjusting modeling results on an aggregate level based on bills vs. models analyses such as these. 5.0 Conclusions This study s principal finding is its documentation of the substantial discrepancies between modeled and actual net energy savings. However, our limitations of sample size and data detail only allow that finding to illustrate the need for further investigations to provide more definitive knowledge of the reasons and remedies for model vs. actual home energy retrofit savings differences. If the state is primarily interested in the net energy savings of home retrofit programs, rather than only technical asset-based (gross) savings, the need for such further study carries some urgency. A valuable next step in pursuit of better explanations of this study s findings would be to interview participating customers to help identify behavioral patterns that affect actual energy savings results. Such post-retrofit adaptations could either augment or counteract the energy-saving effects of home retrofits. Other high priority efforts include comparisons among different simulation models, modifying the EnergyPro model to calibrate properly to energy billing data, relaxing default data values where used in favor of actual measured values, using smartmeter data to provide more detailed disaggregations of actual energy use, and testing the possibility that specific (but apparently so far unreliable) energy savings predictions for each home may not be valued enough by most homeowners to require such complex model-based techniques. ### July

13 6.0 Appendix Overall % savings by baseload/seasonal (utility bills vs. EnergyPro model) Baseload Seasonal Overall kwh Th BTU kwh Th BTU kwh Th BTU Utility bills Overall % 8% 21% 13% 16% 29% 27% 10% 26% 20% EnergyPro Overall % 0% 5% 3% 44% 49% 47% 23% 34% 30% Average savings by baseload/seasonal (utility bills vs. EnergyPro model) Baseload Seasonal Overall kwh Th BTU kwh Th BTU kwh Th BTU Utility bills Avg % 5% 18% 10% 6% 6% 20% 9% 23% 19% SD* 29% 32% 23% 54% 85% 33% 19% 20% 17% Avg annual units , EnergyPro model Avg % 0% 4% 3% 39% 49% 47% 20% 33% 29% SD* 4% 8% 6% 29% 17% 14% 14% 11% 9% Avg annual units , , * Not weighted for electric- only Average savings by pre/post (utility bills vs. EnergyPro model) Pre- upgrade Post- upgrade Savings kwh Th BTU kwh Th BTU kwh Th BTU Utility bills Avg % savings % 23% 19% Avg annual units 10, , , EnergyPro model Avg % savings % 33% 29% Avg annual units 12, , , % of total energy use from baseload (pre- retrofit) kwh therms BTU Utility bills 73% 33% 49% EnergyPro 52% 35% 40% Actual (RASS) 90% 61% 72% Difference in % energy savings (utility bills vs. EnergyPro model) not weighted for all- elec kwh therms MMBT Bills % above 16% 28% 22% versus % below 84% 72% 78% model % =< 5% 10% 16% 14% % =< 10% 25% 33% 27% July

14 % >25% 27% 21% 20% avg diff - 12% - 10% - 11% avg diff (abs value) 19% 18% 17% Average modeled vs. utility bill savings (BTU) EP savings Avg bill savings 0% to 20% 17% 20% to 30% 15% 30% to 40% 24% 40% and up 21% Average % BTU savings by climate zone Bay Area (2, 3, 4) LA Basin (9, 10) Valley (11, 12) Utility bills 18% 21% 18% EnergyPro 34% 27% 28% Sample size Average % BTU savings by year built Before After 1987 Utility bills 22% 12% 8% EnergyPro 32% 25% 20% Sample size July