National Grid s New York EnergyWise and Residential Products Programs

Transcription

1 National Grid s New York EnergyWise and Residential Products Programs Final Impact Evaluation Activities Report National Grid Prepared by KEMA, Inc.

2

3 Table of Contents 1. Executive Summary Program Activity Summary Key Study Methods Results Conclusions and Recommendations Program Overview Study Objectives and Summary Evaluation Methodology On-site Sample Design Methodology Lighting (Electric) Thermostats (Gas) DHW Flow Limit Measures (Gas and Electric) DHW Pipe and Tank Insulation (Gas and Electric) Evaluation Results Lighting Lighting Technical Manual Savings Assumptions Lighting On-Site Results Thermostats Thermostat Technical Manual Savings Assumptions Thermostat On-Site Results DHW - Showerheads Showerhead Technical Manual Savings Assumptions DHW - Showerhead On-Site Results DHW - Faucet Aerators Faucet Aerator TRM Savings Assumptions DHW Faucet Aerator On-Site Results DHW Pipe and Tank Insulation Findings and Recommendations...44 A. On-Site Instrument... A-1 B. Energy Star Program Thermostat Phone Survey... B-1 National Grid i

4 Table of Contents List of Exhibits Table 1: Niagara Mohawk EnergyWise and Energy Star Products Savings, Table 2: Gas Savings in Downstate Territories...3 Table 3: Lighting Savings Input Results Comparison...5 Table 4: CFL Per Unit Net Program Impacts...5 Table 5: Lighting Saturation Before and After Program Participation...6 Table 6: Average Number of Sockets per Home by Room and Bulb Type...7 Table 7: DHW Flow Limit Measure Savings Input Results Comparison...9 Table 8: Pipe and Tank Insulation Installation Rate Comparison...9 Table 9: EnergyWise and Energy Star Products Savings Goals versus Achievement ( ) Table 10: Niagara Mohawk EnergyWise and Energy Star Products Savings, Table 11: Gas Savings in Downstate Territories Table 12: Comparison of Sample Frame and Final Sample Table 13: Final Sample by Sampling Group and Region Table 14: Final On-site Recruitment Disposition Table 15: Logger Placement by Room Type Table 16: Final Phone Survey Disposition Table 17: Lighting Short-Term Persistence Rate Results Table 18: Lighting Delta Watts Results Table 19: Lighting Hours of Use Results Table 20: CFL Per Unit Net Program Impacts Table 21: Lighting Saturation Before and After Program Participation Table 22: Average Number of Lighting Sockets per Home by Room and Bulb Type Table 23: Thermostat Use by Fuel Table 24: Thermostat Use by Data Collection Method Table 25: Comparison of Secondary Thermostat Study Results to Technical Manual Assumptions Table 26: Showerhead Persistence Rate Table 27: Showerhead Flow Rates and Usage Table 28: Faucet Aerator Persistence Rate Table 29: Faucet Aerator Flow Rates List of Figures Figure 1: Logger Installation Profile National Grid ii

5 Table of Contents Figure 2: Thermostat Set-points of All Customers Figure 3: Thermostat Set-points of Customers with Reported Setback National Grid iii

6 1. Executive Summary National Grid commissioned a study to evaluate their EnergyWise Electric and Gas programs, the Residential ENERGY STAR Electric Products and Recycling program (Electric Products), and the Residential ENERGY STAR Gas Products programs (Gas Products). The main objectives of this evaluation included: a) quantifying and/or providing further documentation to support the savings claims associated with the measures that generated the most savings, b) provide recommendations on those measures for forward looking adjustments to the Technical Manual to the extent possible, and c) to estimate hours of lighting use by room type as part of the concurrent regional hours of use (HOU) 1 study performed in New York, Connecticut, Rhode Island, and Massachusetts Program Activity Summary Table 1 summarizes the electric and gas savings acquired by the EnergyWise and Residential Products programs in the Niagara Mohawk territory. These summaries were developed from the 2010 and 2011 annual reports, which provide energy savings based on the formulas and assumptions in the Technical Manual. These savings have not been verified through an independent evaluation. The refrigerator/freezer bounty portion of the Electric Products program had an evaluation finalized in early 2013 and therefore was not examined in this study. The EnergyWise gas savings are comprised of programmable thermostats and domestic hot water flow limiting devices (faucet aerators and low flow showerheads) while EnergyWise electric savings are dominated by CFLs and DHW flow limiting devices. 1 The main objective of the HOU study is to estimate hours of use by room type, not lighting type (technology) as many studies have done in the past. The focus on room types assumes that people are likely to use their lights in a given room the same way regardless of the type of bulbs in the room. This study is ongoing at this time and is expected to be completed by the end of The addition of New York brings the number of states involved in this study to four (also including Rhode Island, Connecticut and Massachusetts). National Grid 1

7 Table 1: Niagara Mohawk EnergyWise and Energy Star Products Savings, Measure Category Niagara Mohawk Savings EnergyWise & Residential Energy Star Products Electric Gas kwh % of Savings Therms % of Savings Electric Products Gas Products Thermostat 15, % 40, % Shell measures (Windows) 46, % 9, % Refrigerator/Freezer Bounty 23,375, % 0 N/A Total 23,437, % 50, % EnergyWise CFL 4,108, % - N/A Lighting-Other 1,225, % - N/A DHW - Flow Limit 4,022, % 236, % DHW - Pipe Insulation 160, % 7, % Thermostat - N/A 180, % Total 9,517, % 424, % Total CFL 4,108, % - N/A Lighting-Other 1,225, % - N/A % DHW - Flow Limit 4,022, % 236, % DHW - Pipe Insulation 160, % 7, % Thermostat 15, % 221, % Shell measures (Windows) 46, % 9, % Refrigerator/Freezer Bounty 23,375, % 0 N/A Total 32,955, % 474, % Table 2 summarizes the Gas Products savings in the KEDNY (Brooklyn Union Gas) and KEDLI (KeySpan East - Long Island) territories. While there are some shell measure savings, the majority of gas savings occurs from thermostats. National Grid 2

8 Table 2: Gas Savings in Downstate Territories Brooklyn Union KeySpan East Total % of End Use Measure (Therms) (Therms) Therms Savings Shell Measures (Windows) 715 1,998 2,713 8% Thermostats 14,175 18,632 32,807 92% Program Total 14,890 20,630 35, % 1.2 Key Study Methods There were three primary activities undertaken as part of this study. On-site assessments at the homes of 60 EnergyWise program participants, including the installation of 320 lighting loggers, to estimate hours of lighting use by room type as part of the regional hours of use HOU study 2 and to provide forward looking adjustments to the Technical Manual to the extent possible. A review of secondary research on programmable thermostats to assist National Grid in ensuring that the assumptions and savings that flow from the Technical Manual are consistent with the findings from other studies. Phone surveys of 25 Electric and Gas Products participants who received thermostats through the program to confirm installation and assess thermostat set-points both prior to and after the installation of the new unit. National Grid and DNV KEMA discussed the inclusion of net factors (free ridership and spillover) as part of this study. We decided not explore net-to-gross (NTG) factors as part of this study for the following reasons: We believe that NTG factors for multifamily direct install CFLs, aerators and showerheads are being examined by ERS as part of their contract with Consolidated Edison and Orange and Rockland. While this study may not be definitive, it will provide alternative NTG factors for possible inclusion in the NY TM. The largest producer of electric savings in the EnergyWise and Energy Star programs is CFL lighting. Recent CFL lighting evaluation efforts have identified many challenges to studying and quantifying NTG that have been difficult to overcome despite extensive examination and funding. Challenges include the nature and constantly changing dynamics of the CFL market (especially with the burgeoning LED market), the lack of comparison areas as CFLs have become increasing 2 The HOU study included the installation of 4,642 loggers in 848 homes in Connecticut, Massachusetts, New York (excluding Nassau and Suffolk counties), and Rhode Island. National Grid 3

9 ubiquitous and the difficulty participants have in providing reliable self-reports around CFL purchasing decisions and behaviors. The largest producers of natural gas savings in the EnergyWise and Energy Star programs are from aerators, showerheads and thermostats. However, the combined savings from these measures represent 10% of the total natural gas savings that are being evaluated in other work scopes that KEMA and National Grid have submitted to the DPS. We also believe that a statewide study of spillover is being considered at this time. Particularly for nonparticipant spillover, we believe a statewide study makes a great deal of sense since it is typically assessed at the market level. Such an approach would likely be more cost effective due to cost sharing for a study that could provide market level results. 1.3 Results This section presents the overall results of the evaluation followed by a summary of recommendations. The results and recommendations rest upon the findings of this study and DNV KEMA s vast experience performing on-site visits and lighting logger studies, secondary research reviews, and telephone surveys. Lighting Results Table 3 provides the current Technical Manual lighting inputs and compares them to the evaluation results based on the performance of 60 on-site visits to EnergyWise participant homes. Although the evaluation persistence rate was 86.7%, it is important to remember the on-site visits occurred two years after installation. The pre to post wattage ratio used in the delta watts calculation in the Technical Manual decreased by 0.19 (or 7.5%) due to program CFLs replacing lower wattage incandescent bulbs than assumed; based on customer-reported baseline wattages. A contributor to this decrease in delta watts may be the Energy Independence and Security Act of 2007 (EISA). On January 1, 2012, EISA legislation began phasing out 100-watt incandescent bulbs and on January 1, 2013, 75-watt incandescents were phased out. While the existing inventory of these bulbs could still be sold in stores, no new shipments could be made as of these dates. On January 1, 2014, 60-watt incandescents will also be phased out. The hours of use and coincidence factor results are based on the HOU study findings 3, which reported 2.3 hours per day with ±3.7% precision at the 90% confidence interval. The summer coincidence factor was calculated to be 14% with ±7.1% precision at the 90% confidence interval, while the winter coincidence factor is 18% with ±5.6% precision at the 90% confidence interval. 3 Northeast Residential Lighting Hours-of-Use Study FINAL, NMR Group, Inc. and DNV GL, May 5, 2014, Hours of use results can be found in Table ES-1 on page IX and coincidence factor results can be found in Table 4-3 on page 65. National Grid 4

10 Input In-Service Rate Delta Watts (Pre to Post Wattage Factor) Table 3: Lighting Savings Input Results Comparison Technical Manual/Tracking System Not provided although believed to assume a 100% in-service rate because EnergyWise is a direct-install program. Evaluation Short Term Persistence Rate: 86.7% (n=518) Precision at 90% Confidence Interval ±2.8% (n=449) ±2.3% Hours of Use 3.2 hours per day 2.3 hours/day (n=4,642) ±3.7% Summer Coincidence Factor ±7.1% Winter Coincidence Factor ±5.6% We are able to calculate a gross realization rate at the CFL unit level based on the hours of use, installed average CFL wattage (16.8 watts), and a delta watt factor. In Table 4 we calculate an ex post tracked per CFL unit annual kwh savings estimate of 33.0 kwh 4 with an associated gross realization rate of 66.5%. We did not address net to gross in this study and as such have reflected the current assumption in the Technical Manual in this table (0.90). The kw realization rate is calculated using the installed average CFL wattage and hours of use and the delta watt factor. Table 4: CFL Per Unit Net Program Impacts Parameter Electric Energy (kwh/yr/unit) Electric Demand (W/unit) Ex Ante Tracked Savings 49.6* 42.5* Ex Post Tracked Savings Evaluation Realization Rate (RR) 66.5% 92.5% Evaluation Net-to-Gross Ratio (NTG) Ex Post Net Impact Realization Rate (RR) 59.9% 83.2% *Weighted average calculated based upon the watts of program installed CFLs from the on-site sample, the current TM CFL to baseline factor (2.53) and the current TM CFL Hours of Use estimate (3.2 hours/day). Weighted average kw calculated based upon the watts of program installed CFLs from the on-site sample and the current TM CFL to baseline factor (2.53). Although CFL saturation is not a savings input, it was calculated through the use of the HOU study onsite protocol. Table 5 shows the lighting saturation in the 60 participating homes before and after they 4 Calculated from the product of the delta watts factor (2.34), annual hours of use (2.3 x 365 = 839.5), and CFL wattage (16.8) and dividing it by 1,000 to convert from kilowatts to watts. National Grid 5

11 participated in the program based on customer self-reported baseline bulb types. As the table shows, the EnergyWise Program increased the CFL saturation in the sampled homes from 20.7% to 59.9%. Before program participation, approximately 70% of the sockets in the sampled homes contained an incandescent bulb, while only 21% contained a CFL. After participating in the program, almost 60% contained CFLs, while only 31% contained incandescents. Of the remaining 9% of sockets, 7.6% contain fluorescent bulbs, 1.0% contains halogen bulbs, 0.3% were empty and 0.2% contained LEDs. Table 5: Lighting Saturation Before and After Program Participation Before Program After Program Bulb Type % of % of Quantity Quantity Total Total CFL % % Incandescent % % Tube Fluorescent % % Halogen % % Empty 3 0.3% 3 0.3% LED 2 0.2% 2 0.2% Total 1, % 1, % Table 6 presents the average number of bulbs by room and bulb type. The average home visited in the sample was found to have 18.9 sockets; 18.8 on the interior and 0.1 on the exterior. By bulb type, the average home in the sample had 11.3 CFLs (7.5 program CFLs and 3.8 non-program CFLs), 5.9 incandescent bulbs, 1.4 fluorescents, 0.2 halogens, and 0.03 LEDs. National Grid 6

12 Thermostat Results Table 6: Average Number of Sockets per Home by Room and Bulb Type Location/Type Mean % Program CFLs % Non-Program CFLs % 20.3% By Area Interior % 20.2% Exterior % 50.0% By Room Bathrooms % 13.5% Bedrooms % 31.6% Kitchen % 12.6% Living Room % 40.8% Dining Room % 11.6% Hallway/Foyer % 15.6% Closets % 11.5% Office/Den % 0.0% Other % 0.0% By Bulb Type Program CFLs 7.5 Incandescent 5.9 Non-Program CFLs 3.8 Tube Fluorescent 1.4 Halogen 0.2 Empty Sockets 0.05 LEDs 0.03 Although it is not provided, the Technical Manual likely assumes that the thermostat in-service rate is 100% because EnergyWise is a direct-install program. According to the tracking system, the on-site sample received 32 program thermostats. All 32 units were found installed and operating at the time of the on-site visits. Due to the fact that other studies have often found lower than anticipated savings from programmable thermostats, this study focused on understanding how people are using their programmable thermostats and how their behavior has varied from how they used their old thermostats. The following bullets provide the evaluation results related to customer use of the thermostats that they received through the program. In the phone and on-sites, we were able to gather operating schedules and use of 63 thermostats (58 controlling gas heat and five controlling electric heat) received through the program. Of these National Grid 7

13 units, 33 (30 gas and 3 electric) were programmed with a winter schedule and 21 of these (19 gas and 2 electric) were programmed with a winter schedule that is different from the schedule used with customers previous thermostats. Thirty eight programmable thermostats were assessed in the on-site and phone survey samples. Thirteen of these (34.2%) were programmed with a summer schedule. Eight of the 38 units (21.1%) were programmed with a summer schedule that is different from the schedule used with customers previous thermostats. Secondary Research: Few households use programmable thermostats in a manner that might be associated with energy savings. This study also found that 17.2% (of customers) did not have their programmable thermostat programmed, 12.4% had a programmable thermostat before the new one was installed, and 69.1% either overrode their programmable thermostat settings two or more times per week or had turned their programmable thermostat off. Only nine respondents, or 1.3%, gave responses that might result in savings as a result of the thermostat installation. 5 Secondary Research: The Massachusetts Wi-Fi study concluded that very few electric-heated homes use the set-point functionality, which likely accounted for the low savings estimates. 6 DHW Flow Limit Measure Results Table 7 provides the current Technical Manual showerhead and faucet aerator inputs and compares them to the evaluation results. To better understand baseline flow rate in this evaluation, the flow rate of showerheads and faucet aerators not replaced through the program were gathered. The table presents the averages of these results %202011%20NGRID%202011%20Wi-Fi%20Thermostat%20Pilot%20Evaluation_Final_04SEP2012.pdf. National Grid 8

14 Installation Rate Table 7: DHW Flow Limit Measure Savings Input Results Comparison Input Technical Manual/Tracking System Evaluation Low Flow Showerheads Only Not provided although believed to Short Term assume a 100% in-service rate because Persistence: 80.4% EnergyWise is a direct-install program. (n=51) Baseline Flow Rate (gpm) Installed Flow Rate (gpm) Showers Taken/Day In-Service Rate Baseline Flow Rate (gpm) Installed Flow Rate (gpm) Electric Water Heater Efficiency Gas Water Heater Efficiency DHW Pipe and Tank Insulation Results 3.25 gpm Follows program tracking data on rebated showerhead flow rate. Untreated units observed on-site: 2.51 gpm (n=22) Precision at 90% Confidence Interval ±11.5% ±0.7% 1.75 gpm (n=41) ±0.0% (n=60) ±12.0% Low Flow Faucet Aerators Only Not provided although believed to assume a 100% in-service rate because EnergyWise is a direct-install program. 2.2 gpm Short Term Persistence: 90.4% (n=114) Untreated units observed on-site: 2.00 gpm (n=29) ±7.1% ±0.0% Follows program tracking data on 1.50 gpm (n=103) ±0.0% rebated showerhead flow rate. Low Flow Showerheads and Low Flow Faucet Aerators (n=19) ±0.0% (n=5) ±1.6% Table 8 compares the assumed Technical Manual pipe and tank insulation installations rates and compares them to the evaluation results. Pipe Insulation In-Service Rate Table 8: Pipe and Tank Insulation Installation Rate Comparison Input Technical Manual/Tracking System Evaluation Not provided although believed to Short Term assume a 100% in-service rate because Persistence: EnergyWise is a direct-install program % (n=18) Tank Insulation In-Service Rate Not provided although believed to assume a 100% in-service rate because EnergyWise is a direct-install program. 1.4 Conclusions and Recommendations Short Term Persistence: 100.0% (n=5) Precision at 90% Confidence Interval ±0.0% ±0.0% The EnergyWise and Residential Products study was largely designed to focus on assessing Technical Manual inputs as a piggyback effort to the ongoing regional HOU study. In this manner, while we can National Grid 9

15 compare and contrast our findings with the current Technical Manual, we are not able to provide a revised or adjusted impact estimate of gross savings. At this time, we are not sure of the process in which the DPS and other stakeholders will consider and undertake revisions to the Technical Manual. While we suggest consideration of Technical Manual input adjustments based upon this study, it is important to note that we do not recommend any changes to the formulas that National Grid using to calculate their claimed savings. In this section, we present some recommendations for consideration with the understanding that there may be studies from other NY energy program implementers that might or might not support the findings here and that a possible outcome of this study is that the results are deemed appropriate for a narrow application as opposed to a general one. In this case, this might mean application to direct install programs for specific findings with one possibility being application exclusively to the National Grid EnergyWise and/or Residential Product program savings. Technical Manual savings input items that are consistent with the findings from this study that do not warrant consideration for Technical Manual changes include: Our findings on lighting measures suggest that the implied EnergyWise Technical Manual inservice rate of 100% remains appropriate. The EnergyWise Program is a direct-install program, for which our study observed an 86.7% short term lighting persistence rate upon two years of installation. The program currently performs QA/QC to ensure measures are installed at the time of program participation and our findings seem to suggest a natural failure rate, so we believe 100% in-service rate remains appropriate for use in the Technical Manual. Water heater efficiencies assumed for DHW Flow Limit Measures in the Technical Manual appear reasonable. Despite the small sample size, the current assumed DHW system efficiencies of 0.97 for electric and 0.75 for gas are nearly the same as those observed in this study (0.98 for electric and 0.77 for gas). Pipe and tank insulation installation rates were observed to be 100% in this study, as assumed in the Technical Manual. The installed flow rate for both showerheads and faucet aerators should continue to follow program tracking data on rebated units, as every program installed showerhead found during the on-sites had a flow rate of 1.75 gpm and every program installed aerator found during the on-sites had a flow rate of 1.5 gpm. The assumed number of showers taken per day in our study was reported to be 1.7, which suggests the current assumption of 2.0 remains reasonable. All thermostats purchased through the Residential Products channel or received through the EnergyWise Program were reported or otherwise found to be installed. We suggest that the National Grid 10

16 currently assumed 100% installation rate in the Technical Manual for this measure continue to be used. The heating ESF 7 in the Technical Manual as derived from the Gas Networks study appears reasonable to continue utilizing in the savings calculations. We do suspect that it is at the high end of what might be expected as our assessment of this input is based upon self -reported temperature changes that might be prone to recall issues, overstatement and not reflect override tendencies. Technical Manual savings input items that might be considered for adjustment based on the findings from this study include: Lighting This study calculated a pre to post wattage factor of 2.34 (±2.3% at 90% confidence interval) with a lower bound of 2.29 and an upper bound of Our findings suggest that the 2.53 factor used in the delta watts calculation in the Technical Manual should be considered for revision as this study result is statistically different. The study ratio is estimated from the on-sites which were performed at homes that participated in a direct install program, which may differ from more general CFL lighting installation applications. Therefore, we suggest assessing this result against inputs gathered as part of future studies and making adjustments based on collective results that utilize this studies observation as a part of that assessment. Alternatively, the Technical Manual might be revised to accommodate pre to post wattage ratios for direct install versus other program channels. Based on the HOU study results and the room distribution of installed EnergyWise Program CFLs, a weighted daily HOU average of 2.3 (840 hours annually) was calculated (±3.7% at 90% confidence interval) with a lower bound of 2.2 and an upper bound of 2.4. These results suggest that the current assumption of 3.2 hours per day (1,168 hours annually) in the Technical Manual be considered for revision. Showerheads In our site work, we noted that roughly 14% of the showerheads received by the sample were removed because customers were not happy with their flow rate. While this might be regarded as a persistence issue, we believe that given the nature of the cause of removal, this result warrants consideration of an adjustment to the assumed in-service rate from 100.0% to 86%. 7 The ratio of energy savings resulting from the installation of a programmable setback thermostat to the annual heating (or cooling) energy. National Grid 11

17 In our site work, we noted that untreated showerheads in participating homes had an average flow rate of 2.51 gpm. The Technical Manual assumes a baseline of 3.25 gpm. We are unable to refute the current TM assumed baseline of 3.25 based on information from this study. However, there is evidence that the TM assumption may be overstated as it is based upon an outdated study and recommend that National Grid exercise opportunities to gather residential baseline showerhead flow rates as part of any future studies to help further the determination of an appropriate TM baseline rate. In the interim, we recommend that National Grid utilize the actual flow rates of removed units to calculate showerhead savings. The current federal regulation mandates that that new showerhead flow rates cannot exceed 2.5 gpm, which is consistent with our observed untreated showerhead findings. Additional evidence that the marketplace baseline is evolving in a manner that is consistent with this mandate may warrant a future TM adjustment. Thermostats Our data collection and secondary research suggests that the 3 degree assumption driving the cooling ESF in the Technical Manual may be overstated. Our findings suggest that the use of a more moderate 2 degree setback in the modeling performed to support the cooling ESF would be more reasonable. Broad Conclusions and Recommendations In the section above, we provide specific results and recommendations. If we were to interpret the impacts of the study results in energy savings, our findings suggest some uneven performance among measures of interest as supported through comparisons of study findings to assumptions and inputs contained the Technical Manual. Based upon the evidence built up from these study efforts, the EnergyWise lighting savings flowing from the Technical Manual are likely overstated along with the cooling savings from programmable thermostats. There is also evidence to suggest that showerhead savings may be overstated. To alleviate this concern, we recommend that National Grid use site specific flow rates of removed units in the savings calculation. The measures that we believe are returning accurate impacts based on the Technical Manual include faucet aerators and heating savings impacts from thermostats, as the findings from our study are generally aligned with those from the Technical Manual from which National Grid is currently calculating savings estimates. This study is different from a traditional impact study in that we have not developed specific realization rates for the all of measures examined 8, but rather applied varying levels of rigor to assessing the inputs used in the savings calculations. We believe studies of this nature can be valuable in environments with technical manuals that are collectively derived, exercised and updated in states and regions where similar 8 This study was only able to calculate a realization rate for lighting. National Grid 12

18 evaluation activities can occur concurrently. This study was built upon a larger regional HOU study designed to rigorously assess one of the most uncertain inputs to one of the most important residential measures installed (lighting). As such, this study was able to gather information on other measures of interest as part of the larger study as well as perform more narrow independent efforts designed to cover remaining measure gaps. This effort allowed us to efficiently gather data to compare and contrast with the inputs and assumptions from the Technical Manual. The final result is the ability to maximize evaluation dollars and build evidence around key inputs that might be considered as part of refining the Technical Manual and associated prospective savings. We recommend that any future studies in New York that occur in the residential sector continue the spirit of gathering information available on baseline or installed conditions for use in building evidence and support for confirmation or disconfirmation of the Technical Manual inputs and assumptions. National Grid 13

19 2. Program Overview The EnergyWise Electric and Gas programs operate in the Niagara Mohawk service territory. This program targets multifamily buildings with five to fifty dwellings and is delivered by two primary vendors that work in partnership. Program participants are provided comprehensive energy use assessments and financial incentives for actions that will improve the electric energy efficiency of multifamily buildings. The Residential Energy Star Electric and Recycling program targeted customers who heat or centrally cool their homes with electricity, and offers financial incentives for the installation of Energy Star labeled replacement windows and 7-day programmable thermostats. The program also offered the removal and proper recycling of functioning secondary refrigerators. The Residential Energy Star Gas Products program target customers who heat their homes with natural gas, and offers financial incentives for the installation of Energy Star labeled replacement windows and 7-day programmable thermostats. These programs have been implemented since 2009 and most of them have been authorized to continue operating it through 2015 by the PSC 9. Only the Energy Star Electric Products and Recycling Program was not approved past calendar year These programs are overseen by National Grid and implemented by vendors selected through a competitive bidding process. These programs can largely be considered as resource acquisition programs; hence a focus on evaluating direct energy savings as a result of program installations. Table 9 summarizes each program s combined goals and achievements as captured in the 2010 and 2011 annual reports, which provide energy savings based on the formulas and assumptions in the Technical Manual. These savings have not been verified through an independent evaluation. Across 2010 and 2011, the EnergyWise program overachieved on its gas and electric savings goals with lower cost than budgeted. The Niagara Mohawk Residential Energy Star Electric Products and Recycling Program largely met its electric savings goal. The Niagara Mohawk Residential Energy Star Gas Products program fell short on the gas goal. Downstate, the combined territories of Brooklyn Union (BUG) and KeySpan East fell short of the savings and participant goals. 9 Order Authorizing Efficiency Programs, Revising Incentive Mechanism, and Establishing a Surcharge Schedule, Issued and Effective October 25, DNV KEMA 14

20 Table 9: EnergyWise and Energy Star Products Savings Goals versus Achievement ( ) Parameter Achieved Goal Elec Gas Elec Gas EnergyWise (Niagara Mohawk) Participants (n) 5,432 1,189 5,600 3,600 Net first-year annual (MWh/therms) 8, ,923 2, ,520 Cost w/o shareholder incentive ($) 2,024,681 1,144,764 2,030,692 2,163,848 Residential Energy Star Electric Products and Recycling and Gas Products (Niagara Mohawk) Participants (n) 17,001 1,099 40,451 1,315 Net first-year annual (MWh/therms) 21,097 45,408 22,767 83,588 Cost w/o shareholder incentive ($) 3,039, ,450 9,502, ,837 Residential Energy Star Gas Products (BUG, KeySpan East) Participants (n) N/A 809 N/A 1,788 Net first-year annual (MWh/therms) N/A 31,968 N/A 83,720 Cost w/o shareholder incentive ($) N/A 535,672 N/A 156,500 National Grid s gas service territory overlaps with approximately one-third of their electric territory in upstate New York. Table 10 summarizes the electric and gas savings acquired by the EnergyWise, Residential Energy Star Electric Products and Recycling and Residential Energy Star Gas Products programs in the Niagara Mohawk territory. Measures provided through the EnergyWise programs are contractor installed, while measures provided through the Residential Energy Star Products programs are customer installed. The refrigerator/freezer bounty program had a recently finalized evaluation and therefore was not examined in this study. EnergyWise gas savings are comprised of programmable thermostats and domestic hot water flow limiting devices (faucet aerators and low flow showerheads) while EnergyWise electric savings are dominated by CFLs and DHW flow limiting devices. DNV KEMA 15

21 Table 10: Niagara Mohawk EnergyWise and Energy Star Products Savings, Measure Category Niagara Mohawk Savings EnergyWise & Residential Energy Star Products Electric Gas kwh % of Savings Therms % of Savings Electric Products Gas Products Thermostat 15, % 40, % Shell measures (Windows) 46, % 9, % Refrigerator/Freezer Bounty 23,375, % - N/A Total 23,437, % 50, % EnergyWise CFL 4,108, % - N/A Lighting-Other 1,225, % - N/A DHW - Flow Limit 4,022, % 236, % DHW - Pipe Insulation 160, % 7, % Thermostat - N/A 180, % Total 9,517, % 424, % Total CFL 4,108, % - N/A Lighting-Other 1,225, % - N/A DHW - Flow Limit 4,022, % 225, % DHW - Pipe Insulation 160, % 4, % Thermostat 15, % 221, % Shell measures (Windows) 46, % 9, % Refrigerator/Freezer Bounty 23,375, % - N/A Total 32,955, % 474, % Table 11 summarizes the Residential Energy Star gas savings in the KEDNY (Brooklyn Union Gas) and KEDLI (KeySpan East - Long Island) territories. While there is some shell measure savings, the majority of gas savings occurs from thermostats. DNV KEMA 16

22 Table 11: Gas Savings in Downstate Territories Brooklyn Union KeySpan East Total % of End Use Measure (Therms) (Therms) Therms Savings Shell Measures (Windows) 715 1,998 2,713 8% Thermostats 14,175 18,632 32,807 92% Program Total 14,890 20,630 35, % 2.1 Study Objectives and Summary In scoping this study, we considered the primary objective to be quantifying and/or providing further documentation to support the savings claims associated with the measures that generated the most savings and to provide recommendations on those measures for forward looking adjustments to the Technical Manual to the extent possible. As indicated earlier, the three primary measure types that are producing the vast majority of program savings in these programs are thermostats for gas, lighting for electric, and DHW flow limit measures (aerators and showerheads) for both fuel types. Therefore, it was decided that the evaluation focus on these measure types. During the course of considering possible evaluation approaches, the NY DPS became aware of a regional hours of use study that DNV KEMA was involved in with the NMR Group. The main objective of the HOU study was to estimate hours of use by room type, not lighting type (technology) as many studies have done in the past. The focus on room types assumes that people are likely to use their lights in a given room the same way regardless of the type of bulbs in the room. In light of this, National Grid and DNV KEMA revised the work scope to piggyback it upon the Regional HOU study. The HOU study multifamily sample design was laid out to achieve 90/10 precision by room type within the non-high rise multifamily sector. This included bedrooms, bathrooms, kitchens, living rooms, and other rooms. However, given the nature of how multifamily is defined for the study (any multi-family building with two or more units/apartments - no wall separating units from basement to roof), we believed it was worthwhile to ensure that sufficient multifamily facilities of interest to Niagara Mohawk were available for the final analysis (five to fifty dwelling units). In addition, a larger multifamily sample size provided the Regional HOU Study with increased ability to draw conclusions at the 90/10 level for additional rooms and/or specific specifically fixtures (dimmable, flood/spot, etc.) as well as provided added analysis support should the actual coefficients of variation (CVs) prove to be higher (poorer) than those assumed in the sample design. DNV KEMA 17

23 National Grid and DNV KEMA discussed the inclusion of net factors (free ridership and spillover) as part of this study. We decided not explore net-to-gross (NTG) factors as part of this study for the following reasons: We believe that NTG factors for multifamily direct install CFLs, aerators and showerheads are being examined by ERS as part of their contract with Consolidated Edison and Orange and Rockland. While this study may not be definitive, it will provide alternative NTG factors for possible inclusion in the NY TM. The largest producer of electric savings in the EnergyWise and Energy Star programs is CFL lighting. Recent CFL lighting evaluation efforts have identified many challenges to studying and quantifying NTG that have been difficult to overcome despite extensive examination and funding. Challenges include the nature and constantly changing dynamics of the CFL market (especially with the burgeoning LED market), the lack of comparison areas as CFLs have become increasing ubiquitous and the difficulty participants have in providing reliable self reports around CFL purchasing decisions and behaviors. The largest producers of natural gas savings in the EnergyWise and Energy Star programs are from aerators, showerheads and thermostats. However, the combined savings from these measures represent 10% of the total natural gas savings that are being evaluated in other work scopes that KEMA and National Grid have submitted to the DPS. We also believe that a statewide study of spillover is being considered at this time. Particularly for nonparticipant spillover, we believe a statewide study makes a great deal of sense since it is typically assessed at the market level. Such an approach would likely be more cost effective due to cost sharing for a study that could provide market level results. DNV KEMA 18

24 3. Evaluation Methodology The evaluation methodology was built around assessing three primary measure types, each with its own unique attributes, opportunities and challenges. Lighting, DHW, and thermostats all had attributes assessed as part of the HOU on-site work. This included loggers and baseline data gathering for lighting, program installed and baseline gpm and water heater efficiencies at both DHW treated and untreated homes, thermostat set-points and pre-existing occupant thermostat behaviors. Thermostats also had a supplemental survey performed to assess customer set-points and use. The methods for these study activities are discussed further below. 3.1 On-site Sample Design Methodology The HOU on-site sample was selected from EnergyWise participants and was designed to accomplish two goals. The first was to identify 60 program participants at which the regional HOU logger protocol would be performed. This protocol was utilized in a total of 848 homes in the regional HOU study. The second was to gather information on the other key savings measures for purposes of assessing the Technical Manual assumptions regarding them. These measures include lighting, programmable thermostats, low flow faucet aerators and low flow showerheads. Given what we were seeking to accomplish, the most desirable and efficient sample design would have been one that provided the greatest number of unique measure types of interest to be assessed on-site. However, targeting only sites that received all four of the measures of interest (lighting, low flow aerators, low flow showerheads and programmable thermostats) would have greatly narrowed the sample frame and could have biased the results. To mitigate this potential bias, we decided to divide the sample of 60 sites so that half (30) of the sample had all four targeted measures and the other half (30) had lighting and at least one of the other targeted measures. This approach provided a balance between maximizing exposure to those measure types of interest while acknowledging and making an accommodation for the possibility of differences among participants with fewer and/or different measure installation profiles. Table 12 compares the sample frame and the sample design of 60 homes based on this methodology. As the table shows, the sample was designed to represent the populations within each of the two sampling groups. DNV KEMA 19

25 Table 12: Comparison of Sample Frame and Final Sample Quantity With All Four Targeted Measures Quantity With Two or Three Targeted Measures % of Two or Three Total Sample Frame % of All % of Region Four Total Program Population (N=2,588) Capital % 1, % 1, % Central % % % Mohawk Valley 8 1.9% % % Northeast % % % Northern 1 0.2% % % Western 0 0.0% % % Population Total % 2, % 2, % Sample Design (n=60) Capital % % % Central % % % Mohawk Valley 1 3.3% 1 3.3% 2 3.3% Northeast 1 3.3% % 4 6.7% Northern 0 0.0% 2 6.7% 2 3.3% Western 0 0.0% 2 6.7% 2 3.3% Sample Total % % % Table 13 presents the final achieved sample and shows that it does not differ from the sample design by more than one site in any of the measure/region groups. Table 13: Final Sample by Sampling Group and Region Quantity With All Four Targeted Measures Quantity With Two or Three Targeted Measures % of Two or Three Total Sample Frame % of All % of Region Four Total Final Sample (n=60) Capital % % % Central 2 6.7% % % Mohawk Valley 1 3.3% 2 6.7% 3 5.0% Northeast 1 3.3% 2 6.7% 3 5.0% Northern 0 0.0% 2 6.7% 2 3.3% Western 0 0.0% 2 6.7% 2 3.3% Sample Total % % % Table 14 presents the final disposition of the recruitment calls made for the 60 on-site visits based on the disposition codes provided in The American Association for Public Opinion Research s (AAPOR) Standard Definitions. 10 Based on the algorithms provided in this document we calculate a 12.9% response rate and a 37.8% refusal rate. These are both less than desirable. However, it is important to keep in mind that EnergyWise targets multi-family customers; who are typically renters and are more transient 10 DNV KEMA 20

26 than a typical homeowner. This population may also be more difficult to contact as they are more likely to have less vested interest in the program and subsequent efforts to improve or gather further information on it. Disposition Code Table 14: Final On-site Recruitment Disposition Disposition Description All Four Two or Three 1.1 Completion Total 2.11 Household-Level Refusal Respondent Never Available Household-Level Language Problem Disconnected Number Number Changed Person Not Household Resident Lighting (Electric) Total Customers Called The NY Technical Manual has CFL savings being calculated from an assumed incandescent to CFL ratio of 3.53 (delta watts being a factor of 2.53), based upon the 2008 California Database for Energy Efficient Resources (DEER) update. In reviewing the DEER report, we understand that value rests upon a standard incandescent baseline. The Technical Manual lighting savings approach also uses an assumed operating hours of 3.2 hours a day, a value based upon a 2003 extended logger study of coupon and catalog sales in Massachusetts, Rhode Island and Vermont as well as a study on coupon lighting from Maine. The current coincident factors in the Technical Manual are 0.08 for average summer on-peak (1p-5p) and 0.30 for average winter peak (5p-7p). To assess lighting in this study, we utilized the logger placement protocols developed for the regional HOU study to install 320 lighting loggers in EnergyWise participant homes, as part of the 4,642 loggers installed for the HOU study. Table 15 below shows the distribution of the loggers installed in the homes of EnergyWise participants by room type, while Figure 1 presents the number of loggers that were installed by month and week for the duration of this study. DNV KEMA 21

27 Table 15: Logger Placement by Room Type Loggers Room Type Installed Bedroom 76 Kitchen 62 Bathroom 58 Living/Family Room 57 Dining Room 33 Hallway 11 Foyer 11 Closet 6 Other 3 Office/Den 3 Total 320 Figure 1: Logger Installation Profile The time of use results from the loggers were examined and placed into the regional study to ensure representation of the 5 to 50 unit multifamily complex type in addition to adding greater statistical power to levels of result disaggregation. As part of these visits, we gathered the location and pre-wattage of the EnergyWise installed lighting, as well as other information to support the exploration of DHW and thermostat savings. DNV KEMA 22

28 3.1.2 Thermostats (Gas) Thermostats have been installed through both the Residential Energy Star Products and EnergyWise programs in upstate and through the Residential Energy Star Products programs down state applications. Thermostats are the only measure currently offered through the Residential Energy Star Products program. Thermostats comprise an estimated 50% of total gas savings from the Residential Energy Star Products and EnergyWise Gas Programs for upstate New York, New York City, and Long Island. This made thermostats a particularly important measure type to examine. In preliminary scoping efforts, DNV KEMA suggested that a billing analysis be performed to quantify the savings from this measure. A key parameter of the thermostat savings in the New York Technical Manual comes from a previously performed billing analysis (Massachusetts, 2007) that provided an energy savings factor of 6.8% for heating (effectively, meaning that the savings calculated by that study were estimated to be 6.8% of a home s annual heating energy consumption.) However, we noted that there have been many billing analyses performed in other jurisdictions on this technology that have resulted in indeterminate levels of savings. While the cause of the difficulty in acquiring gas thermostat savings results from billing analyses are largely speculative, two oft surmised causes are the improper use of the thermostat s features and/or the possibility that the user has historically manually adjusted the thermostat which renders the automated setbacks no more efficient than their previous behavior. Therefore, we decided to approach this study as a two pronged exercise. The first was to review secondary research to assist National Grid in ensuring that the assumptions and savings flowing from the Technical Manual are consistent with findings from other studies. The second was to examine two of the primary reasons speculated to diminish expected savings in other studies. Specifically, we sought to confirm installation and gather information on how the thermostat was being used from two independent efforts. The first was at all on-sites with thermostats installed while the second was for a sample of 25 phone surveys directed at Energy Star Gas Products and Energy Star Electric Products and Recycling program participants who purchased thermostats. In each case, we assessed thermostat set-points both prior to and after the installation of the new unit. As part of the on-site visits we were able to assess 32 EnergyWise thermostat installations to supplement the information gathered through the 25 phone surveys. Table 16 presents the final disposition of the phone surveys based on the AAPOR document mentioned above. Using the algorithms provided in this document, we calculated a very reasonable response rate of 43.1% and refusal rate of 6.9%. DNV KEMA 23

29 Table 16: Final Phone Survey Disposition Disposition Code Disposition Description Total 1.1 Completion Household-Level Refusal Respondent Never Available Household-Level Language Problem Disconnected Number Number Changed 11 Total Customers Called 59 The combination of these two items (secondary review and primary data collection), provided a high level check on the thermostat savings flowing from the Technical Manual DHW Flow Limit Measures (Gas and Electric) In our examination of the Technical Manual, the aerator and showerhead savings have algorithms that are well established and rely upon multiple inputs to produce both gas and electric savings. We noted that several inputs in the Technical Manual would be difficult to quantify with greater accuracy than the assumptions currently in place 11. These assumptions are primarily based on three studies, the oldest dating back to 1992 and the newest, and least empirical, dating to However, there were several savings factors that we sought to update (or verify) as part of this study, including water heater efficiency and program treated and untreated showerhead and aerator flow rates (in gallons per minute or gpm). Number of showers per day was asked of customers. At the 60 sites visited, we were able to assess 51 low-flow showerheads and 114 faucet aerators that were installed through the program. Using the information gathered on-site we performed a re-engineering analysis consistent with the Technical Manual approach to estimate overall impacts from these technologies followed by recommendations on how the Technical Manual might be adjusted to reflect our findings DHW Pipe and Tank Insulation (Gas and Electric) For homes that met the sample design criteria and also had hot water pipe wrap and/or tank wrap installed, we sought only to verify installation of these measures. 11 This includes the assumed minutes per shower, shower water temperature and throttle factor for showerheads and duration of use, number of uses and faucet temperature for aerators. DNV KEMA 24