ENERGY INITIATIVE ENERGY EFFICIENCY PROGRAM Impact Evaluation of Prescriptive and Custom Lighting Installations. National Grid. Prepared by DNV GL

Transcription

1 ENERGY INITIATIVE ENERGY EFFICIENCY PROGRAM Impact Evaluation of Prescriptive and Custom Lighting Installations National Grid Prepared by DNV GL Date: September 25, 2015

2 DNV GL September, 2015 Page 2

3 Table of contents 1 EXECUTIVE SUMMARY Key Study Activities Key Study Results Conclusions and Recommendations 4 2 OVERVIEW Purpose and Overview of the Study Program Population Summary 7 3 METHODOLOGY RESULTS Lighting Profiles Hours of Use 22 5 CONCLUSIONS AND RECOMMENDATIONS APPENDIX A: ON-SITE MEASUREMENT, VERIFICATION AND ANALYSIS METHODOLOGY APPENDICES APPENDIX B: DESCRIPTION OF RESULTS AND FACTORS Realization Rates Savings Factors 33 8 APPENDIX C: RATIO EXPANSION Tables Table 1: Summary of 2011/2012 PY Prescriptive Lighting Gross Energy Realization Rates... 2 Table 2: Summary of 2011/2012 PY Custom Lighting Gross Energy Realization Rates... 2 Table 3: Summary of 2011/2012 Prescriptive and Custom Connected kw Realization Rates... 3 Table 4: Summer kw Factors and Other kwh Savings Factors... 3 Table 3: 2011 and 2012 EI Program Savings... 8 Table 6: 2011 and 2012 EI Lighting Program Activity... 8 Table 5: 2011/2012 Custom and Prescriptive Lighting Final Sample Design... 9 Table 8: Final On-site Recruitment Response and Refusal Rates Table 7: Summary of 2011/2012 PY Prescriptive Lighting Gross Energy Realization Rates Table 8: Summary of 2011/2012 PY Custom Lighting Gross Energy Realization Rates Table 9: Summary of 2011/2012 Prescriptive and Custom Connected kw Realization Rates Table 10: Summer kw Factors and Other kwh Savings Factors Table 11: Summary of Hours of Use vs. Tracking and Tech Manual Table 12: Calculation Example Result Summary Table 13: Tracking Pre-Retrofit Condition Table 14: Tracking Proposed Condition Table 15: Input for Site Specific Holidays Table 16: Logger Profile Summary Table 17: On-Site Installed Condition Table 18: On-Site Pre-Retrofit Condition Table 19: Adjusted Gross On-Site Savings Table 20: General Heating and Cooling COP Assumptions Table 21: Summary of Results and Factors DNV GL September, 2015 Page i

4 Figures Figure 1: Overall Evaluation Plan... 1 Figure 2: Summer Weekday Lighting Profile... 4 Figure 3: Overall Evaluation Plan... 7 Figure 4: Scatter Plot of Prescriptive Evaluation Results for Annual kwh Savings Figure 5: Scatter Plot of Custom Lighting Evaluation Results for Annual kwh Figure 6: Weekday and Weekend Lighting Profiles (All Sites) Figure 7: Summer Weekday Lighting Profile (All Sites) Figure 8: Weekday and Weekend Lighting Profiles (Industrial) Figure 9: Summer Weekday Lighting Profile (Industrial) Figure 10: Weekday and Weekend Lighting Profiles (Retail) Figure 11: Summer Weekday Lighting Profile (Retail) Figure 12: Weekday and Weekend Lighting Profiles (Office) Figure 13: Summer Weekday Lighting Profile (Office) Figure 14: Weekday and Weekend Lighting Profiles (Hospital) Figure 15: Summer Weekday Lighting Profile (Hospital) DNV GL September, 2015 Page ii

5 1 EXECUTIVE SUMMARY This document summarizes the results of an evaluation of National Grid s 2011 and 2012 Energy Initiative (EI) Program in the Niagara Mohawk territory. The EI Program provides rebates for the installation of energy-efficient measures for large commercial and industrial (C&I) customers. The primary objective of this evaluation is to quantify the gross annual energy and summer demand impacts of lighting measures installed through the EI program. The savings and factors of interest to the study includes the coincident summer on-peak factor, connected kw savings and realization rate, kwh savings and realization rate, percent on peak kwh, summer demand HVAC interactive effect factor and kwh HVAC interactive effect factor. The study was designed to utilize on-site verification and monitoring to assess gross impacts. The evaluation was designed to achieve ±10.0% at the 90% confidence level for gross energy (kwh) savings. 1.1 Key Study Activities Figure 1 below presents the overall evaluation plan, which includes the activities conducted for the M&V logger study on 2011 and 2012 EI lighting participants. The logging study was performed at a statistically selected sample of 63 site visits with an average three months of time of use lighting logging. Figure 1: Overall Evaluation Plan Lighting only Logging Study (2011PY and 2012PY) Prescriptive Lighting Stratified Sample Design (32 Sites) Custom Lighting Stratified Sample Design (31 Sites) On-Site Recruitment, M&V Visit Performance Loggers in Field (~3 Months) Final Gross kwh and Coincident kw Program Estimates of Impacts 1.2 Key Study Results Table 1 and Table 2 summarize the energy savings results of the on-site M&V analysis for the EI Program for Prescriptive and Custom, respectively. Results are presented by adjustment factors to demonstrate which computational inputs were influential in the difference between the tracking system estimate of savings and our gross on-site estimate. These factors are further described in Section 3: Methodology. This table presents these results by nature of discrepancy through an incremental ratio, which shows the percent difference between each level of adjustment. The Gross Realization Rate is presented as the cumulative ratio, which shows the percent difference between the evaluated gross savings estimate and the tracking estimate. Showing the results in this manner DNV GL September, 2015 Page 1

6 allows one to see the relative change from one adjustment to another as well as the overall accumulated change relative to the tracking estimate. The annual energy savings gross realization rate for prescriptive lighting without controls was found to be 90.9% with HVAC interactive effects included. The relative precisions for the final impact estimate with interactive is ±13.6%. The primary driver of the realization rate was the decrease in operating hours observed through the on-site M&V, which caused a 10% drop in energy savings relative to that captured in the previous quantity adjustment. Table 1: Summary of 2011/2012 PY Prescriptive Lighting Gross Energy Realization Rates Prescriptive Lighting Only (n=32 Sites) kwh Incremental Realization Ratio Relative Precision at 90% Confidence (±) Tracking 31,146,989 N/A N/A Documentation Adjustment 31,144, % 0.0% Technology Adjustment 29,983, % 4.5% Quantity Adjustment 29,458, % 1.9% Operational Adjustment 26,335, % 11.7% HVAC Interactive Adjustment 28,308, % 2.5% Gross kwh Realization Rate 28,308, % 13.6% The annual energy savings gross realization rate for custom lighting without controls was found to be 92.4% with HVAC interactive effects included. The relative precisions for the final impact estimate with interactive is ±7.0%. Similar to prescriptive above, the primary driver of the realization rate was the decrease in operating hours observed through the on-site M&V, which caused a 6% drop in energy savings relative to that captured in the previous quantity adjustment. Table 2: Summary of 2011/2012 PY Custom Lighting Gross Energy Realization Rates Custom Lighting (n=31 Sites) kwh Incremental Realization Ratio Relative Precision at 90% Confidence (±) Tracking 61,681,541 N/A N/A Documentation Adjustment 61,215, % 1.5% Technology Adjustment 61,447, % 0.5% Quantity Adjustment 60,524, % 1.7% Operational Adjustment 56,747, % 5.8% HVAC Interactive Adjustment 56,969, % 2.3% Gross kwh Realization Rate 56,969, % 7.0% Table 3 presents the summary of prescriptive and custom gross connected kw realization rates. The prescriptive lighting connected kw gross realization rate is 95.9% with a relative precision of ±3.9%. DNV GL September, 2015 Page 2

7 The prescriptive connected kw realization rate was less than 100%, which is the result of small downward adjustments on both technology (delta watts) and quantity installed. The custom connected kw gross realization rate is ±101.4% with a relative precision of 3.6%. Table 3: Summary of 2011/2012 Prescriptive and Custom Connected kw Realization Rates Prescriptive Custom Connected kw kw Incremental Realization Ratio Relative Precision at 90% Confidence (±) kw Incremental Realization Ratio Relative Precision at 90% Confidence (±) Tracking 6,105 N/A N/A 12,348 N/A N/A Documentation Adjustment 6, % 0.0% 12, % 2.0% Technology Adjustment 5, % 2.8% 12, % 1.6% Quantity Adjustment 5, % 1.8% 12, % 2.4% Gross Connected kw Realization Rate 5, % 3.9% 12, % 3.6% Table 4 shows the remaining factors of interest from the on-site M&V work for both prescriptive and custom lighting measures. Note that all precisions in the table are calculated at the 90% confidence level. The estimated summer coincidence factor from this study is 0.79 for prescriptive, and 0.80 for custom. The evaluated values for the kw HVAC interactive effect factor are 1.15 for summer kw and 1.06 for annual kwh for prescriptive. These reflect the percent of gross summer peak kw and gross kwh savings that are due to interactive effects. For custom, both factors remain near 100% since interactive effects are integrated into the custom tracking savings estimates. Table 4: Summer kw Factors and Other kwh Savings Factors Prescriptive Custom Other Ratio Estimation Results Factor (n=32) 90% Confidence Interval (+/-) Factor (n=31) 90% Confidence Interval (+/-) Summer kw Factors Coincidence Factor 79.0% 9.6% 80.4% 8.3% HVAC Interactive Effect Factor 114.8% 3.5% 99.3% 4.9% KWh Factors HVAC Interactive Effect Factor 106.3% 2.5% 100.4% 2.3% Average Hours of Use 90.0% 11.7% 93.9% 5.8% % On Peak KWh 63.5% 5.7% 53.8% 6.2% Figure 2 presents the weighted percent of on time for non-holiday summer weekdays for both prescriptive and custom lighting combined. The 83% noted in hour 17:00 (5 p.m.) is close to our DNV GL September, 2015 Page 3

8 coincident factor estimate for both prescriptive and custom based upon the metering performed in this study. Figure 2: Summer Weekday Lighting Profile 1.3 Conclusions and Recommendations This impact study included the performance of on-site M&V work with re-engineering analysis to derive results that have traditionally not been achievable through other methods (e.g., the determination of coincidence factors, kw and kwh Interactive factors, connected kw and kwh realization rates, load shapes, and average hours of use). The M&V based assessment of prescriptive lighting without controls has a realization rate of 91%, while the custom lighting without controls realization rate was 92%. The following are recommendations based upon the findings from this study. The tracking lighting hours of use assumptions appear to be high in general with hours of use realization rates of 90% and 94% for prescriptive and custom lighting, respectively. However, the tracking hours of use estimates, which are based on vendor estimates, are closer than the default hours of use estimates for many building types in the New York Tech Manual. As such, while there is evidence from this study that the Tech Manual hours might be underestimating actual usage, we encourage National Grid and the DPS to compile added evidence from other New York Program Administrators ( PAs ) to be sure this trend holds across other territories before making Tech Manual revisions on this matter. One particular building type of note was hospitals. Both the tracking estimates and Technical Manual estimates for hours of use in hospitals are found to be very high. One potential reason for this is that many lighting fixtures that are going into hospitals are not being installed exclusively in common areas or in functional areas unique to hospitals. A good DNV GL September, 2015 Page 4

9 portion of lighting installations are being done in medical offices and exam rooms, which follow more of an office type of operating schedule. For grocery sites, the evaluation found hours to be higher than the Technical Manual. Since grocery lighting is dominated by the sales floor, which corresponds with hours of operation, grocery hours tend to be higher. These specific observations apply to a small sample of the projects in this study, but it is important to ensure that vendors take space type into account when estimating hours of use rather than only building type. Likewise, if Technical Manual hours of use estimates are adopted by National Grid, consider the proportion of lighting fixtures that are not going into high use, common areas when deciding on which building type hours to use. Hours of use should be space dependent rather than building dependent. The New York Technical Manual currently assumes a summer kw coincidence factor of 1.0 for commercial indoor lighting measures, which we regard as very high. The estimated summer coincidence factor from this study is approximately 0.8 for both prescriptive and custom. Currently, National Grid is using a 0.8 coincidence factor for custom lighting, which we recommend they continue to use. However, we would suggest that National Grid work with DPS to combine this data with that from other New York PAs to inform this value for prescriptive lighting. Currently, National Grid uses HVAC interactive estimates of 1.19 for summer kw and 1.07 for annual energy savings for prescriptive lighting. These values are proxies that represent average factors from the table of HVAC interactive factors by Building Type in the Tech Manual. This study found the summer kw HVAC Interactive Effect Factor to be 114.8% (1.148), and the kwh interactive factor to be 106.3% (1.063). We would suggest that National Grid work with DPS to combine this data with that from other New York PAs to inform these values. Currently, National Grid uses custom engineering calculations to estimate HVAC interactive effects for custom lighting. This study found that both the summer kw and annual energy savings estimates for HVAC interactive effects are being calculated accurately, with realization rates of 99.3% and 100.4%, respectively. We recommend continued use of custom engineering calculations to estimate HVAC interactive effects for custom lighting. National Grid employs a post-inspection process for lighting projects in which a follow-up visit is performed by company representative. This visit typically includes a visual inspection of installed equipment. DNV GL supports this process, and encourages continued and improved implementation of it. This study did not find many significant quantity discrepancies, which is an indication that the post-installation process is providing benefit. DNV GL believes that the process could be improved by doing more to collect and verify the reasonableness of the assumed hours of use during this post-installation visit. The project documentation provided for this impact evaluation was generally good. We recommend continuing to collect as much information as possible during the implementation process. Specifically, the following pieces of documentation are needed to better evaluate projects following installation: Spreadsheet savings calculations that match tracking estimates, DNV GL September, 2015 Page 5

10 Description of pre-installation conditions if a retrofit project, post-inspection report, TA study (if custom), details of annual operating hours used by space type or location. We recommend that future impact evaluations prescriptive lighting use an error ratio of 0.5 for energy when developing on-site metering sample sizes. The error ratio we recommend for targeting demand savings is a 0.2 error ratio. Likewise, we d target an error ratio of 0.4 for estimating on-site metering sample sizes for custom lighting, since the custom engineering estimates tend to produce results with less variability than prescriptive. Consider implementing a rolling evaluation framework, which would provide ongoing monitoring and feedback into the lighting programs. A continuous evaluation approach could benefit the program by providing fast feedback by targeting certain segments of interest such as the largest projects or specific sectors or technologies. DNV GL September, 2015 Page 6

11 2 OVERVIEW This document summarizes the results of an evaluation of National Grid s Energy Initiative (EI) Program in the Niagara Mohawk territory. The EI Program provides rebates for the installation of energy-efficient measures for large commercial and industrial (C&I) customers. Key measure types installed through the program include lighting, lighting controls, energy management systems (EMS), economizer controls and air-compressors. 2.1 Purpose and Overview of the Study The primary objective of this evaluation is to quantify the gross annual energy and summer demand impacts of lighting measures installed through the EI program. The savings and factors of interest to the study includes the coincident summer on-peak factor, connected kw savings and realization rate, kwh savings and realization rate, percent on peak kwh, summer demand HVAC interactive effect factor and kwh HVAC interactive effect factor. The study was designed to utilize on-site verification and monitoring to assess gross impacts. The evaluation was designed to achieve ±10.0% at the 90% confidence level for gross energy (kwh) savings. Figure 3 below presents the overall evaluation plan, which includes the activities conducted for the M&V logger study on 2011 and 2012 EI lighting participants. The logging study was performed at a statistically selected sample of 63 site visits with an average three months of time of use lighting logging. Figure 3: Overall Evaluation Plan Lighting only Logging Study (2011PY and 2012PY) Prescriptive Lighting Stratified Sample Design (32 Sites) Custom Lighting Stratified Sample Design (31 Sites) On-Site Recruitment, M&V Visit Performance Loggers in Field (~3 Months) Final Gross kwh and Coincident kw Program Estimates of Impacts Lighting logger data collection began in the spring of 2013 and continued through the summer of Most sites had loggers installed for approximately three months, which included some summer months. 2.2 Program Population Summary Table 5 below provides a summary of the 2011 and 2012 EI Program population. This summary breaks out savings for prescriptive lighting with and without controls, and all other non-lighting measures. Combined, the two years of activity tracked 195,363 MWh of annual energy savings (accumulated from 115,581 MWh in 2010 and 79,781 MWh in 2011). Savings related to lighting measures without controls accounts for 69% of tracked savings in 2011, 60% in 2012, and 65% across both program years. Overall, lighting measures and controls are the focus of a majority of program savings, with 70% of total savings installed across years. DNV GL September, 2015 Page 7

12 Program Track Custom Prescriptive Table 5: 2011 and 2012 EI Program Savings Category Total kwh Savings Total Lighting 49,124,705 27,254,962 76,379,667 Lighting Controls 1,387,945 1,171,849 2,559,794 Non Lighting 28,004,505 26,516,432 54,520,937 Lighting 30,377,954 20,478,246 50,856,200 Lighting Controls 4,609,643 1,610,626 6,220,270 Non Lighting 2,076,699 2,748,996 4,825,695 Total kwh Savings 115,583,462 79,783, ,362,562 As discussed earlier, the on-site M&V element of this evaluation focused on 2011 and 2012 prescriptive and custom lighting measures that have been installed in projects without lighting control installations. Therefore, the population was further trimmed down to include only these accounts. This group is comprised of 856 accounts as shown in Table 6. Note that for custom, the evaluation team had to split the lighting savings into primarily lighting systems and primarily lighting controls. Therefore, the custom lighting savings in the table below does include a small percentage of lighting controls savings that could not be stripped out. Table 6: 2011 and 2012 EI Lighting Program Activity Program Name Track Population KWh Savings Large Industrial Prescriptive ,350 Mid-sized Commercial Prescriptive ,191,638 Total Prescriptive ,146,989 Large Industrial Custom 21 27,222,998 Mid-sized Commercial Custom ,458,543 Total Custom ,681,541 Total All Lighting ,828,530 3 METHODOLOGY To estimate gross savings for prescriptive lighting, we performed M&V work with time of use metering at a statistically selected sample of EI program participants. The New York Evaluation Plan Guidance for EEPS Program Administrators 1 provides guidelines for statistical based evaluation work in New York and advises estimating gross savings at ±10% at 90% confidence. DNV GL used Model-Based Statistical Sampling ( MBSS ) methodologies to inform a sample design that met these targets. The final sample design, shown in Table 7, resulted in the selection of 63 site visits of which 32 are prescriptive and 31 are custom sites. 1 _Guidance_Aug_2013.pdf DNV GL September, 2015 Page 8

13 The final stratified sample design is provided in the table below. The first five columns in the table provide the program, project track, stratum number, the cut point used to allocate sites to each stratum, and the number of projects in each stratum. The final three columns show the total tracking savings in each stratum, the number of sample points that were randomly selected from each stratum, and the final case weights. Our sample unit was a project that included all program activity at an account level, as opposed to using application level information as the sample unit. This approach provides an opportunity to cover all lighting systems applications at each site visited as opposed to only evaluating portions of activity that might be reflected in a single application from a site. Table 7: 2011/2012 Custom and Prescriptive Lighting Final Sample Design Program Track Stratum C&I < 2MW Custom Prescriptive Max Savings Projects Total Savings Sample Final Case Weights 1 128, ,581, , ,860, , ,557, , ,459, , ,480, , ,818, , ,943, , ,949, , ,713, Ind > 2MW Custom Prescriptive 2 1,060, ,583, ,965, ,892, ,719, ,034, , , , , , , , , Recruitment for this sample was performed by experienced DNV GL phone recruiters. The backup sample points were used to replace primary sample points that had either refused or had been called five times or more and had not responded to messages. The final response and refusal rates experienced are provided in Table 4 below. The response rate calculated in Table 8 below includes all customers that were eligible and refused the on-site or were eligible and unable to be reached (noncontact). Response Rate 1 is calculated to be 72%. Refusal Rate 1 is calculated to be 12%. We consider a Response Rate 1 of 72% and a refusal rate of 12% to be reasonable. DNV GL September, 2015 Page 9

14 Table 8: Final On-site Recruitment Response and Refusal Rates Disposition Description Number/Rate Complete 62 Refused - Eligible 10 Non-contact - Eligible 13 Total Contacts 86 Response Rate % Refusal Rate % The response rate is an indicator of potential bias associated with sample-specific estimates of population parameters. This means that a low response rate could indicate a self-selection bias of potentially better performing projects, thus inflating the realization rates and savings results. A response rate of 72% suggests that the chance of bias is minimal in general. We examined all noncontacts in our sample and they all appear to be instances where the business was present (e.g., dispositions where we left messages that were not responded to (voic , other), where no one picked up the phone, or we spoke with someone who passed us on to another individual). Data collection performed at all on-sites included physical inspection and inventory, interview with facility personnel, observation of site operating conditions and equipment, and the installation of metering for roughly 3 months. At each site, the DNV GL team performed a facility walk-through that focused on verifying the post-retrofit or installed conditions of each program installed lighting measure in addition to HVAC equipment information to assess interaction. The NY DPS evaluation plan guidance suggests that the National Action Plan for Energy Efficiency ( NAPEE ) Model Energy Efficiency Impact Evaluation Guide provides generally acceptable evaluation approaches for New York. The method for on-sites with metering adheres to International Performance Measure and Verification Protocol ( IPMVP ) Option A as discussed in the document cited above. Specifically, the method employed uses a combination of stipulated factors (wattage) as well as measurements of key factors (i.e., quantity and hours of use) to calculate savings in an engineering model. A full description of the monitoring, verification and analysis methodology is provided in Appendix A: On-site Measurement, Verification and Analysis Methodology Appendices. The evaluation team installed approximately 550 loggers for this study, or approximately 9 loggers per site. Larger sites had about 15 loggers installed on average, while the smaller sites had fewer. When allocating multiple loggers, DNV GL considered the combination of lighting type and hours of use as unique usage areas. More loggers were installed to capture the more significant usage areas with respect to connected wattage and hours. This approach allows for better logger coverage to spaces which contribute the most to energy savings. An 8,760 hourly spreadsheet analysis was developed for each logger series and was used to estimate hourly energy use and diversified coincident peak demand for all lighting sites. A spreadsheet engineering model was used to develop all savings estimates and factors of interest for each sampled site. A typical meteorological year (TMY3) dataset of ambient temperatures for the location closest to each site was used for all interactive savings analyses. The measure level analysis was performed in a manner that allowed the determination of impacts at each primary discrepancy that might cause the gross savings to differ from the tracking savings. Performing the analysis in this manner also allows us DNV GL September, 2015 Page 10

15 to report impacts at each level of adjustment. Each of these adjustments, or discrepancies, is described below: Documentation Adjustment: The Documentation Adjustment reflects any change in savings due to discrepancies in project documentation. Evaluators recalculated the tracking estimates of savings using all quantities, fixture types/wattages, and hours documented in the project file. All tracking system discrepancies and documentation errors are reflected in this adjustment. Technology Adjustment: The Technology Adjustment reflects the change in savings due to the identification of a different lighting technology (fixture type and wattage) at the site than represented in the tracking system estimate of savings. Quantity Adjustment: The Quantity Adjustment reflects the change in savings due to the identification of a different quantity of lighting fixtures at the site than presented in the tracking system estimate of savings. Operational Adjustment: The Operational Adjustment reflects the change in savings due to the observation or monitoring of different lighting operating hours at the site than represented in the tracking system estimate of savings. HVAC Interactive Adjustment: The HVAC Interactive Adjustment reflects changes in savings due to interaction between the lighting and HVAC systems among the sampled sites. Generally, these impacts cause a heating penalty and a cooling credit. This adjustment reflects impacts from electric heating and/or cooling. Other fuels were also calculated and reported on a per kwh saved. All savings results were then expanded to reflect program level impacts through ratio estimation procedures. Please see Appendix A: On-site Measurement, Verification and Analysis Methodology Appendices for a detailed discussion of the full analysis methodology and Appendix C: Ratio Expansion for a detailed discussion of the expansion analysis. 4 RESULTS This section of the study focused exclusively on prescriptive and custom lighting installed without controls installed in the 2011 and 2012 EI Program. There are six specific results of interest from this study activity, which are listed below. We provide program level realization rates for two of these results (kwh and connected demand (kw)). We also provide peak demand (kw) coincidence and interactive factors at the time of summer peak as well as a kwh interactive factor, average hours of use, and percent on-peak savings. Where appropriate, we provide precisions around each result at the 90% confidence interval. The relative precision for the gross realization rate was calculated using iterative propagation of error. That is, the error for each adjustment was carried through into the next adjustment, according to the formula: 1) where +2 is the current adjustment and is the relative precision. A detailed description of all savings factors is presented in Appendix B: Description of Results and Factors. DNV GL September, 2015 Page 11

16 Summer kw Coincidence Factor This is the percentage of the connected kw savings coincident with the summer peak period, which is defined as hour ending 5 pm on the hottest day, excluding holidays (in this case, July 17, 2012 at hour ending 5 p.m.). Summer Demand (kw) HVAC Interactive Effect Factor This is an adjustment factor applied to the gross connected kw savings that are due to interactive effects during the summer onpeak period. Connected kw Realization Rate This is the ratio of the evaluated connected kw savings divided by the tracking connected kw savings. This realization rate is broken out by various adjustments, including documentation, technology and quantity. Average Hours of Use Realization Rate This is the ratio of the evaluated average annual hours of use divided by the tracking estimate for hours of use for lighting measures based on lighting logger data. kwh Realization Rate This is the ratio of the evaluated gross energy savings divided by the tracking gross energy savings. This realization rate is broken out by various adjustments, including documentation, technology, quantity, hours of use and interactive. kwh HVAC Interactive Effect Factor This is this is an adjustment factor applied to the gross kwh savings that are due to interactive effects. Percent On-Peak Energy Savings The percentage of energy savings that occur during onpeak hours defined as non-holiday, weekdays between 7 am and 11 pm. DNV GL September, 2015 Page 12

17 Figure 4 presents a scatter plot of the prescriptive lighting evaluation results versus tracking savings for annual energy savings (kwh). A one-to-one reference line is plotted as a dashed line on the diagonal of the figure. In addition, the final realization rate is plotted as a solid dark line reflecting the average savings-weighted realization rate of all sample points. We also show the final gross realization rate of 90.9% as a line. In general, the scatter points tend to cluster near the dashed line with no anomalous outliers. There is a downward pressure as evidenced by the number of points below the dashed line, which resulted in the 91% realization rate. Figure 4: Scatter Plot of Prescriptive Evaluation Results for Annual kwh Savings DNV GL September, 2015 Page 13

18 Figure 5 presents a scatter plot of the custom lighting evaluation results versus tracking savings for annual energy savings (kwh). A one-to-one reference line is plotted as a dashed line on the diagonal of the figure. As with the chart above, the final realization rate is plotted as a solid dark line reflecting the average savings-weighted realization rate of all sample points. We also show the final gross realization rate of 92.4% as a line. The custom scatter plot shows a very closely clustered around the dashed line, which is an indication that there is little variability in the results. Therefore, we should expect a result with better precision estimates versus the prescriptive results. Figure 5: Scatter Plot of Custom Lighting Evaluation Results for Annual kwh Table 9 and Table 10 summarize the energy savings results of the on-site M&V analysis for the EI Program for Prescriptive and Custom, respectively. Results are presented by adjustment factor to demonstrate which computational inputs were influential in the difference between the tracking system estimate of savings and our gross on-site estimate. These factors are further described in Section 3: Methodology. This table presents these results by nature of discrepancy by providing the incremental ratio, which shows the percent difference between each level of adjustment. The Gross Realization Rate is presented as the cumulative ratio, which shows the percent difference between the evaluated gross savings estimate and the tracking estimate. Showing the results in this manner allows one to see the relative change from one adjustment to another as well as the overall accumulated change relative to the tracking estimate. DNV GL September, 2015 Page 14

19 The annual energy savings gross realization rate for prescriptive lighting without controls was found to be 90.9% with HVAC interactive effects included. The relative precision for the final impact estimate with interactive is ±13.6%. The primary driver of the realization rate was the decrease in operating hours observed through the on-site M&V, which caused a 10% drop in energy savings relative to that captured in the previous quantity adjustment. Table 9: Summary of 2011/2012 PY Prescriptive Lighting Gross Energy Realization Rates Prescriptive Lighting Only (n=32 Sites) kwh Incremental Realization Ratio Relative Precision at 90% Confidence (±) Tracking 31,146,989 N/A N/A Documentation Adjustment 31,144, % 0.0% Technology Adjustment 29,983, % 4.5% Quantity Adjustment 29,458, % 1.9% Operational Adjustment 26,335, % 11.7% HVAC Interactive Adjustment 28,308, % 2.5% Gross Realization Rate 28,308, % 13.6% The annual energy savings gross realization rate for custom lighting without controls was found to be 92.4% with HVAC interactive effects included. The relative precision for the final impact estimate with interactive is ±7.0%. Similar to prescriptive above, the primary driver of the realization rate was the decrease in operating hours observed through the on-site M&V, which caused a 6% drop in energy savings relative to that captured in the previous quantity adjustment. Table 10: Summary of 2011/2012 PY Custom Lighting Gross Energy Realization Rates Custom Lighting (n=31 Sites) kwh Incremental Realization Ratio Relative Precision at 90% Confidence (±) Tracking 61,681,541 N/A N/A Documentation Adjustment 61,215, % 1.5% Technology Adjustment 61,447, % 0.5% Quantity Adjustment 60,524, % 1.7% Operational Adjustment 56,747, % 5.8% HVAC Interactive Adjustment 56,969, % 2.3% Gross Realization Rate 56,969, % 7.0% Table 11 presents the summary of prescriptive and custom gross connected kw realization rates. The prescriptive lighting connected kw gross realization rate is 95.9% with a relative precision of ±3.9%. The prescriptive connected kw realization rate was less than 100%, which is the result of small DNV GL September, 2015 Page 15

20 downward adjustments on both technology (delta watts) and quantity installed. The custom connected kw gross realization rate is ±101.4% with a relative precision of 3.6%. Table 11: Summary of 2011/2012 Prescriptive and Custom Connected kw Realization Rates Prescriptive Custom Connected kw kw Incremental Realization Ratio Relative Precision at 90% Confidence (±) kw Incremental Realization Ratio Relative Precision at 90% Confidence (±) Tracking 6,105 N/A N/A 12,348 N/A N/A Documentation Adjustment 6, % 0.0% 12, % 2.0% Technology Adjustment 5, % 2.8% 12, % 1.6% Quantity Adjustment 5, % 1.8% 12, % 2.4% Gross Connected kw Realization Rate 5, % 3.9% 12, % 3.6% Table 12 shows the remaining factors of interest from the on-site M&V work for both prescriptive and custom lighting measures. Note that all precisions in the table are calculated at the 90% confidence level. The estimated summer coincidence factor from this study is 0.79 for prescriptive, and 0.80 for custom. The evaluated values for the kw HVAC interactive effect factor are 1.15 for summer kw and 1.06 for annual kwh for prescriptive. These reflect the percent of gross summer peak kw and gross kwh savings that are due to interactive effects. For custom, both factors remain near 100% since interactive effects are integrated into the custom tracking savings estimates. Table 12: Summer kw Factors and Other kwh Savings Factors Prescriptive Custom Other Ratio Estimation Results Factor (n=32) 90% Confidence Interval (+/-) Factor (n=31) 90% Confidence Interval (+/-) Summer kw Factors Coincidence Factor 79.0% 9.6% 80.4% 8.3% HVAC Interactive Effect Factor 114.8% 3.5% 99.3% 4.9% KWh Factors HVAC Interactive Effect Factor 106.3% 2.5% 100.4% 2.3% Average Hours of Use 90.0% 11.7% 93.9% 5.8% % On Peak KWh 63.5% 5.7% 53.8% 6.2% 4.1 Lighting Profiles Figure 6 presents the weighted percent on time for non-holiday weekdays and weekends for all sites. These profiles are weighted by connected kw savings from all lighting loggers installed. The pattern of DNV GL September, 2015 Page 16

21 use of lighting during the weekend as well as the week is roughly the same, with hours beginning to ramp up in the 7 a.m. hour and gradually tapering off in the evening. The rate of operation is roughly 40-50% higher during the week on core day hours as compared to the weekend. Figure 6: Weekday and Weekend Lighting Profiles (All Sites) Figure 7 presents the weighted percent of on time for non-holiday summer weekdays for both all sites combined. The 83% noted in hour 17:00 (5 p.m.) is close to our coincident factor estimate for both prescriptive and custom based upon the metering performed in this study. DNV GL September, 2015 Page 17

22 Figure 7: Summer Weekday Lighting Profile (All Sites) Figure 8 through Figure 15 provides the same charts for each of the following building types: Industrial, Retail (including Grocery), Office and Hospital. Figure 8: Weekday and Weekend Lighting Profiles (Industrial) DNV GL September, 2015 Page 18

23 Figure 9: Summer Weekday Lighting Profile (Industrial) Figure 10: Weekday and Weekend Lighting Profiles (Retail) DNV GL September, 2015 Page 19

24 Figure 11: Summer Weekday Lighting Profile (Retail) Figure 12: Weekday and Weekend Lighting Profiles (Office) DNV GL September, 2015 Page 20

25 Figure 13: Summer Weekday Lighting Profile (Office) Figure 14: Weekday and Weekend Lighting Profiles (Hospital) DNV GL September, 2015 Page 21

26 Figure 15: Summer Weekday Lighting Profile (Hospital) 4.2 Hours of Use Table 13 presents a summary of the average evaluated hours of use by building type for both prescriptive and custom projects combined. The first column presents the building type as defined by National Grid in the tracking database. The second column represents the evaluation s attempt at matching the actual site to a corresponding building type from the NY Tech Manual. The table provides the simple average of hours of use for each building type for both the tracking and evaluated estimates. Additionally, the table provides the hours of use estimate from the Tech Manual. As shown below, the evaluated hours of use were generally less than 100% of the tracking estimates, but also greater than 100% versus the Tech Manual estimates. For hospitals, lighting hours of use were significantly less than both the tracking and the Tech Manual estimates. This is likely because most of the installations were found to be going into health offices and exam room space types rather than common spaces in which lights would be expected to operate more frequently. Manufacturing facilities were found to have significantly higher hours of use estimates versus the Tech Manual. The Tech Manual has different categories for industrial, including Manufacturing Facility, Light Manufacturers, Industrial 1 Shift, 2 Shift and 3 Shift options. It is noted that the hours of use for the two and three shift options in the Tech Manual are both higher at 4,730 and 6,631, respectively. If Tech Manual hours are to be used for savings estimation purposes going forward, attention should be given to the appropriate industrial categories. DNV GL September, 2015 Page 22

27 National Grid Building Type Heavy Industrial Light Industrial Table 13: Summary of Hours of Use vs. Tracking and Tech Manual Tech Manual Building Type Count Average Tracking HOU Average Evaluation HOU Evaluated /Tracking Tech Manual HOU Evaluated/ Tech Manual Manufacturing Facility 13 6,788 6,184 91% 2, % Light Manufacturers 11 6,053 5,038 83% 2, % Hospital Hospital 6 4,789 3,567 74% 7,674 46% Small Retail Retail 5 5,358 4,936 92% 4, % Big Box Retail Retail 4 4,873 4,444 91% 4, % Grocery Food Stores 4 6,827 6,001 88% 4, % Secondary School Schools 2 2,828 3, % 2, % Assembly Manufacturing Facility 2 3,828 1,911 50% 2,857 67% Small Office Office (General Office Types) 2 2,982 2,612 88% 3,100 84% Large Office Office (General Office Types) 2 5,137 3,452 67% 3, % Warehouse Warehouse 1 8,760 8, % 2, % Primary School Schools 1 5,670 3,551 63% 2, % Multi Story Retail Retail 1 4,500 3,977 88% 4,057 98% Hospital 1 8,842 9, % 7, % Other Medical Offices 1 8,736 4,915 56% 3, % Office (General Office Types) 1 4,918 3,461 70% 3, % Schools 1 3,898 1,297 33% 2,187 59% Warehouse 1 7,566 5,022 66% 2, % 5 CONCLUSIONS AND RECOMMENDATIONS This impact study included the performance of on-site M&V work with re-engineering analysis to derive results that have traditionally not been achievable through other methods (e.g., the determination of coincidence factors, kw and kwh Interactive factors, connected kw and kwh realization rates, load shapes, and average hours of use). The M&V based assessment of prescriptive lighting without controls has a realization rate of 91%, while the custom lighting without controls realization rate was 92%. The following are recommendations based upon the findings from this study. The tracking lighting hours of use assumptions appear to be high in general with hours of use realization rates of 90% and 94% for prescriptive and custom lighting, respectively. However, the tracking hours of use estimates, which are based on vendor estimates, are closer than the default hours of use estimates for many building types in the New York Tech Manual. As such, DNV GL September, 2015 Page 23

28 while there is evidence from this study that the Tech Manual hours might be underestimating actual usage, we encourage National Grid and the DPS to compile added evidence from other New York Program Administrators ( PAs ) to be sure this trend holds across other territories before making Tech Manual revisions on this matter. One particular building type of note was hospitals. Both the tracking estimates and Tech Manual estimates for Hospital are found to be very high. One potential reason for this is that many lighting fixtures that are going into hospitals are not being installed exclusively in common areas. A good portion of lighting installations are being done in medical offices and exam rooms, which follow more of an office type of operating schedule. For grocery sites, the evaluation found hours to be higher than the Technical Manual. Since grocery lighting is dominated by the sales floor, which corresponds with hours of operation, grocery hours tend to be higher. These specific observations apply to a small sample of the projects in this study, but it is important to ensure that vendors take space type into account when estimating hours of use rather than only building type. Likewise, if Technical Manual hours of use estimates are adopted by National Grid, consider the proportion of lighting fixtures that are not going into high use, common areas when deciding on which building type hours to use. Hours of use should be space dependent rather than building dependent. The New York Tech Manual currently assumes a summer coincidence factor of 1.0 for commercial indoor lighting measures, which we regard as very high. The estimated summer coincidence factor from this study is approximately 0.8 for both prescriptive and custom. Currently, National Grid is using a 0.8 coincidence factor for custom lighting, which we recommend they continue to use. However, we would suggest that National Grid work with DPS to combine this data with that from other New York PAs to inform this value for prescriptive lighting.. Currently, National Grid uses HVAC interactive estimates of 1.19 for summer kw and 1.07 for annual energy savings for prescriptive lighting. These values are proxies that represent average factors from the table of HVAC interactive factors by Building Type in the Tech Manual. This study found the summer kw HVAC Interactive Effect Factor to be 114.8% (1.148), and the kwh interactive factor to be 106.3% (1.063). We would suggest that National Grid work with DPS to combine this data with that from other New York PAs to inform these values. Currently, National Grid uses custom engineering calculations to estimate HVAC interactive effects for custom lighting. This study found that both the summer kw and annual energy savings estimates for HVAC interactive effects are being calculated accurately, with realization rates of 99.3% and 100.4%, respectively. We recommend continued use of custom engineering calculations to estimate HVAC interactive effects for custom lighting. National Grid employs a post-inspection process for lighting projects in which a follow-up visit is performed by a company representative. This visit typically includes a visual inspection of installed equipment. DNV GL supports this process, and encourages continued and improved implementation of it. This study did not find many significant quantity discrepancies, which is an indication that the post-installation process is providing benefit. DNV GL believes that the DNV GL September, 2015 Page 24

29 process could be improved by doing more to collect and verify the reasonableness of the assumed hours of use during this post-installation visit. The project documentation provided for this impact evaluation was generally good. We recommend continuing to collect as much information as possible during the implementation process. Specifically, the following pieces of documentation are needed to better evaluate projects following installation: Spreadsheet savings calculations that match tracking estimates, Description of pre-installation conditions if a retrofit project, post-inspection report, TA study (if custom), details of annual operating hours used by space type or location. We recommend that future impact evaluations prescriptive lighting use an error ratio of 0.5 for energy when developing on-site metering sample sizes. The error ratio we recommend for targeting demand savings is a 0.2 error ratio. Likewise, we d target an error ratio of 0.4 for estimating on-site metering sample sizes for custom lighting, since the custom engineering estimates tend to produce results with less variability than prescriptive. Consider implementing a rolling evaluation framework, which would provide ongoing monitoring and feedback into the lighting programs. A continuous evaluation approach could benefit the program by providing fast feedback by targeting certain segments of interest such as the largest projects or specific sectors or technologies. DNV GL September, 2015 Page 25

30 6 APPENDIX A: ON-SITE MEASUREMENT, VERIFICATION AND ANALYSIS METHODOLOGY APPENDICES A key task in the on-site engineering assessment is the installation of measurement equipment to aid in the development of independent estimates of savings. This appendix reviews that process for the sampled prescriptive lighting without controls on-site sample. The type of measure influences the measurement strategy used. Lighting, the technology of interest to this study, is a time dependent measure type that runs at a constant load. Mathematically, hour-ofday and day-of-week are usually the most relevant variables in the energy savings analysis of these measures. The primary metering equipment used for this study is time of use lighting loggers, which were installed for a year. The following section outlines the generic methodology for time-dependent lighting measures. Monitoring. The time-of-use (TOU) loggers used in this study provide measured hours of use. These small devices use specialized sensors photocells in the case of lighting measures to sense and record the dates and times that a device turns on and off. This TOU data will be used to support the evaluation in two key ways: To develop summer peak coincidence factors, and To develop annual hours of use. The measure scope influenced the appropriate number of loggers and systems monitored for each site. Factors that drove the number of installed loggers include the number of unique schedules at the site, and the anticipated level of variation among the schedules within a particular space type. Verification. A detailed inventory was performed for each installed measure. This inventory included a verification of the quantity and technologies installed from the program, as well as customer reported operating hours for specific equipment, pre-existing equipment types and location of installed equipment. Methods of control were also be examined and inventoried at this time. Other variables that were gathered and analyzed include the types of heating and cooling systems serving the areas of the installed measure for the calculation of interactive HVAC effects. Analysis. This section serves as a detailed example that illustrates the calculation of all savings and adjustment factors. DNV GL modified a single line item from a site in a similar study to serve as an example of the calculation methods. The table below presents a summary of all savings parameters for this particular example. DNV GL September, 2015 Page 26