Custom Impact WO033 Evaluation Plan Impact Evaluation

Transcription

1 Final Version Custom Impact WO033 Evaluation Plan Impact Evaluation Submitted to: Energy Division California Public Utilities Commission 505 Van Ness Ave. San Francisco, CA Submitted by: Itron, Inc Broadway, Suite 1800 Oakland, CA (510) and KEMA, Inc. 155 Grand Avenue, Suite 500 Oakland, CA December 27, 2011

2 Table of Contents 1 Introduction Claimed Energy Impacts by IOU for 2010 Custom Projects Goals and Objectives Evaluation Priorities and Researchable Issues Evaluation Overview Recent Related Research Evaluator Contact Information Overview of Work Order Composition and Programs Composition of 2010 Claim Programs in the Custom Evaluation Overview of Measure Groups Included in the Evaluation Aggregate Measure Groups High Impact Measure Groups Other Measure Aggregation Considerations Data Sources Program Tracking Data and IOU Data Project Specific Documentation End Use Metering of Selected Projects On-site EM&V Audits Telephone Surveys of Participating Customers, Trade Allies, and Account Representatives Decision-Maker Survey Trade Ally Interviews Account Representatives Vendor Telephone Surveys Secondary Data Sources Literature and Specification Research and Review CPUC/IOU Studies on CALMAC CPUC Evaluation Data Sets Data from Other EM&V Studies and Sources Websites from Equipment Manufacturers and Industry Associations DEER and DEER Updates Conference Papers and Published Articles Standard References and Equipment Efficiency Tables Coordination and Communication Progress Reporting to ED ED Approval of Work Information Flow and Mechanisms Cross-study Sampling Coordination Itron, Inc. i Table of Contents

3 5.5 Project Assignment Coordination and Continuity Evaluation Study Coordination Sharing of Data and Results Coordination with IOUs Special Considerations for WO Ex-ante Review Overview Sample Coordination Communication and Feedback to IOUs Early Results Reporting IOU Input on Evaluation Methods IOU Input on Evaluation Reporting Sample Design Impact Methods Objectives and Goals Guiding the Impact Methodologies Baseline Selection Dual Baseline Gross Energy and Demand Impact Evaluation M&V Approach Overview Rigor Levels Site-Specific M&V Plan (Enhanced and Basic Rigor Levels) EUL/RUL Measure Cost Impact Shapes Reliability, Bias and Uncertainty Energy Savings Methodology (kwh and therm) Interactions Between Measures Impact Methodology for Peak Demand Aggregation of Results Lower Rigor Evaluation Activities Supporting WO Net Energy and Demand Impact Evaluation Impact Evaluation Procedures Timeline Work Plan and Budget Work plan Task 1: Evaluation Planning Task 2: Project Management (including Coordination and Reporting) Task 3: Tracking System Management and Sampling Task 4: Data Collection Instrument and Tool Development Task 5: Gross Impact Evaluation Task 6: Net Impact Evaluation Task 7: Lower Rigor Points to Support Program Assessment Task 8: Annual Reporting Early Feedback / Regulatory Support and Consultation Budget Itron, Inc. ii Table of Contents

4 List of Tables Table 1-1: Claimed Energy Impacts by IOU for 2010 Projects in the Custom Impact Work Order (WO033) Table 1-2: Evaluator Contact Information and Roles Table 2-1: WO Savings Claims by Program Table 2-2: WO033 Contribution to 2010 Savings Claims by Program Table 2-3: WO Program Savings Claim Relative to Portfolio Claim Table 2-4: 2010 Metrics Relative to Goals for Programs that Significantly Overlap with WO033* Table 2-5: 2010 Expenditures Relative to Budgets for Programs that Significantly Overlap with WO033* Table 3-1: WO Savings Claim by Measure Group Table 7-1: Schedule of Feedback and Reports to IOUs Table 8-1: Summary of Overall Sample Sizes for WO033 Impact-Related Effort Table 8-2: Illustrative Example of One Possible M&V, LR, and Incremental NTG Sample Allocation Approach (all points shown are per program except total row) Table 9-1: Peak Demand Days by Climate Zone Table 9-2: Information Sources for Three Levels of NTGR Analysis Table 10-1: Timeline Table 11-1: Budget by Task and Time Period Table 11-2: Sample Points by Evaluation Component Table 11-3: Cost Inputs and Sample Sizes for Lower Rigor Points Table 11-4: Net-to-Gross Cost Inputs and Sample Sizes List of Figures Figure 1-1: Diagram Illustrating Decision-Based Reporting Periods Figure 2-1: 2010 Savings Claim by Work Order Figure 2-2: 2010 Disaggregated Therm Savings Claim by Work Order Figure 2-3: WO033 Contribution to 2010 Savings Claim for Deemed Versus Non-Deemed Measures Figure 2-4: WO033 (Versus Combined Other Work Order) Contribution to 2010 Savings Claim Expressed in Terms of Source Energy Itron, Inc. iii Table of Contents

5 Figure 3-1: 2010 WO033 Savings Claim by Aggregate Measure Group Figure 8-1: Examples of Relative Precision, at the 90 Percent Confidence Level, with a Finite Population of 500, for Varying Error Ratios and Sample Sizes Itron, Inc. iv Table of Contents

6 1 Introduction This research plan addresses the impact evaluation of California investor-owned utilities (IOU) energy efficiency programs, focusing on custom measures. Impact evaluations serve many purposes including independent estimation of program and measure impacts, provision of recommendations to improve programs, support for costeffectiveness analyses, development of data and findings in support of future program planning, and support for strategic planning. This evaluation plan has been developed to address the highest priority evaluation objectives for the custom projects under CPUC Energy Division (ED) direction, and CPUC Work Order 33 (WO033) 1. The scope of work includes independent estimation of gross and net savings and development of findings and recommendations that can be used to improve program and measure effectiveness. The measures and projects within the custom group are extremely diverse and site-specific, with savings that range widely across projects (by several orders of magnitude). As a result, an efficient evaluation approach requires extensive site-specific evaluation utilizing stratification by project size as the primary sampling approach, with possible segmentation across domains such as investor-owned utility (IOU), sector, program grouping, fuel, measure grouping or end use. In addition, there are a very large number of programs with custom measures in the nonresidential sector 2. As addressed in Section 8 of this plan, there are a number of tradeoffs that must be considered in developing a sampling and reporting plan for this custom-focused work order. The evaluation is organized into two periods to address the effect on custom program implementation of the CPUC s recent ex-ante-related Decision (D ). 3 This research plan primarily addresses the first of these periods, the before-decision period. While activities in the after-decision period are outlined in this plan, these activities will be further detailed, as necessary, through addenda to this plan. 1 WO033 is referenced at 2 At the time this plan was developed, more than 100 programs included custom measures that are part of the Work Order 33 evaluation effort. 3 We refer to these two periods as before-decision (BD) and after-decision (AD). The before-decision (BD) period includes the program cycle period prior to D ; the after-decision (AD) period refers to the remainder of the program cycle. Itron, Inc. 1-1 Introduction

7 Programs, measures and measure groupings that fall within the custom measure impact evaluation are assessed in order to estimate gross and net energy savings. In general, the custom measure impact evaluation covers non-deemed programs and measures (generally excluding lighting, which will be addressed in the Nonresidential Downstream Lighting WO029). Custom measures typically include heating and cooling, refrigeration, process and miscellaneous end uses across the commercial, industrial and agricultural sectors. Net-to-gross (NTG) estimates may be developed for some deemed measures. 4 The same NTG method would be used for nonresidential custom and deemed measures. However, under the current WO033 evaluation plan, we will not incorporate primary data collection in support of gross impact evaluation objectives for deemed measures. If primary data collection for deemed measures is added to the project, the plan for these activities will be described in an addendum to this plan. This Research Plan describes the goals and objectives of the Custom Impact evaluation under WO033; outlines the researchable issues; describes the custom measure programs and measures to be evaluated; identifies data sources that will be used; presents the approach for sampling; summarizes the methods that will be used to determine gross and net impact; explains how coordination and communication will occur with other related evaluation studies; and discusses the need for and approach to providing timely feedback to the IOUs. Energy savings claims from the measures assigned to the custom impact work order (WO033) represent a significant contribution to the overall savings portfolio for the IOUs energy efficiency programs (about 24% of electric savings claims and 71% of gas savings claims in 2010 on a statewide basis). It is important to conduct an impact evaluation on the custom measures, not only due to their share of the IOU savings portfolio, but also because of the heterogeneous nature of these measures and the potentially high uncertainty of savings for custom measures. For 2010, the IOU tracking data for custom measures has thousands of entries statewide with statewide savings claims totaling 918 GWh and nearly 145 MW. Statewide gas savings claims from custom measures total 47.3 million therms. Approximately $117 million in custom measure incentives were paid for 2010 projects. 1.1 Claimed Energy Impacts by IOU for 2010 Custom Projects Table 1-1 illustrates the claimed energy savings included in the custom evaluation using the current measure group mapping 5 to impact evaluation work order. 4 If development of net-to-gross (NTG) estimates for deemed measures that fall under WO033 is pursued, the associated evaluation plan will be presented as an addendum to this plan. 5 CPUC Energy Division created these measure groups to facilitate the aggregation of like measures for the purposes of dividing the evaluation responsibilities by work order and to enable evaluation reporting by measure, where feasible. Itron, Inc. 1-2 Introduction

8 Table 1-1: Claimed Energy Impacts by IOU for 2010 Projects in the Custom Impact Work Order (WO033) IOU Electric Energy (GWh) Impacts by IOU Electric Demand (MW) WO Savings Claim Gas Energy (Million Therms) 6 PG&E SCE SDG&E SCG Total Total 2010 Savings Claim PG&E 1, SCE 1, SDG&E SCG Total 3, WO033 Percentage of Claim PG&E 33% 31% 84% SCE 15% 13% 3% SDG&E 25% 19% 62% SCG 0% 0% 68% Total 23% 20% 71% The 2010 project population assigned to WO033 entails over 50 measure groups and over 100 programs. This research plan identifies the highest impact measures and programs contributing to WO033 based on 2010 IOU tracking data. The evaluation team is currently analyzing IOU tracking data for the period through Q This information will be used to inform a more detailed sample design that will be provided as an addendum to this plan. 6 Gas savings reported includes only tracking records with positive therm impacts, thereby not addressing the significant number of negative records, primarily representing increased heating due to lighting efficiency measure installations interactive effects. Itron, Inc. 1-3 Introduction

9 1.2 Goals and Objectives The overarching goals of this impact evaluation for custom measures and programs are to verify and validate the energy efficiency savings claims reported from IOU energy efficiency programs; to provide feedback on how well program procedures and savings methods comport with energy efficiency policies, requirements, and expectations; and provide recommendations on how custom programs can be further improved or refined to support overall and custom-related energy efficiency goals and objectives. This is accomplished by balancing cost, accuracy and precision in the data collection and analysis conducted. Gross savings, free ridership and net savings (kwh, kw and therms) will be estimated. Evaluation savings estimates span the full lifecycle of the relevant measures. IOU savings claims will be adjusted using evaluation-based gross realization rate 7 and NTG ratio estimates. Where feasible, for any given project assignment, other evaluation objectives may also be included, for example, assessments of installed/incremental cost (in coordination with the Measure Cost Study, WO017) and load shape reporting. A goal of the first phase of this impact evaluation is to provide the IOUs with feedback that can be used to make any necessary corrections to improve their current programs (particularly if the CPUC extends the current program cycle during a one or two-year bridge period), as well as feedback on what aspects of program design and implementation are successful (this latter activity will be coordinated and integrated with process evaluation work conducted under WO012). For this to be successful, evaluations must be timely in their reporting of results and feedback to allow utility staff adequate time for making necessary improvements. Impact reports will be prepared on an annual basis starting in July of 2012 and ending in July of 2014, supplemented by periodic ad hoc feedback that will be provided to the IOUs in a memorandum format. These impact results and feedback will be designed to contribute to program and portfolio planning and policy making for the next program cycle. In order to extract the greatest value from this custom measure impact evaluation, the data and information collected will be leveraged to the greatest extent possible. Energy efficiency impact realization rates, baseline and standard practice assessments, program feedback, and measure cost data will be shared across research areas to support cost effectiveness analysis, current cycle program improvements, strategic planning, future program planning and design, and updates to the Database for Energy Efficiency Resources (DEER). 1.3 Evaluation Priorities and Researchable Issues Estimation of gross and net savings (kwh, kw, therms) from IOU custom measures, programs and measure groupings is a high-priority research issue and the focus of the evaluation effort. 7 Realization rates are the ratio of evaluated ex post savings to ex-ante claimed savings. Itron, Inc. 1-4 Introduction

10 The gross impact estimates for sampled measures will be based on field inspections, measurements, and extensive engineering analysis (i.e., M&V). The sampling domains for which impact results will be developed are discussed in Section 8, along with related issues such as number of sample points and estimation methods. The priorities for this evaluation effort and the researchable issues that this evaluation seeks to examine are described as follows: 1. Estimating the level of achieved gross impact savings, determining what factors characterize gross realization rates, and, as necessary, assessing how realization rates can be improved. 2. Estimating the level of free ridership, determining the factors that characterize free ridership, and, as necessary, providing recommendations on how free ridership might be reduced. 3. Estimating participant spillover from efficient equipment installations made as a result of the program but without the provision of an incentive. 4. Providing timely feedback to IOUs to improve program effectiveness. 5. Analyzing the extent to which recommendations from the Impact Evaluations have been implemented by the IOUs, and making a qualitative assessment to determine if the implementation of those recommendations resulted in changes to higher net and gross realization rates. 6. Determining whether the impact estimation methods, inputs, and procedures used by the IOUs and implementers are consistent with the CPUC s policy and decisions and best practices. 8 The evaluation will identify issues with respect to impact methods, inputs, and procedures and make recommendations to improve IOU savings estimates and realization rates and areas where CPUC policy could be expanded, clarified or improved. 7. Improving baseline specification, including collecting and reporting on dual baseline. Estimating the extent of any program-induced acceleration of replacement of existing equipment and, in such cases, the RUL of the pre-existing equipment. 8. Collecting data and information to assist with other research or study areas, which could include measure cost estimation, cost effectiveness analysis, load shapes for measures, updates to DEER, strategic planning, providing information for benchmarking program success under WO012, and future program planning. Evaluated programs will be benchmarked against the applicable Best Practices using the Self- Benchmarking scoring tool available from the Best Practices website ( The tool will be updated for additional Best Practices researched 8 See NR-5 Nonresidential Best Practices Report at Itron, Inc. 1-5 Introduction

11 during this evaluation cycle via Work Order 12. This benchmarking will occur through the Program Assessment task in Work Order 12. In addition to the M&V-oriented gross impact work described above, additional site level evaluation work will be accomplished using lower rigor points. Lower rigor points are described in greater detail below in section 1.4 and in Section 8. Lower rigor points are intended to expand the reach of the evaluation, through targeted oversampling, to programs that would not otherwise receive much attention based on M&V sample allocation alone. M&V and lower rigor points will be used to examine program procedures (including measure eligibility and other rules); analyze strengths and weaknesses of project applications; and provide feedback on individual program s impact estimates. The purpose of the combined M&V and lower rigor based points is to describe and comment on a program s procedures, rules and impact estimation processes. For example, the project reviews will examine whether or not baseline specifications are appropriate; if early replacement procedures are being adhered to; if program eligibility rules are consistent with overarching regulatory guidance and program rules; and whether program rules differ across programs. This impact-related, program-level feedback will support a broader Program Assessment currently being planned under Work Order 12. The method that will be used to develop net-to-gross ratios will be coordinated with the ex-ante review/early EM&V Work Order (WO002) 9 and the crosscutting EM&V guidance activity under WO Evaluation Overview To the extent custom measures, measure groupings and programs share core similarities, the proposed evaluation approaches will be consistently applied. However, given the heterogeneity of custom measures, specialized and unique approaches by project are also expected. This custom impact evaluation study will be guided by the California Energy Efficiency Evaluation Protocols: Technical, Methodological, and Reporting Requirements for Evaluation Professionals 11. The following protocols will be used, along with other protocols developed or prescribed by ED: Impact Evaluation Protocol Gross Energy Impact Protocol Gross Demand Impact Protocol 9 WO002 is referenced at 10 WO048 is referenced at 11 Available at: Itron, Inc. 1-6 Introduction

12 Participant Net Impact Protocol Indirect Impact Evaluation Protocol Measurement and Verification Protocol Sampling and Uncertainty Protocol Evaluation Reporting Protocol The custom measure gross impact assessment will involve standard M&V approaches to the extent appropriate and practical, with on-site data collection, monitoring, and analysis for a representative sample of custom measures, measure groupings and programs in order to: a) develop ex-post estimates of the energy and demand savings for each project in the sample, and b) apply those findings back against the full participant population to obtain a complete estimate of program impacts. M&V rigor levels will vary based on the size, complexity, and overall importance of individual projects. The evaluation team will utilize IOU and implementer collected site specific information, including M&V data, supplemented by data collection through this evaluation. In addition, a net-to-gross assessment is planned using telephone survey data collection and selfreport methods to derive net program impacts. A non-residential net-to-gross guidance team composed of ED staff, contractors and consultants is developing the general approaches to be used by all affected energy efficiency program evaluations. Similar to variability in M&V rigor level applied, net-to-gross rigor will be based on the size, complexity, and overall importance of individual projects. More information regarding the net-to-gross assessment is found in section 9 of this plan. The periods for which reporting is being performed in the evaluation cycle for custom impacts is affected by a decision issued on July 14, 2011 (Decision No ) regarding the ex-ante review and freezing process 12. Savings claims for custom and non-deer measures are subject to different treatment depending on whether they were reported before or after the date of that decision. For this reason, this research plan sets the reporting periods as the before-decision (BD) period and an after-decision (AD) period. These periods have been respectively defined, for ease of tracking claims, as the first half of the evaluation cycle (2010/2011 Q1-Q6 covering 2010 through June 2011) and the last half of the evaluation cycle (2011/2012 Q7-Q12, covering July 2011 through 2012). These reporting periods are illustrated in Figure 1-1, showing evaluation treatment for different periods (e.g., including some pre- and post-installation M&V in the AD period), the integration of WO002-driven project assignments (i.e., for those sites with 12 Ex-ante review and freezing work efforts are managed and implemented under WO002. WO033 is providing support for WO002 ex-ante review by using evaluation methods that may be used to inform ex-ante review of future projects. Itron, Inc. 1-7 Introduction

13 pre-installation M&V) and the likely timing of IOU AD project completion under the evaluation timeframe following WO002 ex-ante review (EAR) and pre-installation M&V efforts. Figure 1-1: Diagram Illustrating Decision-Based Reporting Periods There is a wide variety of sampling options and tradeoffs. The proposed sample sizes for the overall custom evaluation are discussed in Section 8, a brief summary of which is provided below. A total of 1,860 sample points is currently planned, consisting of: 600 M&V points (200 BD and 400 AD points), 300 lower rigor (LR) points (100 BD and 200 AD), and 960 NTG-only points (480 BD and 480 AD). The M&V sample will be used for estimating gross realization rates. The NTG points would be matched with the M&V points in the M&V gross impact sample. To inform custom program performance, there will be additional NTG sample points allocated to programs and program groups. With 600 points, the sample will likely only support a 90/10 relative precision level for a small number of sampling domains. Consequently, the primary sampling domains for the M&V sample design will be IOU and fuel. The M&V sample will be supplemented by both lower rigor and additional net-to-gross sample points. The LR points will be combined with the M&V points to produce impact-related findings; however, only the M&V points will be used to generate gross impact realization rates, the LR points will be focused on qualitative feedback. We will target sample impact-related points per program (drawing from both the M&V and LR samples). This will likely enable program-level feedback for roughly 30 to 40 programs. Similarly, to provide NTG results for multiple programs, we include 960 supplement NTG sample points. NTG will be conducted for all points including M&V and LR, thus there will be a total of 1,860 NTG sample points. We estimate that we will need 25 to 60 NTG points per program (averaging around 40 points per program) to Itron, Inc. 1-8 Introduction

14 produce program-level NTG results at the 90/10 and 90/15 relative precision level (with the precision level depending on relative program importance). As a result, we estimate that NTG results will be provided for around 30 to 40 programs or more. Note that the samples and precision targets discussed above are for the combined BD and AD periods, that for the entire cycle. Precision levels for the BD or AD periods individually will be lower due to the commensurably smaller sample sizes for each period. In addition, note that the above sample applies only to custom measures. To inform program assessment activities under WO012, it will be important to identify how programs are now implemented by the IOUs with regard to common characteristics of the various programs and program groups. The assessment of sampling options to date has focused primarily on 2010 participation data. Certain program or measure groupings could become more or less important relative to others based on the distribution of savings for 2011 and The evaluation will first develop a sampling approach for the BD period to get work underway, and then integrate real-time AD sampling once data become available. More detail is provided in Section 8 of this research plan surrounding sample frames, the BD and AD periods of program performance, pre-installation M&V, post-installation M&V (as well as opportunities to perform both pre- and post-installation M&V on a sub-sample of points), and other relevant domains and strata. A fully articulated sample plan, based on the recently received IOU tracking data through Q2 2011, will be provided for the BD period in an addendum to this plan. Finally, selected special measure studies may eventually be included in the WO033 scope, though none are planned at this time. These special measure studies would involve measures which are of special interest and warrant further attention by expanding upon results from the evaluation cycle. ED has identified possible measures of special interest for the custom evaluation to be steam traps, pump testing and pump-off controllers. The IOUs have recently identified pump testing, retrocommissioning and monitoring-based commissioning as measures of interest; some or all of these measures (and others as the programs realize greater contributions) may potentially be analyzed in a special measure study or through selected program assessments where possible. A combination of IOU tracking systems review, application review, literature review, and supporting interviews will be used to develop any special measure studies. 1.5 Recent Related Research This custom measure impact evaluation will be informed by a previous body of relevant evaluation work, as well as the Energy Efficiency Program Best Practices Study for custom-type Itron, Inc. 1-9 Introduction

15 incentive programs 13. Evaluations performed for the PG&E Fabrication, Process and Manufacturing Contract Group 14, PGE s Agricultural and Food Processing Evaluation, 15 Southern California Industrial and Agricultural Contract Group 16, and the Standard Performance Contracting (SPC) program for program years 1998 through 2005 are particularly relevant to this work scope. The research approach will be designed based in part on the approaches used and information contained in these prior studies, and other studies nationwide, and where possible, to answer any researchable questions that are relevant. SPC studies include the following: Program year Program year Program years Program year Program year Program Year Evaluator Contact Information A listing of the key individuals responsible for the evaluation planning effort, along with their responsibilities and contact information is provided in Table Energy Efficiency Best Practices Study - Large Nonresidential Comprehensive Incentives Programs (NR5), prepared by Quantum Consulting, Inc. for Pacific Gas & Electric Company under the auspices of the California Public Utility Commission in association with the California Energy Commission, San Diego Gas and Electric, Southern California Edison, and Southern California Gas Company Itron, Inc Introduction

16 Table 1-2: Evaluator Contact Information and Roles Name Company Address Role Phone Kay Hardy CPUC ED 505 Van Ness Ave, San Francisco, CA Peter Lai CPUC ED 320 West 4 th St, Suite 500, Los Angeles, CA Mike Rufo Itron 1111 Broadway, Suite 1800, Oakland, CA Kris Bradley Itron 1111 Broadway, Suite 1800, Oakland, CA Al Lutz Itron 1111 Broadway, Suite 1800, Oakland, CA Jennifer Fagan Ann Peterson Itron Itron 1111 Broadway, Suite 1800, Oakland, CA Broadway, Suite 1800, Oakland, CA CPUC Project Manager, WO033 CPUC Project Manager, WO002 Technical Advisor Technical Advisor Project Manager /Manage Gross Impacts and Engineering Manage Netto-Gross and Market Impacts Project Support katherine.hardy@cpuc.ca.gov peter.lai@cpuc.ca.gov Mike.Rufo@itron.com Kris.Bradley@itron.com Al.Lutz@itron.com Jennifer.Fagan@itron.com Ann.Peterson@itron.com Fred Coito Kema 155 Grand Ave, Suite 500, Oakland, CA John Stoops Kema 155 Grand Ave, Suite 500, Oakland, CA KEMA Project Manager KEMA Project Manager fred.coito@kema.com john.stoops@kema.com Itron, Inc Introduction

17 2 Overview of Work Order Composition and Programs The Custom Impact Evaluation principally involves custom, non-deemed measures found in over 100 programs. Because it will not be possible to fully address all programs that fall within this evaluation, the evaluation will address common attributes and issues with custom-oriented programs and selected programs and program groupings. A few programs have participation that is entirely custom measures; others have a custom focus, where custom measures are targeted by the program or constitute a majority of the savings; custom measures are found in other programs with a deemed focus and this evaluation also addresses those offerings. It is also noted that several measure groups selected for evaluation under the Custom Impact Work Order (WO033) presently include a substantial deemed savings claim. These and other attributes of WO033 are presented in greater detail in this section and other sections in this research plan, particularly Section Composition of 2010 Claim This section highlights the relative importance of the impacts included in the Work Order 33 Custom Impact Evaluation to the total energy efficiency portfolio, characterizes the share of the impacts associated with custom (non-deemed) and deemed measures, and summarizes impacts, costs, and goals by program. An important note is that the program achievements and description in this section are based on 2010 accomplishments and these may shift in future years. Tracking data from IOUs will be used on a periodic basis to determine changes in relative program performance and accomplishments. IOU program tracking records supporting the 2010 savings claim are examined using aggregate segmentation in this section in order to initially describe the composition of WO033. It is instructive to examine the composition of tracking records and prominent programs that fall under the WO033 evaluation, and draw comparisons where possible versus the entire 2010 claim and the other work orders. It is expected that the composition of the custom measures and population will change from the 2010 year to years 2011 and 2012, perhaps significantly. For these reasons, an assessment of 2010 savings claims and the claims by measure groups and programs are illustrative, and sampling and other research considerations will benefit at a future date from the availability of additional tracking data records, particularly through incorporating all records from the entire BD period. Itron, Inc. 2-1 Work Order Composition and Programs

18 This section describes the fraction of the total portfolio claim accounted for by WO033, what fraction of that claim is associated with non-deemed measure savings, and the relative importance of electric versus gas savings claimed on a source MMBtu basis for comparative purposes, using a heat rate of approximately 10,000 BTU/kwh. Measure group and aggregate measure group savings claims are addressed in Section 3. Tables in both Sections 2 and 3 show gross savings figures for electric and gas claims. Figure 2-1 presents the fraction of the total 2010 portfolio claim that is accounted for by WO033. WO033 accounts for 24% of the electric energy savings claim, second only to Residential Lighting (WO028). WO033 also represents 107% of the natural gas energy savings claim, exceeding 100% due to the influence of interactive effects. 23 Figure 2-1: 2010 Savings Claim by Work Order 2010 Electric Savings Claim by Work Order 2010 Gas Savings Claim by Work Order Work Order Millions of Therms Percent WO028 - Res Ltg (16.7) -38% WO029 - NonRes Ltg (3.0) -7% WO032 - HVAC 2.7 6% WO033 - Custom % WO034 - BCE (2.1) -5% WO035 - Appl Recyc (0.2) 0% WO036 - Audit - 0% WO042 - Res Misc % WO046 - Res WholeBldg 2.1 5% Total % To illustrate this point, natural gas claims greater than or equal to zero have been disaggregated and presented in Figure 2-2, side-by-side with natural gas claim adjustments that are less than zero. 23 Gas savings of WO033 can be larger than 100% of the total portfolio gas savings claimed due to interactive effects and increased heating requirements, generally but not exclusively associated with lighting measures and associated savings claim adjustments. Itron, Inc. 2-2 Work Order Composition and Programs

19 Figure 2-2: 2010 Disaggregated Therm Savings Claim by Work Order Measure Savings Greater than or Equal to Zero Measure Savings Less than Zero 22% 3% 0% 4% 0% -1% -1% -9% 0% -2% 0% -13% 0% Total Millions of Therms: 67 71% Total Millions of Therms: % WO028 - Res Ltg WO029 - NonRes Ltg WO032 - HVAC WO033 - Custom WO034 - BCE WO035 - Appl Recyc WO036 - Audit WO042 - Res Misc WO046 - Res WholeBldg An important finding from this disaggregation exercise is that WO033 accounts for 71% of all positive gas savings claims (67 million therms) and only 1% of all negative gas interactive claim adjustments. Compared with the entire portfolio, this makes WO033 relatively important in terms of evaluating positive gas savings claims and shows that the evaluation will be relatively free from addressing interactive effects. In other areas of the portfolio, the high level of negative gas claim adjustment (23 million therms) indicates that interactive effects are significant and an important part of the overall portfolio evaluation. Figure 2-3 presents a breakout of WO033 savings claims that are deemed versus non-deemed. Deemed electric and gas claims represent 20 to 25% of the total for this work order. The electric deemed claims are concentrated (78%) among refrigeration measures such as strip curtains, door gaskets, and case replacements. Electric deemed measures also represent a much smaller savings claim in WO033 for the HVAC, motor, process, building envelope and other measure groups. Gas deemed claims are associated with steam traps (the bulk of that claim, and four times greater than non-deemed claims for steam traps), as well as pipe and tank insulation and process measures. Itron, Inc. 2-3 Work Order Composition and Programs

20 Figure 2-3: WO033 Contribution to 2010 Savings Claim for Deemed Versus Non- Deemed Measures 2010 Electric Accomplishments by Work Order Non-WO33 2,989 WO Non-Deemed 690 Deemed 228 GWh 2010 Gas Accomplishments by Work Order Work Order Millions of Therms Percent WO028 - Res Ltg (16.7) -38% WO029 - NonRes Ltg (3.0) -7% WO032 - HVAC 2.7 6% WO033 - Custom % WO034 - BCE (2.1) -5% WO035 - Appl Recyc (0.2) 0% WO036 - Audit - 0% WO042 - Res Misc % WO046 - Res WholeBldg 2.1 5% Total % Non-Deemed 37.4 Deemed 10.1 Millions of Therms To examine the relative importance of electric versus gas 2010 claims for WO033, Figure 2-4 presents total source MMBtu claims by utility, segmented by electric versus gas. For WO033, gas represents about 1/3 of the total savings claim on a source MMBtu basis, with PG&E and SCG accounting for the majority of that claim. Also shown is the remainder of the portfolio, less WO033. For these other work orders, the claim is dominated by electric, with the overall gas claim being negative. As noted earlier, this negative gas claim for other work orders is composed of positive and negative savings (20 million therms of positive savings and 23 million therms of negative savings), yielding a net overall claim of negative 3 million therms. Again, WO033 differs sharply in aggregate from these other work orders, with 48 million therms of positive savings and a trivial amount of negative savings claimed. Itron, Inc. 2-4 Work Order Composition and Programs

21 Figure 2-4: WO033 (Versus Combined Other Work Order) Contribution to 2010 Savings Claim Expressed in Terms of Source Energy 20,000,000 15,000,000 Gas Elec 10,000,000 5,000,000 - (5,000,000) WO33 Other WOs WO33 Other WOs WO33 Other WOs WO33 PG&E SCE SDG&E SCG Other WOs 2.2 Programs in the Custom Evaluation The programs examined in this Custom Impact Evaluation primarily address industrial and manufacturing facilities, water supply and treatment and wastewater treatment, oil and gas extraction, oil refining and production, and commercial custom (non-deemed) program offerings. As noted above, commercial deemed savings measures are also present, with a concentration in electric refrigeration measures applicable to grocery stores and several gas measures, including steam traps. Although these programs initially also included residential custom (non-deemed) offerings, those projects are relatively small and will be addressed under (and likely formally assigned) to other work orders. Each program within the scope of WO033 offers one or more of the following interventions in order to encourage end-users to upgrade to energy efficiency measures: site specific facility assessments, feasibility studies, project incentives, facility audits, installation support services, pump testing, and specialized training. Although this subsection of the plan addresses programs that fully involve or partially overlap with the population of IOU tracking system records that define the custom study population, as discussed in Section 8, programs will not constitute a sampling domain and target for stand-alone gross impact realization rate results but will be addressed as a domain for qualitative impact Itron, Inc. 2-5 Work Order Composition and Programs

22 findings and NTG results. A total of approximately 117 programs in the 2010 program year included ED-defined measure groups that are covered by the WO033 Custom Impact evaluation. The 2010 program claim (kw, kwh and therms) for WO033 is found in Table 2-1 below, along with a comparison against the statewide portfolio 2010 savings goals. A total of 36 out of the identified 117 programs included in the Custom Impact Work Order (WO033) each have WO033 savings that account for 1% or more of each IOU s savings (kwh, kw or therms) claim for Any programs that do not meet this threshold are aggregated into a remaining programs category by IOU labeled as Other Programs. WO033 accounts for 24%, 20% and 107% of the kwh, kw and therm savings claim for 2010 in the statewide portfolio. WO performance expressed as a percentage of program savings goals is 40%, 29% and 100% of the 2010 portfolio electric energy, electric demand and gas therm goals. Table 2-2 presents both the WO033 share of each program s claim, as well as the entire program s claim across all work orders and a WO033 percent of total program claim. On average, for the 117 programs that WO033 addresses, WO033 accounts for roughly half of the electric savings and the majority of gas savings. Program savings claims for all tracking system records associated with the Custom Impact Work Order (WO033) were compiled for program year 2010 to determine high impact programs (and measure groups addressed in Section 3) those that contributed a large share of the overall portfolio savings. High impact programs are defined as having a minimum contribution of approximately 1% of each IOU s claims in kwh, kw or therms. WO033 savings claims are expressed in Table 2-3 as a percent of total IOU claims in 2010; this yields 36 high impact programs. Also shown for comparison purposes is the WO033 savings claim expressed as a percent of total statewide claims in This yields 21 programs that exceed the 1% threshold (defined this alternate way). 24 These percentages are presented below in Table 2-3, showing the programs that contribute more than one percent to a given IOU by kw, kwh or therm claim. Percentage contributions by WO033 to the portfolio savings claim are also presented in Table 2-2. Itron, Inc. 2-6 Work Order Composition and Programs

23 Table 2-1: WO Savings Claims by Program Program ID Program Name 2010 Claim WO Claimed Savings kwh kw Therms PGE21011 COM Calculated Incentives Program 83,788,860 9,967 1,009,868 PGE21012 Commercial Program Deemed 39,643,065 7,942 1,539,473 PGE21021 IND Calculated Incentives Program 53,803,305 6,282 14,659,709 PGE21022 Industrial Programs Deemed 4,602, ,327,992 PGE21031 AGR Calculated Incentives Program 35,624,784 6,523 3,323,997 PGE21032 Agricultural Program Deemed 89,510,349 20,426 1,381,690 PGE21035 Pump Efficiency Services 18,778,036 2,186 - PGE21042 Savings by Design Coml NC 48,137,830 11, ,890 PGE21261 CA Community Colleges EE Partnership 2,758,408 1, ,938 PGE21262 UC/CSU EE Partnership 14,475,411 2,179 1,316,413 PGE2147 San Francisco EW Partnership 1,722, ,878 PGE2182 Boiler EE Program 120, ,358 PGE2185 Energy Smart Grocer 16,892,656 1,197 3,885 PGE2186 Enhanced Automotive Initiative ,340 PGE2222 EE Services for Oil & Gas Production 44,248,407 4,245 - PGE2223 Heavy Industry EE Program 22,490,505 3,012 1,656,890 PGE2225 Refinery EE Program 2,661, ,239 Other Programs Other PG&E Programs 68,377,393 11, ,815 Total PG&E Custom Savings 547,635,681 89,065 29,911,377 PG&E Total Portfolio Savings 1,671,530, ,482 17,503,953 SCE-SW-002B Commercial Energy Efficiency Program Calculated 41,332,777 6,641 - SCE-SW-002C Commercial Energy Efficiency Program Deemed 20,074,458 3,582 (132,470) SCE-SW-003B Industrial Energy Efficiency Program Calculated 109,660,429 13,299 - SCE-SW-004B Agriculture Energy Efficiency Program Calculated 19,246,001 2,704 - SCE-SW-005A New Construction Program 53,053,962 11, ,462 Other Programs Other SCE Programs 59,078,149 10,147 12,558 Total SCE Custom Savings 302,445,776 47,478 27,550 SCE Total Portfolio Savings 1,963,140, ,422 3,099,439 Itron, Inc. 2-7 Work Order Composition and Programs

24 Table 2-1 (Cont d): WO Savings Claims by Program WO Claimed Savings Program ID Program Name kwh kw Therms 2010 Claim SCG3603 #SW-AgB Deemed ,531 SCG3607 #SW-ComA Calculated ,269 SCG3608 #SW-ComB Deemed ,017 SCG3611 #SW-IndA Calculated - - 9,101,839 SCG3612 #SW-IndB Deemed - - 4,077,913 Other Programs Other SCG Programs ,054 Total SCG Custom Savings ,613,624 SCG Total Portfolio Savings 3,080,937 1,918 23,088,770 SDGE3101 SW-AgB Deemed 89, ,664 SDGE3105 SW-ComA Calculated 6,601, ,200 SDGE3106 SW-ComB Deemed 8,273, ,532 SDGE3109 SW-IndA Calculated 1,500, ,820 SDGE3110 SW-IndB Deemed 454, ,805 SDGE3117 Local03 - Local Non-Residential (BID) 39,456,447 4, ,597 SDGE3118 SW-NCNR - NRNC Savings By Design 8,142,219 1, ,285 SDGE3162 3P-NRes02 - SaveGas - Hot Water Control - - 9,625 SDGE3170 3P-NRes13 - Retro commissioning (RCx) 955, ,364 Other Programs Other SDG&E Programs 2,056, ,639 Total SDG&E Custom Savings 67,528,628 8,358 1,718,529 SDG&E Total Portfolio Savings 268,479,825 43, ,839 Total WO033 Claimed Savings 917,610, ,900 47,271, Portfolio Savings 2010 Statewide Portfolio Claimed Savings 3,906,231, ,471 44,354, WO033 Savings Claim Relative to 2010 Portfolio Goals 2010 Total Portfolio Goals 2,276,000, ,000 47,100,000 WO033 % of 2010 Portfolio Goals 40.3% 28.9% 100.4% Itron, Inc. 2-8 Work Order Composition and Programs

25 Table 2-2: WO033 Contribution to 2010 Savings Claims by Program Program ID PGE21011 Program Name COM Calculated Incentives Program WO033 Claimed Savings Total Program Savings (Including WO033 and Other WO Assignments) WO033 As a Percentage of Total Program Savings kwh kw Therms kwh kw Therms kwh kw Therms 2010 Claim 83,788,860 9,967 1,009, ,410,209 15,223 1,014, % 65.5% 99.5% PGE21012 Commercial Program Deemed 39,643,065 7,942 1,539, ,906,824 38,895 (26,327) 19.7% 20.4% % PGE21021 IND Calculated Incentives Program 53,803,305 6,282 14,659,709 57,642,126 6,741 14,664, % 93.2% 100.0% PGE21022 Industrial Programs Deemed 4,602, ,327,992 17,034,591 4,032 1,231, % 20.5% 107.9% PGE21031 AGR Calculated Incentives Program 35,624,784 6,523 3,323,997 42,482,292 7,351 3,369, % 88.7% 98.6% PGE21032 Agricultural Program Deemed 89,510,349 20,426 1,381, ,627,728 22,904 1,263, % 89.2% 109.4% PGE21035 Pump Efficiency Services 18,778,036 2,186-18,778,036 2, % 100.0% 0.0% PGE21042 Savings by Design Coml NC 48,137,830 11, ,890 51,250,300 11, , % 93.5% 100.9% PGE21261 CA Community Colleges EE Partnership 2,758,408 1, ,938 3,467,383 1, , % 90.9% 99.7% PGE21262 UC/CSU EE Partnership 14,475,411 2,179 1,316,413 17,720,259 2,829 1,322, % 77.0% 99.6% PGE2147 San Francisco EW Partnership 1,722, ,878 18,911,638 3, , % 8.6% 51.6% PGE2182 Boiler EE Program 120, , , , % 100.0% 100.0% PGE2185 EnergySmart Grocer 16,892,656 1,197 3,885 17,240,838 1,231 2, % 97.2% 139.7% PGE2186 Enhanced Automotive Initiative , , % 0.0% 100.0% PGE2222 EE Services for Oil & Gas Production 44,248,407 4,245-44,248,407 4, % 100.0% 0.0% PGE2223 Heavy Industry EE Program 22,490,505 3,012 1,656,890 22,911,958 3,100 1,669, % 97.1% 99.2% PGE2225 Refinery EE Program 2,661, ,239 2,661, , % 100.0% 100.0% Other Programs Other PG&E Programs 68,377,393 11, , ,266,546 48,480 3,777, % 22.8% 16.8% Total PG&E Custom Savings 547,635,681 89,065 29,911, ,681, ,164 31,619, % 51.1% 94.6% PG&E Total Portfolio Savings 1,671,530, ,482 17,503,953 Itron, Inc. 2-9 Work Order Composition and Programs

26 Table 2-2 (Cont d): WO033 Contribution to 2010 Savings Claims by Program Program ID SCE-SW- 002B SCE-SW- 002C SCE-SW- 003B SCE-SW- 004B SCE-SW- 005A Other Programs Program Name WO033 Claimed Savings Total Program Savings (Including WO033 and Other WO Assignments) WO033 As a Percentage of Total Program Savings kwh kw Therms kwh kw Therms kwh kw Therms 2010 Claim Commercial Energy Efficiency Program 41,332,777 6,641-76,691,065 11, % 56.9% 0.0% Commercial Energy Efficiency Program 20,074,458 3,582 (132,470) 183,096,445 51,979 (150,791) 11.0% 6.9% 87.8% Industrial Energy Efficiency Program 109,660,429 13, ,317,744 15, % 87.2% 0.0% Agriculture Energy Efficiency Program 19,246,001 2,704-19,429,217 2, % 99.9% 0.0% New Construction Program 53,053,962 11, ,462 69,646,184 14, , % 78.5% 116.9% Other SCE Programs 59,078,149 10,147 12, ,432,252 53, , % 18.9% 4.2% Total SCE Custom Savings 302,445,776 47,478 27, ,612, , , % 31.8% 10.2% SCE Total Portfolio Savings 1,963,140, ,422 3,099,439 SCG3603 #SW-AgB Deemed , , % 0.0% 99.4% SCG3607 #SW-ComA Calculated , , % 0.0% 99.9% SCG3608 #SW-ComB Deemed , ,200, % 0.0% 58.2% SCG3611 #SW-IndA Calculated - - 9,101, ,609, % 0.0% 94.7% SCG3612 #SW-IndB Deemed - - 4,077, ,104, % 0.0% 99.4% Other Programs Other SCG Programs ,054 6, , % 0.0% 51.0% Total SCG Custom Savings ,613,624 6, ,035, % 0.0% 91.7% SCG Total Portfolio Savings 3,080,937 1,918 23,088,770 Itron, Inc Work Order Composition and Programs