Report. Idaho Home Energy Savings Program

Transcription

1 Report Idaho Home Energy Savings Program Prepared for: Rocky Mountain Power November 12, 2010

2 Prepared by: Brian Hedman Charlie Bicknell Philip Sieper Cheryl Winch Brian Shepherd The Cadmus Group M. Sami Khawaja, Ph.D. Vice President The Cadmus Group, Inc.

3 Table of Contents 1. Executive Summary...5 Overview of Evaluation Activities...5 Realized Gross Savings...6 Net-to-Gross...8 Cost-Effectiveness...8 Summary of Findings...9 Summary of Recommendations Program Overview Program Description...12 Eligibility Requirements...12 Incentive Levels Evaluation Methods Analytical Methods...15 Evaluation Question Overview...15 Impact Questions 15 Process Questions 16 Participant Surveys...16 Billing Analysis...18 Dealer Surveys...21 CFL Nonparticipants and CFL Retailer Surveys...21 Stakeholder Interviews...22 Secondary Research...22 Secondary Data Analysis...23 Data Sources Survey and Interview Findings Customer Survey Analysis...25 Marketing 25 Individual Measure Findings...26 Attic Insulation 26 Floor Insulation 29 Wall Insulation 30 Clothes Washers 32 The Cadmus Group, Inc. / Energy Services i

4 Refrigerators 36 Dishwashers 38 Electric Water Heaters 40 Windows 42 Lighting Fixtures 44 Dealer Survey Analysis...45 Introduction Error! Bookmark not defined. HES Marketing and Program Awareness 45 Pricing 50 Program Perception Among Dealers and Suggestions to Improve HES 51 Clothes Washer Recycling Through HES Program 52 Utility Staff and Implementer Interviews...54 Program Management 54 Market Barriers 59 Future Program Issues Program Results Impact Findings...63 Summary of Program Participation...63 Realization Rate... Error! Bookmark not defined. Billing Analysis Results...65 CFL Leakage Analysis...66 NTG Estimation Cost-Effectiveness Analysis Appendices Appendix A: Available Measures Tariff Information...79 Appendix B: Cost-Effectiveness Inputs...84 Appendix C: Dealer Survey Summary...85 Appendix D: Dealer Sales Perceptions...86 Appendix E: Attic, Floor and Wall Insulation Participant Survey Instrument...87 Appendix F: All non-insulation Measures Participant Survey Instrument...88 Appendix G: CFL Nonparticipant Leakage Survey Instrument...89 Appendix H: CFL Retailer Survey Instrument...90 Appendix I: Dealer Survey Instrument...91 The Cadmus Group, Inc. / Energy Services ii

5 Appendix J: Implementer Discussion Guide...92 Appendix K: Program Logic Model...93 Appendix L: Billing Analysis Final Model...96 Appendix M: Freeridership Matrix...98 Appendix N: Realization Rate Matrix The Cadmus Group, Inc. / Energy Services iii

6

7 1. Executive Summary PacifiCorp (Rocky Mountain Power and Pacific Power) offers the Home Energy Savings (HES) Program in Utah, Wyoming, Idaho, Washington, and Northern California. HES was first made available to Rocky Mountain Power s customers in May The HES Program provides incentives for residential customers to facilitate the purchase of efficient products and services. This report focused on HES program activities for 2006, 2007, and 2008 in Idaho. PacifiCorp offers this program throughout the five state service territories where it offers demand-side management programs 1. Together the demand side management programs in Idaho acquired more than 16,363 MWh/YR of energy savings (Gross at Generation) in Within PacifiCorp s Idaho service area; the HES program is responsible for 54% of the savings that the utility realizes from residential efficiency programs. 2 Overview of Evaluation Activities The program targets the following key measures: Lighting: Upstream incentives for manufacturers to reduce retail prices on compact florescent lamps (CFLs). Customer incentives for light fixtures and ceiling fans. Appliances: Customer incentives for clothes washers, dishwashers, refrigerators, and high-efficiency electric storage water heaters. HVAC: Customer incentives for high-efficiency heating and cooling equipment and services, duct sealing, and evaporative cooling equipment. Windows and Insulation: Customer incentives for attic, wall, and floor insulation, and high efficiency windows. Appliance Recycling: Incentives for recycled clothes washers. The Cadmus Group Inc s (Cadmus) evaluation of the HES Program consisted of eight primary tasks, feeding into two evaluations: impact and process. The impact evaluation evaluated two key components: the realization rate and the net-to-gross ratio (NTG). The realization rate accounted for both measures not providing assumed program savings (those which were not installed in originally intended location or been removed), as well as instances where the measure was saving a customer more energy than originally expected due to an adopted change in behavior or a conservative savings estimate. NTG the combination of freeridership and spillover discounted savings from units that would have been installed in the program s absence, and credits the program for savings achieved through influence, but was not accounted for in program records. The process evaluation investigated topics such as: program satisfaction; the event sequence from a customer learning about a program through incentive payment; and 1 2 PacifiCorp manages demand-side management programs in five of its six state jurisdictions. Programs in Oregon are managed by the Energy Trust of Oregon. Based on information contained in PacifiCorps 2009 Review of DSM Programs Idaho located at The Cadmus Group, Inc. / Energy Services 5

8 other qualitative issues. Table 1 illustrates the relationships between the eight primary tasks and three evaluation components (process, realization, and NTG). Table 1. Summary of Evaluation Approach Action Impact Realization Rate Net-to-Gross Process Participant Survey X X X CFL Nonparticipant Survey X X X CFL Retailers X X Billing Analysis Retailer Surveys Stakeholder Interviews Secondary Research Secondary Data Analysis X X X X X Details Used for calculating the NTG, and assessing effectiveness of the program s implementation. (n=429 customers who purchased 549 qualifying measures) Used for calculating CFL leakage out of the service territory. (n=141)/realization rate. 3 Used for calculating CFL leakage out of the service territory. (n=11)/realization rate. 4 Used for calculating insulation realization rate. (n=258 participants and 1,032 control customers) 5 Provides insight into calculating NTG and assessing program delivery. (n=30) Provides insight into program design and assessing program delivery. (n=4) Review results of recent residential products and services evaluations of other utility programs to compare for best practices Determines per-unit savings, based on deemed reported savings, DEER, and RTF. Realized Gross Savings The first step in conducting the impact evaluation was to confirm achieved gross savings from the program. The data provided by Rocky Mountain Power s program implementation contractor, PECI was compared to Rocky Mountain Power s previously reported results for each of the three program years. Deemed savings values were validated by comparing the PECI claimed values against similar measures from the 2006 and 2008 Database for Energy Efficient Resources (DEER) in California as well as the Northwest Power and Conservation Council Regional Technical Forum (RTF) 5th and 6th Power Plans. 6 This analysis determined that the savings per measure values used by Rocky Mountain Power were reasonable and consistent. In the comparison of per measure values, insulation measures varied the most with the values used CFL Leakage was found to be inconclusive for the Idaho Territory. Ibid Due to too low participation in Idaho to perform an adequate billing analysis, the billing analysis was used from the Pacific Power HES Washington territory. More detail of the billing analysis is provided in the Billing Analysis section of the report. and respectively. The Cadmus Group, Inc. / Energy Services 6

9 by Rocky Mountain Power. The savings for insulation measures were higher than those assumed in DEER but lower than the values assumed by RTF Zone 3. 7 The realization rate for most measures was determined by verifying installation rates of program measures through phone surveys. For insulation measures, we attempted to determine a realization rate using and analyzing billing analysis methodology data however it was found that the level of participation for the years evaluated were too low for the results to be statistically relevant to rely on this method. Consequently, the billing analysis performed for the Pacific Power Washington HES Program was used as a proxy for the Idaho territory based on similar climate and appliance saturation. The CFL realization rate represented the leakage rate outside of Rocky Mountain Power s territories, measured through surveys of noncustomers residing in close proximity to Rocky Mountain Power s service territory. Cadmus evaluated leakage of program-subsidized CFLs out of the Rocky Mountain Power service territory. This analysis covered Rocky Mountain Power s Idaho territory during 2006 through While leakage was studied, and is presented in this report, the findings only attempted to measure the number of bulbs for which Rocky Mountain Power provided incentives, which ultimately left its service territory. The study did not take into account other factors such as the rural nature of the territory, lending itself to a very high reverse leakage effect or other regional programs offered by Bonneville Power Administration (BPA) and Northwest Energy Efficiency Alliance (NEEA) in the same territory which would cut down the number of bulbs sold through the Rocky Mountain Power CFL program. Consequently this report assumes that the net leakage is zero, and additional study is recommended to measure these other impacts as well as to confirm the findings from this initial leakage study. Additionally, the self reported data collected from retailers and consumers tends to be unreliable due to a lack of incentive for them to confirm the historic purchases. The leakage study in Idaho has a sensitivity of ± 22.8% without accounting for the reverse leakage effect. Table 2 shows aggregate realization rate values for each program year, based on survey responses, insulation billing analyses, and CFL leakage. 7 Estimates of savings per square foot for wall, floor and attic insulation were derived from Regional Technical Forum (RTF) data in conjunction with heating and cooling degree day data as RTF does not include cooling load savings calculations. The Cadmus Group, Inc. / Energy Services 7

10 Table 21 in this report s Gross Savings section shows realization rates for each measure. Table 2. HES Gross Energy Savings Installation Adjusted Program Year Gross Savings (kwh) Realized Gross Savings (kwh) Realization Rate , , % , , % , , % All Years 1,554,972 1,515, % Net-to-Gross Freeridership and spillover results, determined through phone surveys, were applied to realized gross savings to estimate the net energy savings for each measure. NTG averages for 2006, 2007, and 2008 were 0.74, 0.84, and 0.83, respectively. Table 3 summarizes evaluation results for each program year and totals all three years. Table 3. Evaluated Savings Summary Program Year Realized Gross Savings (kwh) Evaluated Net Savings (kwh) NTG Factor , , % , , % , , % All Years 1,515,981 1,240, % Cost-Effectiveness Program benefits are comprised of energy savings and their associated avoided costs. The energy savings used in the cost-effectiveness analysis are the evaluated kwh from this study. Table 4, Table 5, and Table 6 present the results of the cost-effectiveness analysis for the Program in 2006, 2007, and 2008 respectively. Table 7 summarizes cost-effectiveness analysis for the program for all years ( ). All analyses are based on the Rocky Mountain Power 2008 IRP 60% load factor (LF) eastside residential lighting decrement 8. The analysis shows the program to be cost-effective each of the three years evaluated under PTRC, TRC, UCT and PCT cost benefit perspectives. Detailed incremental costs and measure lives can be found in Appendix B. The mix of measures included by year influenced the cost-effectiveness tests significantly. Specifically, the relative mix in the number of CFLs sold and the amount of insulation participation had the most significant impact. 8 IRP decrements are detailed in Appendix G of PacifiCorp s 2008 Integrated Resource Plan Volume II Appendices: urce_planning_6.pdf The Cadmus Group, Inc. / Energy Services 8

11 Table 4. Cost-Effectiveness Summary for 2006 IRP 60% LF Eastside Residential Lighting Decrement Cost Effectiveness Test Levelized $ / kwh Costs Benefits Net Benefits Benefit / Cost Ratio Total Resource + Conservation Adder (PTRC) $0.089 $146,766 $121,436 -$25, Total Resource No Adder (TRC) $0.089 $146,766 $110,396 -$36, Utility (UCT) $0.065 $107,868 $110,396 -$2, Ratepayer Impact (RIM) $0.137 $226,252 $110,396 -$115, Participant (PCT) $0.040 $65,539 $145,025 $79, Lifecycle Revenue Impact ($/kwh) $ Discounted Participant Payback (Years) 2.73 Table 5. Cost-Effectiveness Summary for 2007 IRP 60% LF Eastside Residential Lighting Decrement Cost Effectiveness Test Levelized $ / kwh Costs Benefits Net Benefits Benefit / Cost Ratio Total Resource + Conservation Adder (PTRC) $0.061 $298,430 $385,716 $87, Total Resource No Adder (TRC) $0.061 $298,430 $350,651 $52, Utility (UCT) $0.053 $261,044 $350,651 $89, Ratepayer Impact (RIM) $0.132 $647,746 $350,651 -$297, Participant (PCT) $0.034 $165,539 $514,854 $349, Lifecycle Revenue Impact ($/kwh) $ Discounted Participant Payback (Years) 0.97 Table 6. Cost-Effectiveness Summary for 2008 IRP 60% LF Eastside Residential Lighting Decrement Cost Effectiveness Test Levelized $ / kwh Costs Benefits Net Benefits Benefit / Cost Ratio Total Resource + Conservation Adder (PTRC) $0.118 $585,447 $424,308 -$161, Total Resource No Adder (TRC) $0.118 $585,447 $385,734 -$199, Utility (UCT) $0.091 $450,394 $385,734 -$64, Ratepayer Impact (RIM) $0.176 $873,045 $385,734 -$487, Participant (PCT) $0.073 $360,706 $648,304 $287, Lifecycle Revenue Impact ($/kwh) $ Discounted Participant Payback (Years) 4.12 The Cadmus Group, Inc. / Energy Services 9

12 Table 7. Cost-Effectiveness Summary for IRP 60% LF Eastside Residential Lighting Decrement Cost Effectiveness Test Levelized $ / kwh Costs Benefits Net Benefits Benefit / Cost Ratio Total Resource + Conservation Adder (PTRC) $0.088 $932,184 $848,427 -$83, Total Resource No Adder (TRC) $0.088 $932,184 $771,298 -$160, Utility (UCT) $0.07 $741,392 $771,298 $29, Ratepayer Impact (RIM) $0.151 $1,586,250 $771,298 -$814, Participant (PCT) $0.051 $532,384 $1,186,449 $654, Lifecycle Revenue Impact ($/kwh) $ Summary of Findings Key findings from The Cadmus Group s evaluation follow below: 2006 was the first year the program was offered in Idaho and as seen often in the first year, there are significant costs in setting up the program which impact the benefit/cost ratio of the program. Although the program benefit/cost ratio improved in 2007, in 2008 the other regional programs 9 offered by BPA and NEEA in the same territory cut down the number of bulbs sold through the Rocky Mountain Power CFL program, which negatively impacted the benefit/cost ratio of the entire program. Excluding CFL participants, program participation and gross savings between 2007 and 2008 (the two full program years) increased 54%. Additionally, participation increased for 10 of 12 measures that had participation in 2007 and During the same period, gross savings (kwh) increased for 6 of 12. However, CFL participation declined dramatically between 2007 and This decline can be attributed to the misalignment of pricing of bulbs offered through this program and those offered through the BPA and NEEA programs. When CFL participants are included during the same period, overall program participation and gross savings (kwh) decreased by 19% and 20 % respectively. The NTG factors of Rocky Mountain Power s program were consistent with those found in recent evaluations of other utility programs offering similar measures. In particular the freeridership is consistent with other reports, while the approach taken for the HES spillover is more conservative compared with evaluations where spillover was quantified. Evaluation results indicate that about a third of the total program CFLs subsidized through the HES Program may have leaked outside the territory. 10 However, due to the 9 10 During the program cycles, incentives for lighting measures were offered by three regional programs, in overlapping service territories. While the three programs (BPA, NEEA and Rocky Mountain Power) were effectively coordinated for most of this time, each underwent restructuring at different times in The result was that the programs were out of sync in 2008 and participation in Rocky Mountain Power program dropped off significantly, negatively impacting the program cost-effectiveness. This level of leakage is consistent with areas that were considered in close proximity with other territories in a recent study for California IOUs (PG&E, SCE, and SDG&E) performed by KEMA and Cadmus. The study did The Cadmus Group, Inc. / Energy Services 10

13 rural nature of the territory and competing programs the reverse leakage would be equal or more than the corresponding leakage, a benefit of a coordinated regional approach to influence the adoption of a measure i.e. point-of-purchase buy-down program designs. Gross savings were primarily from CFLs, clothes washer, and attic insulation, accounting for 51%, 18%, and 14%, respectively, for the combined three years. The same three measures CFLs, clothes washer-tier two, and attic insulation accounted for 12%, 22%, and 30%, respectively, of incentive dollars spent over the combined three years. Overall, 83% of participants were satisfied; 53% of total participants were very satisfied ; 30% were satisfied. Providing standard offerings in vastly different demographic, geographic, and climatic areas proved challenging, and required consistent analysis and monitoring. Summary of Recommendations Enhance the quality control process. Program site inspections in regions with greater participation revealed the need to alter the quality control methodology to increase effectiveness. PECI has already addressed this by increasing the number of staff conducting inspections and instituting use of more sophisticated equipment for greater precision during inspections (such as laser measures for square footage) for the HES program. Procedures for selecting QC participants should also be reviewed, as customers without phone numbers appear to have been eliminated from the pool for QC visits. Program promotion. Relationship building should continue with dealers and trade allies as these channels directly contact prospective participants at times when customers decide about equipment purchases. Benefit-Cost Ratio. The benefit-cost ratios presented indicate the program is struggling. As described in the key findings, Cadmus would suggest that the first year of the program was impacted by startup costs and 2008 was impacted by the significant reduction in the number of CFLs. The CFL numbers in 2009 were higher than in 2007 and would indicate that the benefit-cost ratio would look positive. Program data collection. PECI should investigate options for more robust data collection processes and construct a database designed to allow better and more efficient information tracking and data transfer between implementer and utility as needed. not take into account other factors such as the rural nature of the territory, lending itself to a very high reverse leakage effect or other regional. Consequently this report assumes that the net leakage is zero. The Cadmus Group, Inc. / Energy Services 11

14 2. Program Overview Program Description The Idaho Home Energy Savings Program (HES) is part of Rocky Mountain Power s ongoing Demand-Side Management (DSM) resource acquisition program. The program s overarching objective is to decrease customer electricity usage (kwh) by offering incentives for more efficient products and services received or installed by Idaho customers. The incentives offered through the program are designed to encourage customers shopping for new appliances to seek high efficiency models, and those looking for services to use the most energy-efficient option. The HES Program provides incentives for residential customers to purchase efficient products and services. The program targets the following key measures, which affect a home s overall efficiency: Lighting: Upstream incentives for manufacturers to reduce retail prices on compact florescent lamps. Customer incentives for light fixtures and ceiling fans. Appliances: Customer incentives for clothes washers, dishwashers, refrigerators, and high-efficiency electric storage water heaters. HVAC: Customer incentives for high-efficiency heating and cooling equipment and services, duct sealing, and evaporative cooling equipment. Windows and Insulation: Customer incentives for attic, wall, and floor insulation, and for windows. Appliance Recycling: Incentives for recycled clothes washers. Eligibility Requirements Residential electric customers in Rocky Mountain Power s service territory and landlords owning rental properties served by Rocky Mountain Power, where tenants are billed on standard rate schedules, qualify for the HES program. To participate, customers must purchase a qualified product or service, complete an application, and provide required documentation, such as product receipts. Incentive checks are issued within 45 days of receipt of a completed and approved incentive application. Incentive Levels The program offers incentives for qualified energy-efficient appliances, lighting, heating, and cooling equipment, windows and insulation, and certain appliance recycling measures. Table 8 shows variations for each measure, and indicates when measures were introduced to the program. Appendix A: Available Measures Tariff Information contains a copy of the full tariff schedules, including a list of incentive levels for each product eligible in 2006, 2007, and Rocky Mountain Power and PECI have made to program targets over the three years since the measures May 2006 introduction. The Cadmus Group, Inc. / Energy Services 12

15 Table 8. Program Measures and Incentive Variations Measure Start Date Start Incentive 2008 Incentive 11 CAC/Heat Pump Tune-Ups 5/31/2006 $125 $125 Ceiling Fans 1/1/2007 $20 $20 Central A/C Equipment 1/1/2007 $275 $275 CFL 5/31/2006 Manufacturer buydown not-to-exceed $1.35 and Retail price not-to-exceed $1.50 Manufacturer buydown not-to-exceed $1.35 and Retail price not-to-exceed $0.99- $2.75 Clothes Washer (MEF ) 5/31/2006 $75 $0 Clothes Washer (MEF ) 5/1/2008 $50 $50 Clothes Washer (MEF 2.0+) 5/1/2008 $100 $100 CW Recycling 5/31/2006 $25 $25 Dishwasher 1/1/2007 $20 $20 Duct Sealing 5/31/2006 $200 $200 Electric Water Heater 5/31/2006 $50 $50 Evaporative Cooler 5/31/2006 $275 $100 Fixture 5/31/2006 $20 $20 Heat Pump Conversion 5/1/2008 $375 $375 Heat Pump Upgrade 5/1/2008 $275 $275 Insulation: Attic 5/31/2006 $1.00/sq. ft. $.50/sq. ft. Insulation: Floor 5/31/2006 $1.00/sq. ft. $.50/sq. ft. Insulation: Wall 5/31/2006 $1.00/sq. ft. $.50/sq. ft. Proper CAC Installation 1/1/2007 $125 $125 Proper CAC Sizing 1/1/2007 $75 $75 Refrigerator 5/31/2006 $20 $20 Windows 5/31/2006 $1.50/sq. ft. $1.50/sq. ft. 11 Incentive value available on December 31, 2008 The Cadmus Group, Inc. / Energy Services 13

16

17 3. Evaluation Methods Analytical Methods The evaluation plan consisted of eight primary tasks. Table 9 presents a brief description of each task. Subsequent chapters provide additional details regarding methodologies for each evaluation task. Table 9. Summary of Evaluation Approach Action Impact Realization Rate Net-to-Gross Process Participant Survey X X X CFL Nonparticipant Survey X X X CFL Retailers X X Billing Analysis Retailer Surveys Stakeholder Interviews Secondary Research Secondary Data Analysis X X X X X Details Used for calculating the NTG, and assessing implementation. (n=429 customers who purchased 549 qualifying measures) Used for calculating CFL leakage out of the service territory. (n=141) 12 Used for calculating CFL leakage out of the service territory. (n=11) 13 Used for calculating the insulation realization rate. (n=258 participants and 1,032 control customers) 14 Provides insight into calculating NTG and program delivery. (n=30) Provides insight into program design and delivery. (n=4) Reviews results of recent residential products and services evaluations. Determines per-unit savings, based on deemed reported savings, DEER, and RTF. Evaluation Question Overview The HES Program covers a broad range of products and services, each with varying participation levels. Evaluation questions focused on measures estimated to produce the greatest kwh savings or incurring the largest incentive dollar costs (the primary measures). Key impact and process evaluation questions driving the study are listed below. Impact Questions 1. What were the gross and net energy savings generated by a participating product or service? CFL Leakage was found to be inconclusive for the Idaho Territory. Ibid. Due to too low participation in Idaho to perform an adequate billing analysis, billing analysis was used from the Pacific Power HES Washington territory. More detail of the billing analysis is provided in the Billing Analysis section of the report. The Cadmus Group, Inc. / Energy Services 15

18 2. What percentage of participating appliances and services would have been purchased in the program's absence? 3. What additional appliances and services were purchased due to the program? 4. How many CFLs left the service territory? 5. How did evaluated savings compare to previously reported savings for each program year? 6. What were actual program costs and benefits? Process Questions 1. How did program participants become aware of the program? 2. How satisfied were participants with program delivery, including schedule, communications, implementer performance, incentive, and overall? 3. What improvements did participants recommend? 4. From the implementer s perspective: how well did the program work? What could be improved? Participant Surveys The evaluation team administered a participant survey for residential appliances and service evaluations. The surveys included questions addressing: Context and Decision-Making Processes. These questions addressed key aspects of the customer s decision-making process, and informed freeridership, spillover, and verification analysis. Program Satisfaction. These questions collected process-related responses regarding participants satisfaction with the program. The questions also addressed the likelihood participants would refer others to the program. Other questions sought to determine the customers sources of program information, descriptions of products and services, consideration of alternative products and services, and demographics. The sampling plan addressed all of study goals, including determining energy savings for each of the three calendar years. As energy consumption, operating patterns, and purchasing decisions were unique to each measure, accurate determination of energy savings required independent evaluation of each primary measure. PECI provided multiple files, including customers, measures, retailer, gross savings, incentive dollars, and units for each calendar year. Once the files were verified, the evaluation team assigned each participant a random number, and prioritized the call list based on these numbers. Table 10, Table 11, and Table 12 show numbers of participants, actual numbers of survey responses, and precision estimate at the 90% confidence interval based on the number of surveys. The Cadmus Group, Inc. / Energy Services 16

19 Table 10. Measure Participation 15 Measure All Years Ceiling Fans Central A/C Equipment CFL 918 1, ,996 Clothes Washer ,126 Clothes Washer-Tier One Clothes Washer-Tier Two Dishwasher Electric Water Heater Evaporative Cooler 1 1 Fixture Heat Pump Upgrade 1 1 Insulation: Attic Insulation: Floor Insulation: Wall Proper CAC Install Proper CAC Sizing Refrigerator Windows Total 1,135 2,693 2,175 6,003 Table 11. Participant Surveys Completed Measure All Years Ceiling Fans Central A/C Equipment CFL Clothes Washer Clothes Washer-Tier One Clothes Washer-Tier Two Dishwasher Electric Water Heater Evaporative Cooler Fixture Heat Pump Upgrade Insulation: Attic Insulation: Floor Insulation: Wall Proper CAC Install Proper CAC Sizing Refrigerator Windows Total The participant count includes counts of individual incentives: for appliance measures, customers are limited to one incentive per household, except for ceiling fans and fixtures; for home improvement, the number of participants equals the number of total jobs; for HVAC, contractor incentives are not counted as a separate participant; for CFLs, participants are estimated at 10 bulbs per participant, a rate based on information from PECI and the PAC-E Annual Report of Idaho Demand Side Management Activities. The Cadmus Group, Inc. / Energy Services 17

20 Table 12. Level of Precision 16 Measure All Years Ceiling Fans N/A N/A N/A N/A Central A/C Equipment N/A N/A N/A N/A CFL N/A N/A N/A N/A Clothes Washer 12% 9% 9% 1.75% Clothes Washer-Tier One N/A N/A 57% 57.46% Clothes Washer-Tier Two N/A N/A 9% 9.42% Dishwasher N/A 14% 10% 7.96% Electric Water Heater 82% 40% 21% 18.17% Evaporative Cooler N/A N/A N/A N/A Fixture 0% 82% 34% 29.24% Heat Pump Upgrade N/A N/A N/A N/A Insulation: Attic N/A 19% 12% 9.95% Insulation: Floor N/A 47% 32% 26.51% Insulation: Wall N/A 37% 34% 24.52% Proper CAC Install N/A N/A N/A N/A Proper CAC Sizing N/A N/A N/A N/A Refrigerator 16% 12% 9% 6.68% Windows N/A 45% 18% 16.51% Overall 10.52% 6.2% 4.2% 3.35% Billing Analysis During the evaluation a billing analysis was attempted for the Idaho insulation participants. After applying the screening criteria only 93 participants remained. This pool of participants was too small and did not yield robust enough savings estimates to accurately assess actual energy savings. Because the Pacific Power Washington HES program expected savings and participant climate was a close proxy for the Idaho program the Washington billing analysis realization rate was extrapolated to the Idaho program for insulation measure installations. 17 The savings estimate was determined from a pooled conditional savings (CSA) regression model, including the following groups: Insulation (combined attic, wall, and floor insulation for ); and Nonparticipant homes, serving as the baseline. Program Data and Billing Analysis Methodology Several data sources were used to create the final billing analysis modeling dataset, including: Program data were collected and provided by PECI (including account numbers, measure type, installation dates, square footage of insulation installed, and expected savings for the entire participant population) Level of precision at the 90% confidence interval. Billing analysis was only performed for the insulation measures. Energy savings achieved through installation of other measures were not large enough, relative to the total energy consumption of households where installed, to allow reliable billing analysis. The Cadmus Group, Inc. / Energy Services 18

21 Control group data. The control group was selected from nonparticipating customers, whose energy use matched pre-participation energy use of participating households. To ensure adequate coverage of the nonparticipating population, the evaluation team included four times more nonparticipants than participants in the billing analysis. Billing data were provided by Rocky Mountain Power, including all customer accounts in the service territories being evaluated (Idaho, Washington, and Utah). The data included: meter-read dates, days in the billing cycle, and five years of kwh consumption for both participants and the large pool of nonparticipants. The final sample used in the billing analysis savings model consisted of 258 participants and 1,032 control customers. 18 Weather data included daily average temperatures from January 1995 to December The billing analysis team matched program data with billing data and separated the data into groups of participants and nonparticipants. Daily heating and cooling degree days were matched to respective monthly read date periods in billing data for use in weather-adjusted savings modeling. Data Screening Table 13 shows participant and nonparticipant screening criteria and attrition for Idaho and Table 14 shows participant and nonparticipant screening criteria used in the billing analysis from Washington. To ensure the final model used complete pre- and post-participation billing data, the analysis team selected accounts with a minimum of 300 days in the pre- and post-periods (i.e., before the program s 2006 launch, and after the measures had been installed). Additionally, the analysis team removed outlier accounts with less than 1,000 kwh per year or more than 50,000 kwh per year. As expected savings were low compared to pre-period usage, a stringent 30% change screen was used in the analysis. While typically a 50%-70% change screen is applied to most billing analyses where major measures were done, a 30% change screen was placed on the billing data in this case since large savings and large increases in consumption for both participants and nonparticipants were not expected given the small expected savings relative to the pre usage. This screen also ensures that the nonparticipants are better matched with participants. Any account changing usage by more than 30% from the pre- and post period was removed from analysis. 18 The sample for the billing analysis savings model attempted in Idaho consisted of 93 participants and 372 control customers. The Cadmus Group, Inc. / Energy Services 19

22 Table 13. Screen for Inclusion in Billing Analysis for Idaho Screen Nonparticipant Attrition (n) Participant Attrition (n) Remaining Nonparticipant (n) Remaining Participant (n) Original Measures Participant Database and 10 times Nonparticipant Sample by Zip 2, code Matched Billing Data Sample (accounts that could be 2, mapped to the billing data) Less than 300 days in pre or post period 21-1, Less than 1,000 kwh in pre or post period , More than 50,000 kwh in pre or post period 3-1, Changed Consumption by more than 30% from pre to , post 19 Expected Savings over 70% of pre Consumption , No Expected Savings , Nonparticipant Sample Selection Final Sample The expected engineering estimates of savings were very low compared to the pre-usage. A 30% change screen was placed on the billing data since large savings and large increases in consumption for both participants and nonparticipants were not expected given the small expected savings relative to the pre usage. This screen also ensures that the nonparticipants are better matched with participants. 20 If the expected engineering estimates of savings exceed 70% of the pre-consumption, then either there was a mismatch between the participant measure installation data and the billing data account or address, or the participant had a vacancy in the pre-period. Either one of these, would have led to unreasonable savings estimates for that customer, hence they are dropped. The Cadmus Group, Inc. / Energy Services 20

23 Table 14. Screen for Inclusion in Billing Analysis for Washington 21 Screen Nonparticipant Attrition (n) Participant Attrition (n) Remaining Nonparticipant (n) Remaining Participant (n) Original Measures Participant Database and 10 times Nonparticipant Sample by Zip 4, code Matched Billing Data Sample (accounts that could be 4, mapped to the billing data) Less than 300 days in pre or post period , Less than 1,000 kwh in pre or post period , More than 50,000 kwh in pre or post period 21-3, Changed Consumption by more than 30% from pre to 1, , post Expected Savings over 70% of pre Consumption , No Expected Savings , Nonparticipant Sample Selection 1,844-1, Final Sample 1, Dealer Surveys Cadmus conducted interviews with 30 participating Idaho dealers. These 30 dealers were chosen from a list of participating dealers provided by PECI. In order to ensure random selection of dealers, the evaluation team assigned each dealer a random number. Once the random numbers were assigned, the list was sorted in descending order by incentive totals and then again by the random number. The sample list was verified to have representation of the more significant measures based on gross kwh saved and dealer categories including big box stores, smaller retailers, and contractors. The surveys sought to collect information about the following attributes: effectiveness of program marketing materials; dealer program promotion; and overall dealer perceptions of the program. Additionally, dealers were asked about their methods of recycling old clothes washers. CFL Nonparticipants and CFL Retailer Surveys Cadmus conducted two surveys. The first targeted households not served by Rocky Mountain Power (noncustomers) within 20 miles of Rocky Mountain Power service territory. The second targeted retailers participating in the Rocky Mountain Power program. 21 The Washington billing analysis was applied to Idaho due to Idaho data limitations and relatively comparable service territory characteristics. The billing analysis screen table shows the attrition in the Pacific Power Washington HES program billing analysis model as the screening criteria were applied. The Cadmus Group, Inc. / Energy Services 21

24 The household survey sample consisted of 141 randomly selected households in areas surrounding Rocky Mountain Power s service territory. Surveyors asked these non-rocky Mountain Power customers: how many CFLs their household had purchased since the beginning of 2006; stores from which they purchased those CFLs; and their percentage of purchases from each of those stores. In each region surveyed, the goal was to complete 70 surveys with non- Rocky Mountain Power customers to achieve a 90% confidence with a +/- 10% level of precision. The retailer survey sought to obtain information on the number of CFLs sold since the beginning of 2006, and the percentage of sales subsidized by the program. Surveys were completed with all 11 stores reported to have CFL sales subsidized through the program. The evaluation team combined the household and retailer survey responses to determine numbers of CFLs purchased outside the service territory since 2006, and the percentage of those being program bulbs. This approach provided an estimate of historic CFL leakage, though the methodology relied on respondents memories. In can be difficult for people to recall specifics related to purchases of relatively inexpensive items, such as CFLs. The reverse leakage was not quantified for this report. Stakeholder Interviews To assess the program's effectiveness and implementation, the evaluation team conducted interviews with a number of stakeholders familiar with the program. Specifically, the evaluation team interviewed four Rocky Mountain Power and PECI stakeholders. Table 15 provides details regarding stakeholders interviewed. Table 15. Stakeholder Interviews Title Demand Side Management, Director Class 2 DSM Segment Manager, All Sectors Residential Programs, Director Residential Programs Associate Director Organization Rocky Mountain Power Rocky Mountain Power PECI PECI Project managers conducted stakeholder interviews using a guide designed to examine the program s design, delivery, management, communication, effectiveness, and future. Information obtained from stakeholders informed the following evaluation elements: Determination of program progress; and Identification of changes during implementation. Secondary Research To ensure the evaluation aligned with industry best practices for residential product and service evaluations, the team reviewed multiple evaluation reports for methodology and participant survey instruments. The Cadmus Group, Inc. / Energy Services 22

25 Data collected through secondary research aided the following evaluation elements: Development of the gross savings methodology; Development of the participant, nonparticipant, and retailer survey instruments; and Development of the net savings methodology. Secondary Data Analysis To leverage existing participating measure data sources, the evaluation team obtained datasets detailing energy consumption of similar measures from the 2006 and 2008 DEER database as well as the RTF 5th and 6th Power Plans. Combining this information with data on program participants yielded an estimate of the program s gross savings. Detail regarding this process is provided in the following chapter. The analysis informed the following evaluation elements: Determination of estimated per-unit energy savings; and Estimate of program gross savings. Data Sources The evaluation team used the following data sources to inform the impact and process evaluations: Final program databases (provided by PECI). Information gathered through participant, nonparticipant, and retailer surveys. Information gathered through stakeholder interviews. The ENERGY STAR sales database. 22 Other recent residential appliance and services evaluations. The 2006 and 2008 DEER database in California, and the Northwest Power and Conservation Council RTF 5th and 6th Power Plans ENERGY STAR Resources for Appliance Manufacturers and Retailers: and respectively. The Cadmus Group, Inc. / Energy Services 23

26

27 4. Survey and Interview Findings In addition to collecting information needed to gauge freeridership and spillover, the end-use customer surveys and dealer interviews explored a number of other topics, including marketing efforts, new vs. existing installations, incremental costs, market trends, and program satisfaction. Each of these areas is discussed below. Customer Survey Analysis Marketing Investment in mass media channels, such as television, radio, and print advertising, was comparatively less than more targeted approaches. In Idaho, 38% of insulation participants heard about the program from a friend or family member and about one-fourth heard about the program from the contractor who installed qualifying insulation in their home. Bill inserts were reported by 13. Survey responses indicate that newspaper and television advertising were not as effective as word of mouth or contractor promotion. These findings appear to be in line with investment levels and are presented in Figure 1. More than one-fourth of windows and appliances participants heard about the program from the dealer who sold the measure or the installer who installed the measure. One-fifth of participants indicated that they heard about the program from inserts in their monthly utility bills. Another 10% indicated that they heard about the program by word-of-mouth. 24% of windows and appliances participants reported hearing about the program from another utility source such as newsletters or the Rocky Mountain Power website. These findings are shown in Figure 2 below. Figure 1. How Insulation Participants Heard About HES Program (n = 55) In-store marketing material TV ad Newspaper Other Salesperson Don't Know Bill insert Contractor/Installer marketing material Friend/Family Member 2% 2% 4% 5% 5% 7% 13% 24% 38% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% The Cadmus Group, Inc. / Energy Services 25

28 Figure 2. How Windows and Appliances Participants Heard about HES Program (n = 372) Utility Energy Newsletter Utility Welcome Kit Print Ad Utility Call Center Other Online Ad / RMP Web Site Brochure Word of Mouth Don't Know Bill insert Dealer/Installer 2% 3% 3% 3% 5% 6% 7% 10% 14% 20% 26% 0% 5% 10% 15% 20% 25% 30% Individual Measure Findings Attic Insulation Attic insulation participants were asked to discuss what they considered to be the most important criteria when purchasing the measure. 20% of the 50 participants discussed the need to regulate the inside temperature of their home as key criteria. Another 18% cited energy bill reductions as key criteria for purchasing the measure. The incentive was the most important criteria for 16% and price was discussed by 12% of participants as key criteria for purchasing attic insulation. Figure 3 below summarizes these findings as well as additional criteria discussed by attic insulation participants. The Cadmus Group, Inc. / Energy Services 26

29 Figure 3. Most Important Criteria for Purchasing Attic Insulation (n = 50) Financing Replace Old Insulation Contractor/Installer Availability Contractor/Installer Pricing Product Brand Energy Efficiency Price Don't Know Program Incentive Reduce Energy Bill Regulate Inside Temperature 2% 2% 4% 4% 4% 4% 12% 14% 16% 18% 20% 0% 5% 10% 15% 20% 25% Participants cited a number of different factors that motivated them to purchase attic insulation through the HES Program. Most respondents (42%) reported that their old insulation was no longer sufficient or missing. The program incentive motivated 14% of participants to install this measure. Both potential energy savings and potentially lower utility bills each influenced 10% of participants to install the measure. As shown in Figure 4 below, 18% also cited other factors that influenced their decision to install attic insulation. These responses included general heating and cooling needs, home remodeling, fixing incorrectly-installed insulation, and recommendations to buy the measure. Figure 4. Factors that Motivated Purchase of Attic Insulation (n = 50) Don't Know 6% Reduce Energy Bill Energy Savings Program Incentive Other 10% 10% 14% 18% Old Insulation Insufficient/Missing 42% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Participants also discussed changes in the comfort of their home after installing attic insulation. 64% of the 44 participants reported that in general the inside temperature of their homes is comfortable. While not specifically asked in this question, one-fourth of participants reported lower energy bills following the installation of attic insulation. Two participants indicated that there was no noticeable difference and one participant reported that his air conditioner and The Cadmus Group, Inc. / Energy Services 27

30 furnace ran less frequently after installing attic insulation. Figure 5 below summarizes these findings. Participants were also asked how their air conditioner or furnace usage has changed since installing the measure. Of the 16 respondents, 81% indicated that their air conditioner or furnace runs less frequently and 6% reported that their HVAC system provided better heating and cooling since installing the measure. Figure 6 summarizes these findings below. Figure 5: Reported Changes in Comfort of Home Attic Insulation (n = 44) Less Noise Don't Know AC/Furnace Run Less No Difference 2% 2% 2% 5% Lower Energy Bills 25% Comfortable Inside Temperature 64% 0% 10% 20% 30% 40% 50% 60% 70% Figure 6. Changes in AC/Furnace Attic Insulation (n = 16) Cools/Warms Better, 6% Don't Know, 13% Runs Less Frequently, 81% Survey administrators also asked participants about any changes in their monthly utility bill since installing attic insulation. Three-fourths of the 27 respondents indicated that their monthly bill was lower than before installing attic insulation. One participant reported that their bills were higher than before installing the measure. These findings are presented in Figure 7 below. The Cadmus Group, Inc. / Energy Services 28

31 Figure 7. Changes in Monthly Utility Bill Attic Insulation (n = 27) Slightly Lower, 18% Higher, 4% Significantly Lower, 18% Lower, 56% No Change, 4% Floor Insulation Seven floor insulation participants discussed key criteria for purchasing the measure. In addition to criteria related to price, two participants indicated that the R-value was important and one participant stated that he simply needed to replace existing insulation. These findings are summarized in Figure 8. Figure 8. Key Criteria for Purchasing Floor Insulation (n =7) Contractor/Installer Pricing 1 Price 2 Other Four indicated that following the installation of the measure, their inside temperature was generally more comfortable. Additionally, one respondent reported that his AC/furnace runs less frequently since installing the measure. One respondent reported that his monthly utility bill was slightly lower following the installation of the measure. Floor insulation participants also discussed factors that motivated them to purchase the measure. For three (43%) of these participants, their old wall insulation was insufficient to meet the heating and cooling needs of their home. Two more participants (29%) reported that they The Cadmus Group, Inc. / Energy Services 29

32 installed the measure as part of a home remodeling project. These findings are summarized in Figure 9 below. Figure 9. Factors That Motivated Purchase of Floor Insulation (n = 7) Don't Know 1 Heating/Cooling Needs 1 Home Remodel 2 Old Insulation Insufficient Wall Insulation Nine wall insulation participants discussed key criteria for purchasing the measure as shown in Figure 10. Participants discussed regulating inside temperature, reducing the monthly energy bills, higher R-value, ease of installation, and the need to replace old insulation as key criteria for installing the measure. Figure 10. Key Criteria For Purchasing Wall Insulation (n = 9) Price Don't Know Contractor/Installer Pricing Replace Old Insulation Ease of Install R Value Reduce Energy Bill Regulate Inside Temperature Following the discussion of criteria, participants cited various factors that motivated them to purchase and install the wall insulation measure. Three participants installed the measure because their old insulation was insufficient to meet the heating and cooling needs of their homes. Two participants discussed other factors such as general heating and cooling needs and to correct a prior faulty installation. The Cadmus Group, Inc. / Energy Services 30

33 Figure 11. Factors that Motivated Purchase of Wall Insulation (n = 8) Reduce Energy Bill Program Incentive Energy Savings Other 2 Old Insulation Insufficient Among five participants, three reported that since installing the measure, the inside temperature of their homes has become more comfortable. While not directly asked in this section of the survey, two participants indicated that their AC or furnace runs less and that their monthly utility bills are lower. Figure 12 presents these findings below. When asked directly, three wall insulation participants reported that their AC or furnace runs less frequently since installing the measure. Figure 13 presents these findings. Four participants also provided information regarding changes in their monthly utility bills since installing the measure. All four participants reported a difference since installing the measure; with two participants each reporting lower bills and slightly lower bills. These findings are presented in Figure 14. Figure 12. Changes in Comfort of Home Since Installing Measure (n = 5) Lower Energy Bills 1 AC/Furnace Run Less 1 Comfortable Inside Temperature The Cadmus Group, Inc. / Energy Services 31

34 Figure 13. Changes in AC/Furnace Since Installing Measure (n = 4) Cools/Warms Better 1 Runs Less Frequently Figure 14. Changes in Monthly Utility Bill (n = 4) Slightly Lower, 50% Lower, 50% Clothes Washers Participants were asked to list the factors that motivated them to purchase a qualifying clothes washer. More than half (57%) of respondents indicated that their old clothes washer was broken and therefore needed to be replaced. For the 22% of respondents who cited other factors, the most common factor was appliance features such as front-loading, greater washing capacity, or water efficiency.13% purchased a new clothes washer to take advantage of potential energy savings associated with high efficiency appliances. The program incentive motivated 2% of participants to purchase a new clothes washer. Additionally, some participants purchased a new clothes washer following the purchase of a new home. These findings are presented in Figure 15 below. The Cadmus Group, Inc. / Energy Services 32

35 Figure 15. Factors that Motivated Purchase of Clothes Washer (n = 226) Dealer Recommendation 2% Program Incentive 2% Reduce Energy Bills 4% Energy Savings 13% Other 22% Old CW Not Working Properly/Broken 57% 0% 10% 20% 30% 40% 50% 60% The majority of participants (223) installed the clothes washers in their homes; three were installed in locations other than primary residences. These 223 participants were then asked about changes in usage, product satisfaction, and incentive satisfaction. The weekly average of typical wash loads was 7.7 when weighted by the distribution of participant responses. The weekly distribution of wash cycles is presented in Figure 16. Of 97 participants reporting a change in their number of weekly wash loads, 87% reported doing fewer loads per week and 13% reported doing more loads per week. Figure 17 below summarizes wash load frequencies as reported by program participants. Figure 16. Distribution of Weekly Wash Loads Clothes Washers (n = 220) The Cadmus Group, Inc. / Energy Services 33

36 Figure 17. Change in Weekly Wash Loads Clothes Washers(n = 97) More Loads, 13% Fewer Loads, 87% 34% of responding participants indicated that the new clothes washer was easier to use. More than one-fourth reported that the new clothes washer was better in general; however not citing any specific reason. Other participants reported greater washing capacity (14%) and that their new clothes washer uses less water (8%). These findings are presented in Figure 18. Nearly all participants (87 %) were satisfied overall with the amount of the incentive; while the remaining reported that they would have been satisfied with a slightly greater incentive. Approximately 80% of participants who installed high efficiency clothes washers reported that they were very satisfied with their new appliance. Another 17% reported being somewhat satisfied. The remaining respondents reported that they were either not very satisfied or not at all satisfied with their new clothes washer. These findings are presented in Figure 19. Overall, 65% are very satisfied with the program and 29% are somewhat satisfied with the program. Only 1% of participants reported that they were not at all satisfied with the program. These results are shown in Figure 20. The Cadmus Group, Inc. / Energy Services 34

37 Figure 18. Changes in Comfort and Ease of Use Clothes Washers (n = 135) Cleaner Clothes 2% Drier Clothes Other Shorter Cycles 4% 5% 5% Uses Less Water 8% Greater Washing Capacity 14% Better In General 27% Easier To Use 34% 0% 5% 10% 15% 20% 25% 30% 35% 40% Figure 19. Participant Satisfaction with Clothes Washers (n = 222) Somewhat Satisfied, 17% Not Very Satisfied, 2% Not At All Satisfied, 2% Very Satisfied, 79% The Cadmus Group, Inc. / Energy Services 35

38 Figure 20. Overall Program Satisfaction Clothes Washers (n = 223) Don't Know, 3% Not Very Satisfied, 2% Not At All Satisfied, 1% Somewhat Satisfied, 29% Very Satisfied, 65% Refrigerators Just over half of the refrigerator participants purchased a new refrigerator because their old one was not working properly or was broken, as shown in Figure 21 below. Potential savings associated with installing the measure was cited by 13% and the program incentive was cited by 3%. Another 10% reported purchasing the measure following a new home purchase. Figure 21. Factors that Motivated Purchase of Refrigerators (n = 112) Home Remodel Program Incentive 3% 3% Other Features New Home Purchase Energy Savings 9% 7% 10% 13% Old Ref. Not Working Properly/Broken 55% 0% 10% 20% 30% 40% 50% 60% Participants were asked if there had been any changes in ease of use with their new refrigerator. About one-fourth of participants were happy with new features such as a bottom freezer and water/ice cube dispenser. One-fifth cited more space and greater storage capacity as a key change associated with their new refrigerator. Approximately the same number of participants The Cadmus Group, Inc. / Energy Services 36

39 reported no changes in comfort and ease of use as did participants who indicated their refrigerators were generally better. These findings are shown in Figure 22. Figure 22. Reported Changes for New Refrigerators (n = 87) Uses Less Energy Quieter Better Cooling Features Runs Less Less Space Not As Good As Old Better In General No Change More Space Features 1% 1% 2% 5% 5% 7% 17% 18% 20% 24% 0% 5% 10% 15% 20% 25% 30% Overall, participants were very satisfied with their new refrigerators. A combined 9% of participants were either not very satisfied or not at all satisfied with their refrigerator. Nearly all participants (90%) were satisfied with the amount of the incentive and 60% of participants were satisfied overall with the incentive program for energy-efficient refrigerators. Measure and program satisfaction results are presented in Figure 23 and Figure 24. Figure 23. Participant Satisfaction with Refrigerators (n = 103) Not Very Satisfied, 6% Not At All Satisfied, 3% Somewhat Satisfied, 16% Very Satisfied, 75% The Cadmus Group, Inc. / Energy Services 37

40 Figure 24. Overall Program Satisfaction Refrigerators (n = 104) Don't Know, 4% Not At All Satisfied, 2% Not Very Satisfied, 1% Somewhat Satisfied, 33% Very Satisfied, 60% Dishwashers Dishwasher participants discussed different factors that motivated them to purchase an energyefficient dishwasher. 58% of participants purchased a new dishwasher because their old one was inoperable or broken. 9% purchased a new dishwasher because of potential energy savings associated with installing the. Additionally, 18% discussed other factors that motivated their decision to purchase a new dishwasher. These include features, price, and installations following a new home purchase. Figure 25 summarizes these findings. Figure 25. Factors that Motivated Purchase of Dishwashers (n = 79) Price 5% New Home Purchase 5% Features 5% Energy Savings 9% Other 18% Old DW Not Working Properly/Broken 58% 0% 10% 20% 30% 40% 50% 60% 70% On average, participants run their dishwashers 5.8 times per week. Figure 26 shows the distribution of reported weekly cycles. In terms of overall satisfaction with the new dishwasher, 70% of participants were very satisfied and 23% were somewhat satisfied. The Cadmus Group, Inc. / Energy Services 38

41 The remaining participants reported that they were not very satisfied or not at all satisfied with their new dishwasher. Figure 26. Distribution of Weekly Cycles Dishwashers (n = 78) About one-fourth of dishwasher participants reported that their new dishwasher runs quieter than their old one Figure 27. Another quarter indicated that their new dishwasher results in cleaner dishes. Participants also indicated that their new dishwasher is generally better, easier to use, and has greater washing and loading capacity. Figure 27. Changes in Comfort and Ease of Use Dishwashers (n = 38) Not As Good As Old No Change 3% 3% Greater Washing Capacity 10% Easier To Use 13% Better In General 21% Cleaner Dishes Quieter 24% 26% 0% 5% 10% 15% 20% 25% 30% In terms of measure satisfaction 70% reported that they were very satisfied and 23% reported that they were somewhat satisfied. Only 2% reported that they were not at all satisfied with their new dishwasher. Overall, 59% were very satisfied with the incentive program for dishwashers and 29% were somewhat satisfied. These findings are shown in Figure 28 and Figure 29 below. The Cadmus Group, Inc. / Energy Services 39

42 Figure 28. Participant Satisfaction with Dishwashers (n = 79) Not Very Satisfied, 5% Not At All Satisfied, 2% Somewhat Satisfied, 23% Very Satisfied, 70% Figure 29. Overall Program Satisfaction Dishwashers (n = 78) Don't Know, 8% Not Very Satisfied, 3% Not At All Satisfied, 1% Somewhat Satisfied, 29% Very Satisfied, 59% Electric Water Heaters 39% of electric water heater participants installed the measure to take advantage of potential energy savings. Replacing an electric water heater that was broken or not working properly was a secondary factor reported by 28% of participants. Another 17% reported that new trends and features associated with electric water heaters motivated them to purchase the measure. These findings are presented in Figure 30 below. The Cadmus Group, Inc. / Energy Services 40

43 Figure 30. Factors that Motivated Purchase of Electric Water Heaters (n = 18) Recommendation 11% Reduce Energy Bills 5% Trends/Fashions 17% Old Unit Not Working Properly/Broken 28% Energy Savings 39% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Electric water heater participants also discussed changes in comfort and ease of use associated with their purchase. Figure 31 summarizes these results. 46% of participants reported that hot water lasts longer and 15% reported that their water heats up faster as a result of their new electric water heater. However, 38% of participants who installed the measure did not feel that there had been any change in comfort associated with their new electric water heater. Figure 31. Changes in Comfort and Ease of Use Electric Water Heaters (n = 13) Faster Water Heating 2 No Change 5 Hot Water Lasts Longer In general, participants are satisfied with their electric water heaters; with 80% reporting that they are very satisfied and 13% reporting that they are somewhat satisfied. Only 7% are not very satisfied with the measure. Overall, program satisfaction was mixed among participants. While only 36% reported that they are very satisfied, 35% reported that they are either not very The Cadmus Group, Inc. / Energy Services 41

44 satisfied or not at all satisfied with the program, commenting that the incentive amount was too low. Additionally, 29% of participants could not convey their satisfaction with the program. Figure 32 and Figure 33 present these findings. Figure 32. Participant Satisfaction with Electric Water Heaters (n = 15) Somewhat Satisfied, 13% Not Very Satisfied, 7% Very Satisfied, 80% Figure 33. Overall Program Satisfaction Electric Water Heaters (n = 14) Not At All Satisfied, 14% Not Very Satisfied, 21% Very Satisfied, 36% Don't Know, 29% Windows More than half of the windows participants purchased the measure because their old windows were defective or not functioning properly. About one-fifth of participants discussed other motivating factors such as installations following a home remodel, price, quality, and style. Potential energy savings associated with installing energy-efficient windows motivated 14% of participants to purchase the measure. Only 5% of participants cited the incentive as a motivating factor. Figure 34 presents these findings below. The Cadmus Group, Inc. / Energy Services 42

45 Figure 34. Factors that Motivated Windows Purchased (n = 20) Program Incentive 5% Don't Know 10% Energy Savings 14% Other 19% Old Windows Defective 52% 0% 10% 20% 30% 40% 50% 60% Nine windows participants reported changes in comfort and ease of use after installing qualifying windows. Figure 35 shows that five participants (56%) reported that new windows provide better heating and cooling efficiency. Three reported that there had been no changes as a result of installing the measure and one participant reported that the new windows are not as good as their old ones. Figure 35. Changes in Comfort and Ease of Use Windows (n = 9) Not As Good As Old 1 No Change 3 Better Efficiency Windows participants are largely satisfied with the measure; with 87% reporting that they are very satisfied and 13% reporting that they are somewhat satisfied. Overall, two-thirds of participants were satisfied with the program and 27% were somewhat satisfied with the program. Another 7% could not say for certain whether they were satisfied or dissatisfied. Figure 36 and Figure 37 present these findings below. The Cadmus Group, Inc. / Energy Services 43

46 Figure 36. Participant Satisfaction with Windows (n =15) Somewhat Satisfied, 13% Very Satisfied, 87% Figure 37. Overall Program Satisfaction Windows (n = 15) Don't Know. 7% Somewhat Satisfied, 27% Very Satisfied, 66% Lighting Fixtures Seven lighting participants discussed various factors that influenced their decision to purchase this measure. As shown in Figure 38, three participants purchased the measure for potential energy savings and one participant replaced defective lighting. Two participants discussed other factors such as measure price and that a spouse had requested they purchase the measure. One participant was not sure what motivated him to purchase qualifying lighting. Overall, all seven lighting participants were very satisfied with their new lighting and very satisfied with the incentive program for lighting. The Cadmus Group, Inc. / Energy Services 44

47 Figure 38. Factors Motivating Purchase of Lighting (n = 7) Don't Know 1 Old Lighting Defective 1 Other 2 Energy Savings Dealer Survey Analysis HES Marketing and Program Awareness Dealers were asked at the onset of each interview how they had heard about the HES Program. All 30 dealers reported that they knew about the program and indicated how they initially heard about the program. 38% of the 30 dealers heard about the program from HES staff who delivered marketing materials to their store. The remaining dealers heard about HES primarily through word of mouth, the utility, or through other dealers. Figure 39 below summarizes these findings. Figure 39. HES Program Knowledge Dealers (n = 29) Prior Knowledge 3% Other Retailer Utility 17% 17% Word of Mouth 24% HES Staff 38% 0% 5% 10% 15% 20% 25% 30% 35% 40% Just over 64% of dealers also reported that they received program and marketing materials from Rocky Mountain Power or PECI staff this past year. Dealers were asked to rate the information provided by Rocky Mountain Power on a scale of 1 to 10; with 1 meaning that there was too The Cadmus Group, Inc. / Energy Services 45

48 little information and 10 meaning that there was too much information. Figure 40 shows that 30% of dealers rated the marketing material as a 6; indicating that in general the material was appropriate. One fifth of dealers rated the material as 7 and another fifth rated the material as either an 8 or a 10; indicating that the material was perhaps too cumbersome or difficult to understand. When weighted by frequencies of dealer responses, the average rating is Feedback about the material was generally good as well. More than half of the 17 dealers who responded to this question indicated that overall the materials were very informative and that nothing was lacking. Four dealers suggested that they would like to see more information regarding rules, qualifying models, and updated information. Two dealers requested that Rocky Mountain Power supply them with more incentive applications each year to accommodate growing demand for energy-efficient products. Figure 40. Marketing Material Dealer Ratings (n = 20) Rating of 10 5% Rating of 8 15% Rating of 7 20% Rating of 6 30% Rating of 5 20% Rating of 4 Rating of 3 5% 5% 0% 5% 10% 15% 20% 25% 30% 35% 30% of dealers reported that the product clings (i.e. product stickers) used to advertise the program incentive were very useful. The product clings make it easier for dealers to sell energyefficient appliances because the efficiency ratings and incentive information are easy for customers to see. 20% discussed other marketing materials such as pamphlet and website information as the most useful. Another fifth cited incentive applications as the most useful marketing material. Figure 41 below summarizes these findings. Additionally, dealers suggested that Rocky Mountain Power deliver more product clings and incentive applications each year. The Cadmus Group, Inc. / Energy Services 46

49 Figure 41. Most Useful Marketing Material Dealer Point of View (n = 20) FAQs 5% Posters 10% List of Qualified Products 15% Applications Other 20% 20% Product Clings 30% 0% 5% 10% 15% 20% 25% 30% 35% Overall, dealers use various methods to inform their customers of the HES incentive program. Of the 30 dealers, 53% inform customers about the HES Program during the initial sales presentation. Another 20% inform customers by directing them to qualifying appliances with product clings. Four dealers (13%) indicated that they do not inform customers about the program at all. Figure 42 summarizes these findings. Additionally, about half of the 30 dealers reported that over the last four years these tactics have increased while the other half reported that these promotion tactics have not changed at all during the last four years. Figure 42. Program Promotion Methods by Dealers (n = 30) Applications 3% Posters/Print Ads Do Not Promote 10% 13% Product Clings 20% Sales Presentation 53% 0% 10% 20% 30% 40% 50% 60% Dealers were also asked which program marketing materials seem to make the greatest impression on customers. 23% cited the combination of product clings and a sales person as the most effective. Another 20% reported that customers typically respond best when a sales person actively discusses the program. 17% reported that customers are most likely to respond to the product clings. Other marketing reported by dealers consisted of in-store advertising such as posters and word of mouth among customers. One dealer reported that he does not promote the program and one other dealer reported that he did not receive marketing materials (each accounts The Cadmus Group, Inc. / Energy Services 47

50 for 3% of dealers). Both of these dealers are contractors and generally would not promote the program or receive marketing materials anyway. Figure 43 summarizes these findings. Figure 43. Program Marketing Materials That Customers are Most Likely to Respond To (n = 30) Did Not Receive Materials Do Not Promote Flyers Don't Know Other Marketing Product Clings Sales Person Product Clings, Sales Person 3% 3% 10% 10% 13% 17% 20% 23% 0% 5% 10% 15% 20% 25% In addition to the program marketing materials and signage, all 30 dealers reported that they promote energy efficient options for customers at least some of the time. Of the 30 dealers, 77% indicated that they always promote these options. 10% reported that they often promote energyefficient options, and another 13% reported that they sometimes promote energy-efficient options. Figure 44 presents these findings. Figure 44. Dealer Promotion of Energy-Efficient Options by Frequency (n = 30) Often, 10% Sometimes, 13% Always, 77% The Cadmus Group, Inc. / Energy Services 48

51 Dealers were also asked to discuss key selling points 24 for energy-efficient appliances. 40% of dealers reported that customers are mostly interested in the potential cost savings associated with installing energy-efficient appliances (i.e. lower monthly energy bills). 23% cited the program incentive as the key selling point for energy-efficient measures. Another 23% also cited potential energy savings as a key selling point and two participants (7%) believe that product features are the key attractor for energy-efficient appliances. Figure 45 presents these findings below. Figure 45. Key Selling Points for Energy-Efficient Measures (n = 30) Other Features 7% 7% Energy Savings Program Incentive 23% 23% Cost Savings 40% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Additionally, 26 dealers were confident that on the whole their customers understand the energyrelated benefits of using high efficiency appliances. Among the four who did not think that their customers understand energy-related benefits, some cited demographics as a possible cause. These dealers reported that they believed that people from areas with less affluence tend to be less concerned with energy-related benefits and more concerned with their personal budgets. Survey administrators also asked dealers to rate their typical customer on a scale of 1 to 10 to determine how interested a customer is in switching to an energy-efficient appliance. Responses were most heavily weighted in the rating category of 8; indicating that overall dealers perceive the average customer as interested in purchasing an energy-efficient appliance. When weighted by dealer frequencies, the average rating is Figure 46 summarizes these ratings. 24 For the 12 dealers who cited potential cost savings as a key selling point, all of them also listed a secondary selling point such as program incentive, energy savings, and product features. The Cadmus Group, Inc. / Energy Services 49

52 Figure 46: Customer Ratings by Dealers Interest in High Efficiency Appliances (n = 28) Rating of 10 Rating of 9 Rating of 8 Rating of 7 Rating of 6 Rating of 5 Rating of 3 7% 11% 7% 11% 4% 21% 39% 0% 10% 20% 30% 40% 50% Dealers were also asked to list energy-efficient appliances that their customers are most and least aware of. According to more than half of the dealers surveyed, customers are keenly aware of incentives available for clothes washers; particularly the incentives associated with front-loading clothes washers. Customers were not as aware of the incentives for refrigerators or deep freezers. A number of dealers remarked that the $20 incentive for purchasing an energy-efficient refrigerator does not seem to incentivize people as much as the greater incentive for highefficiency clothes washers. Pricing Dealers reported different pricing methods for energy-efficient appliances. Standard-mark-up was most frequently reported (38%), as shown in Figure 47 below. About one-fourth of dealers indicated that their pricing policies reflect manufacturer recommendations for energy-efficient appliances. The remaining dealers determined prices based on customer willingness to pay, other dealer prices, or subcontractor installation fees. Figure 47. Dealer Pricing Methods (n = 29) Subcontractor Fees 3% Other Dealer Prices Willingness to Pay 10% 10% Don't Know 14% Manufacturer Recommendation 24% Standard Markup 38% 0% 5% 10% 15% 20% 25% 30% 35% 40% When discussing prices or price differences with customers, 41% of dealers reported that they discuss product life-cycle costs, the value of potential energy savings associated with the measure, costs of regular maintenance and repairs, and the discount from the program incentive. About one-third reported that they mostly discuss the upfront cost to the customer minus the The Cadmus Group, Inc. / Energy Services 50

53 program incentive if applicable. Another 28% incorporate an estimate of potential energy savings as well as the program incentive if applicable. Figure 48 below summarizes these findings. Figure 48. Pricing Methods Used to Sell High Efficiency Appliances (n = 29) Comparison of upfront equipment costs and estimate of energy savings (program incentive) 28% Comparison of upfront equipment cost (including program incentive) 31% Comparison of lifecycle cost including value of energy savings (including program incentive) 41% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Dealers did not believe that their participation in the program affected the way in which they present price differences to customers. About half of dealers (53%) said that the program and marketing materials did not have an impact while 40% reported that the program did have an impact on how they present prices to customers. Dealers also explained that for refrigerators and dishwashers, there are very few non-efficient options on the market so they typically sell highefficiency appliances anyway. Program Perception Among Dealers and Suggestions to Improve HES Nearly all of the Idaho HES participating dealers offered positive feedback about the program. Only five dealers were not happy with the program; three of which were contractors. The following dealer quotes are most representative of the 24 positive reviews 25 : It is a good program; beneficial to the customer and helps reduce their electricity/gas bill. Good program - great tool for us on the floor to sell products. The program can help us be perceived as a green company. Any incentive that can get the general public to upgrade and save energy is good. I want to know more about it - think it's a good idea, but don't know much about the program. The following dealer quotes represent some of the five negative responses about the HES program 26 : These first three positive comments came from interviews with dealers/retailers and the fourth comment came from an interview with a participating contractor. The first two negative comments came from interviews with participating contractors and the third negative comment came from a dealer interview. The Cadmus Group, Inc. / Energy Services 51

54 Would be nice to get more information so I can offer it to customers. Program is under-utilized, needs to be promoted much more than it is. I think more people would be interested in bringing their homes up to current energy standards. They are really picky about paperwork. They kick applications for a missing date - a little too picky. Don't put sticker on wrong model, and if so, don't make retail store pay the rebate the customer thought they were getting. Dealers also offered suggestions on how Rocky Mountain Power can improve the program and therefore increase market penetration of high efficiency appliances. Some dealers suggested that the standards for efficiency change too often and make it difficult to stay current with the program. This dealer also reported that constantly having to refer to an information sheet gives customers the impression that staff are not knowledgeable about the program. Another dealer suggested that Rocky Mountain Power should increase marketing efforts for smaller towns and areas outside of major metropolitan areas. Lastly, one dealer suggested that Rocky Mountain Power should provide information to customers that explain why the utility is paying them to purchase an energy-efficient appliance. This dealer also indicated that this might allay suspicions among customers. Clothes Washer Recycling Through HES Program Among 19 dealers who reported selling clothes washers, 13 of them recycle old clothes washers when customers purchases high-efficiency models. However, only two dealers recycle clothes washers through the HES program 27. Furthermore, more than three-fourths of dealers were not aware that such a Program exists. Figure 49 and Figure 50 summarize these findings. Figure 49. Dealers who Recycle Clothes Washers Through HES Program (n = 13) Don't Know, 2 Yes, 2 No, 9 27 The data PECI provided for the HES program in did not indicate participation in the clothes washer recycling program. It is assumed that these dealers are more recent participants in the program. The Cadmus Group, Inc. / Energy Services 52

55 Figure 50. Dealers Familiar with Program Requirements (n = 13) Yes, 3 No, 10 All 13 dealers who recycle clothes washers provided feedback regarding the type of clothes washer typically purchased by customers who want to get rid of their old one. Just under onefourth of dealers reported that customers typically purchase either a high efficiency clothes washer (MEF > 2.0) or a lower efficiency unit (MEF between 1.72 and 1.99). Other clothes washers, as reported by three dealers, were generally described as energy-efficient models or front-loading models. See Figure 51 below. Figure 51: Clothes Washers Purchased After Recycling Old Unit (n = 13) Energy Star MEF > 2.0, Energy Star 1.72 < MEF < Other Energy Star 1.72 < MEF < 1.99 Energy Star MEF > 2.0 Don't Know % 5% 10% 15% 20% 25% More than three-fourths of dealers reported that they both deliver new clothes washers to customers and haul the old one away to a recycler. One reported using a contractor for delivery and hauling. Figure 52 summarizes these findings. Dealers were also asked if they test old units for operability prior to taking it to a recycling center. Four dealers test old units for operability, three do not test for operability, and six dealers did not know. Among the three who do not test for operability, one reported that the testing is too time-consuming, most of them do not work, and they do not plan to resell them anyway. Table 16 summarizes these findings. Table 16 also presents results regarding serial number collection, frequency of clothes washer recycling, and receipt collection. Six dealers record model and serial number while four do not. Additionally, only two dealers reported turning in receipts to the utility. Six dealers reported that The Cadmus Group, Inc. / Energy Services 53

56 they always recycle while four reported that they do not always recycle. Survey administrators probed further and learned that one dealer scraps old units and sells functioning parts. Figure 52. New Clothes Washer Delivery Methods (n = 13) Don't Know, 2 Dealer Uses Subcontractor, 1 Dealer Delivers, 10 Table 16. Clothes Washer Recycling Questions (n = 13) Dealer Yes No Don t Know Total Tests for Operability Collects Model/Serial # Always Recycles CW Turns In Receipts Utility Staff and Implementer Interviews Cadmus staff interviewed multiple parties to gain an in-depth understanding of the Rocky Mountain Power HES program. The interviews included key utility staff members as well as implementation staff directly involved in program operations. As HES is implemented by a third party, utility staff interviews primarily focused on high-level management and regulatory issues, while program-level details generally were addressed in interviews with implementers. Interview results reflected this divided focus. Program Management Program Design The HES Program spans five states. As the areas served have distinctive demographic and geographic characteristics, implementers have tailored program offerings as much as possible to unique constituent characteristics, as tariffs allow: We try to balance customization to the five service areas with standardization that leverages the resources available Implementer Interview The Cadmus Group, Inc. / Energy Services 54

57 The programs initial design was intended to create a fuel blind program, offering energyefficiency measures to all customers regardless of the utility s service. Implementers explained: We put every measure we could think of in the initial mix and then worked through all the savings analyses to determined the amount of deemed savings for each measure. 29 Analysis of program cost-effectiveness provided estimated savings for each measure in each state, reflecting the territories climate. Seeking to make each measure independently costeffective, program designers also included measures such as insulation and ENERGY STAR windows in natural gas-using areas to provide comprehensive program offerings. For the initial program design, implementers were required to manage the program so at least half of the overall program savings could be attributable to non-lighting measures. This was partly due to the relative ease of obtaining lighting savings, compared to other measures and the desire to make the program have a direct impact that is more visible to customers. By contrast, lighting s upstream incentive approach often made the program invisible to customers. Program staff have since realized benefits of a bundled program approach including, to some extent, avoiding duplication in management, marketing, and program delivery. The bundled approach also offers several advantages: consistent design for the utility Web site and in promotional materials; economies of scale; an ability to leverage synergies in product offerings; and an ability to analyze the different markets and test new approaches with market actors. Reflecting on how the program s structure compares with those of similar programs, one implementer said: This program is the most complex I ve ever dealt with. There are savings and efficiencies by shared marketing, but we ve learned we cannot apply the same tactics in every market. 30 As a result, implementers provided input to program design to reflect changes in the market place. Key program design findings include: The HES Program was designed to be a comprehensive, one stop shop for home efficiency measures. Measures included in the program had to be individually cost-effective. The combination of multiple measures and different state regulatory environments resulted in a highly complex program to manage. Program Strengths The multiple measure approach made HES Program participation more convenient for customers as the program offered a single application for incentives and access qualified resources for measure installation and sales Ibid Ibid The Cadmus Group, Inc. / Energy Services 55

58 Program Challenges Despite the advantage offered by some economies of scale, providing a high service level for smaller markets represented a significant challenge. Rocky Mountain Power staff remarked on the difficulty of tuning into each market: Small market point-of-sales are less sophisticated and there are fewer dealers/retailers. They need support and training in the same way that large retailers do, but their lower sales volume represents higher administrative costs. This makes it tough to operate within the required cost-benefit ratio in certain markets. Adjusting the program to meet rapid marketplace changes proved another challenge for the program, which could not be modified until tariffs changed. One implementer characterized this challenge as: Trying to pull free market levers in an environment of regulatory controls. A third challenged presented itself with through companion incentives from various sources, including federal stimulus funding, federal income tax credits, manufacturers, and other utilities serving the same customers or in adjacent territories. In particular this was evident in the misalignment of pricing of bulbs offered through this program and those offered through the Bonneville Power Administration (BPA) and Northwest Energy Efficiency Alliance (NEEA) programs. During the program cycles, incentives for lighting measures were offered by two regional programs, in overlapping service territories. While the three programs (BPA, NEEA and Rocky Mountain Power) were effectively coordinated for most of this time, each underwent restructuring at different times in The result was that the programs were out of sync in 2008 and participation in Rocky Mountain Power program dropped off significantly, negatively impacting the program cost-effectiveness. Changes to Program and Goals CFLs have been the most cost-effective measure, and contributed the largest component of gross energy savings for the program. Lighting measure incentives were delivered using an upstream approach, which provided incentives to CFL manufacturers to lower costs to retailers. In recent years, the program expanded from incentives on basic twister models to a variety of specialty light bulbs. However as Rocky Mountain Power indicated in their 2008 Annual Report: The Home Energy Savings program under-performed in 2008 due to a reduction in lighting savings and increased costs associated with weatherization measures. Retail locations and delivery interactions associated with regional lighting offerings were all contributing factors to these results. In 2006 and 2007 the Home Energy Savings program benefited from its alignment with and savings attributions from the regional lighting program. In 2008 Rocky Mountain Power intended to continue its support of the regional effort when the company made its changes to the 2008 Home Energy Savings program's lighting measures. However due to timing differences of when the lighting initiative redesigns occurred, the opposite occurred, the two programs ended up not in alignment on the pricing of specialty bulbs. This lack of alignment The Cadmus Group, Inc. / Energy Services 56

59 presented two options to the company; 1) the Home Energy Savings offer could be changed again in 2008 to correct the situation or 2) Rocky Mountain Power, though PECI, could seek other paths to ensure the program's lighting savings were secured and wait until the next program change before aligning again with the regional program. While the options were not mutually exclusive, the company elected to not change the program twice in 2008 and instead to pursue a small market strategy of enrolling smaller retailers in the market, those more strategically located in the company's rural service areas in Idaho. Many of these smaller retailer chains indicated an interest at the corporate level; however delays in generating interest at the individual store level resulted in a delay in the discounted lighting options becoming available on store shelves. As a result, lower than anticipated CFL bulb sales contributed to reduced energy savings through the Home Energy Savings program in This misalignment as well as the positive response from smaller retailers to the program helped shape changes to the program moving forward. Insulation participant numbers increased in The increase was based on a significant reduction in costs driven by an increase in insulation contractors and a reduction in raw material costs. With contractors increasing their marketing activities to customers and utility incentives, insulation participation increased beyond the 2008 estimates. Because insulation was intended to be offered as part of a cost-effective package with other measures and does not pass cost effectiveness standards on a standalone measure basis, adjustments to the insulation program were also in order. Program changes included: Regional program realignment; Enrollment of small retailers in the CFL program; Addition of efficient, specialty light bulbs, in addition to standard twister CFL; Specialized dealer incentives for duct sealing; and, Realignment between costs and incentives as well the addition of a per house cap to the Insulation program. Marketing PECI has developed marketing plans, materials and proposed marketing budgets for each state, with review and approval by Rocky Mountain Power. Estimated conversion targets for each marketing tactic have driven priorities for channel and media weight. Budgets were developed, using the previous year as a guide, and goals, expenses, and projected savings were adjusted. Marketing efforts primarily focused on key customer contact channels through retailers, trade allies, and the company web site. Investment in mass media channels, such as television, radio, and print advertising, was comparatively less than more targeted approaches. Marketing promotions drove prospective participants to the program s Web page for information and applications. PECI also provided creative design for marketing materials, and worked with the Rocky Mountain Power marketing team to incorporate edits and move to scheduling. The Cadmus Group, Inc. / Energy Services 57

60 Implementers reported evaluating and building on the market approaches that proved the most effective. Recently, implementers have started exploring a marketing segmentation approach, targeting messages to specific customer characteristics. An online web survey has also been introduced recently. Data from efforts will be evaluated to determine the best use of available marketing dollars. In summary: Awareness Marketing strategies have focused on retailers, trade allies, and the web site; PECI has started using a market segmentation approach to target specific, higherpotential customers. While point-of-sale contacts with retailers and trade allies were considered the primary mechanism for program awareness, the call center experienced increased inquiries about the program immediately following a bill insert promotion. Training PECI employed two staff as designated trainers for the program. They focused on HVAC contractors, providing training and updates on how to conduct AC tune-ups, duct sealing, and proper procedures for insulation. A primary training issue has been identifying differences between conditioned and unconditioned space, as this is a critical criterion for application approval. Rocky Mountain Power staff has remained current regarding changes in program components by attending measurement-related conferences, staying up to date with code and standard changes, studying current and upcoming topics (such as smart grid technology); and sitting on national panels. Communication Interviews with implementers indicated PECI valued communication as an important customer service component. They have tailored communication to the needs of their stakeholders, which include: Rocky Mountain Power executives, regulatory stakeholders, contractors, dealers, and customers. For external stakeholders, such as customers, implementers have posted program updates to the web site, provided updates to the direct services call center, and issued public notices. PECI has called contractors directly when tariff changes have taken effect. Internal Reporting Implementers screen program applications for completeness prior to entering applications into a database. The database is then used to generate a report for Rocky Mountain Power and trigger incentive payments. Rocky Mountain Power reviews applications, and then sent them back to the implementation project manager. PECI also fields occasional ad hoc data requests, and provide custom reporting to program staff. The Cadmus Group, Inc. / Energy Services 58

61 Financial reports were generated weekly, with PECI pulling information from the database and aligning it with invoice reporting to Rocky Mountain Power. On the 28 th of each month, a participation update report is generated for each state; this report included every measure; monthly and year-to-date progress, and goal projections. Year-end reports were generated to review program results. Quality Control Quality controls have been established at a number of points in program delivery. Both PECI and Rocky Mountain Power staff reviewed applications. Data were reconciled from the database and incentive checks are generated. Implementers searched for anomalies in participation reporting, seeking to identify issues before they could impact cash flow or cost-effective delivery or customer service levels. PECI also conducted inspections on a minimum of 5% of HVAC, insulation, and window installations. To eliminate any bias, inspectors were not provided with incentive information. Rocky Mountain Power staff inspected a number of applications to verify square footage, insulation levels, window types, and other factors. If anomalies occurred in data quality or contractor reports, they increased numbers of inspections conducted with those particular vendors. Based on site visits that we conducted for the Utah HES program 31 we determined that the PECI s inspections had not yielded higher realization rates. PECI stated that they have increased staffing levels to do more inspections. Further, they have recently worked to improve tools used in inspections, such as using laser measures for square footage. Rocky Mountain Power staff described proactive efforts to address quality: Continuous improvement is driven by promptly addressing the problems but also by trying to avoid potential problems. In summary: Quality Control inspections have been established for 5% of HVAC, insulation, and window installations. PECI increased staff to complete more inspections, and instituted more sophisticated equipment for greater precision in inspections, such as laser measures for square footage. Market Barriers HES program participation barriers varied by measure type. The following paragraphs outline known barriers to the primary measure types. Lighting The rural nature of the territory and other regional programs offered by Bonneville Power Administration (BPA) and Northwest Energy Efficiency Alliance (NEEA) in the same territory presented barriers for the CFL program. In particular this was evident in the misalignment of pricing of bulbs offered through this program and those offered through the Bonneville Power 31 Site visits were only conducted in Utah due to the very high participation in the insulation measure in that state. The Cadmus Group, Inc. / Energy Services 59

62 Administration (BPA) and Northwest Energy Efficiency Alliance (NEEA) programs. While the three programs (BPA, NEEA and Rocky Mountain Power) were effectively coordinated for most of this time, each underwent restructuring at different times in The result was that the programs were out of sync in 2008 and participation in Rocky Mountain Power program dropped off significantly. Additional lighting market barriers included: concerns about mercury in bulbs; inconvenient recycling options; compromises to bulb lives when placed in a dimmable socket; the tone or color of light produced; and delayed start of bulbs. The program addressed these barriers by using information about newer CFL bulb features and specialty bulb availability. The program Web site also provided information on recycling options for expired CFL bulbs. Insulation Insulation market barriers included: the process required for qualifying a home for the insulation incentive; homeowners processes for finding qualified installers and confirming the jobs-met program requirements; and upfront costs. To address these barriers, implementers worked closely with builders and contractors to try to transform the market, working to inform builders and homeowners of the need for and value of higher insulation levels. Appliance Barriers to high-efficiency appliances largely involved prices and higher incremental costs. Awareness for the ENERGY STAR label seemed high, as many manufacturers promoted ENERGY STAR products independently. For dishwashers, where 95% qualified as ENERGY STAR, energy savings on a per-unit basis became marginal, making the measure less costeffective as a program offering. As most appliances were bought and sold at retail outlets, implementers worked to inform sales staff of benefits from efficient equipment and reduce market barrier effects. Another approach was continuing to advocate higher-efficiency standards; so, as a base offering, ENERGY STARlabeled products provided greater energy savings than non-energy STAR products. HVAC Market barriers for high-efficiency HVAC equipment included: awareness; high incremental costs; and, in some climates, payback periods and varying equipment efficiencies. HVAC equipment was generally installed as needed, replacing nonfunctioning units. Implementers noted: It s not typically something for which many homeowners shop around. Opportunities for higher-efficiency equipment could be missed if plumbers or HVAC technicians were unaware of incentives available or, in some cases, simply chose to move inventory readily available. The Cadmus Group, Inc. / Energy Services 60

63 Implementers provided HVAC contractors with information required to help sell higherefficiency equipment. The program s involvement ensured units were installed to proper standards and addressed subtle issues related to climate zones. Future Program Issues In 2012, federal standards will require all light bulbs have higher efficiency ratings than current standard incandescent bulbs. CFLs thus may essentially become the new standard, which will significantly affect all lighting programs. While program staff expects continued savings from CFLs after new regulations come into effect, they anticipate types of qualifying CFL bulbs will change. Looking ahead to 2012, program staff indicated: There are still a lot of unknowns for lighting. We think there will still be a lot of product out there in 2012, and we don t believe all incandescent bulbs will be completely phased out. Implementers said their strategy is still developing, but they are looking at newer lighting technologies, such as LEDs, which may provide interim solutions. PECI also believes a smart grid s impact on residential customers presents another unknown. Currently, many utilities are exploring or proceeding with implementing smart grid technologies in their service territories, but the extent customers will become interested in their energy consumption remains unclear. Program participation could increase as customers become more aware of their energy usage; alternatively, competing products or technologies may become available, supplanting current program offerings. Consumer electronics may also offer a potential opportunity for energy savings that could be explored through HES program offerings. Currently, televisions, computers, and monitors are the only electronic equipment qualifying for ENERGY STAR labels. As home electronics manufacturers continue to adopt energy-efficiency technologies (e.g., DVRs, game consoles, and sound components), further opportunities may emerge for energy savings. Program Development In 2010, PECI plans to implement the program more proactively. These efforts will include: tightening the quality control process; focusing on the trade ally network; and continuing to monitor trends in data collected and evaluate associated market implications. PECI is working to streamline the manufacturer contracting process; so changes can be made within a 30 to 60 day timeframe. Future program development will be influenced by AARA-funded companion incentives. Program staff and implementers will assess effects of these incentives, especially focusing on impacts on clothes washers and electric water heaters. Rocky Mountain Power views this program as a vehicle for incenting additional energy-saving measures for homes, as technologies evolve. The Cadmus Group, Inc. / Energy Services 61

64

65 5. Program Results Impact Findings The following five subsections present the impact evaluation s findings: 1. Summary of Program Participation 2. Review of Terminology 3. Determination of Average Annual Gross Energy Consumption 4. Determination of Gross Savings 5. Determination of Net Savings Summary of Program Participation Starting May 2006 and continuing throughout 2007 and 2008, Rocky Mountain Power offered the HES program to its Idaho customers. By the end of this period, approximately 6,000 customers had participated in the program. Annual program participation rose from 1,135 at the program s inception to 2,693 in 2007 and dropped to 2,175 participants in During the two full program years (2007 and 2008), the program participation was reduced by 19%, predominantly, due to CFL participant numbers dropping off at this time. Table 17 shows annual program volumes to better illustrate participation trends. The Cadmus Group, Inc. / Energy Services 63

66 Table 17. Annual Program Participation Measure All Years Ceiling Fans Central A/C Equipment CFL 918 1, ,996 Clothes Washer ,126 Clothes Washer-Tier One Clothes Washer-Tier Two Dishwasher Electric Water Heater Evaporative Cooler 1 1 Fixture Heat Pump Upgrade 1 1 Insulation: Attic Insulation: Floor Insulation: Wall Proper CAC Install Proper CAC Sizing Refrigerator Windows Total 1,135 2,693 2,175 6,003 Several factors impact the varying participation: Increased awareness: The general population has become more aware of ENERGY STAR products and their benefits. Advertising: Rocky Mountain Power increased advertising during this timeframe, and more retailers and contractors became involved with the program. Other programs: Similar local and national programs were running at the same time. Economic factors: The economic downturn may have been a factor, especially for potential participants planning on upgrading their home s efficiency. In the context of a challenging home mortgage market where the ability to upgrade by moving to a new home became more limited, incentives and education around a product s lifecycle costs could have convinced participants to take action in their current homes. Retailers and contractors also turned to existing housing stock improvements rather than new home construction The participant count only included counts of individual incentives, For appliance measures, customers were limited to one incentive per household, excepting ceiling fans and fixtures; for home improvement, participant numbers of equaled numbers of total jobs; for HVAC, contractor incentives were not counted as separate participants; and, for CFLs, participants were estimated at 10 bulbs per participant. The 10-bulb rate per participant was based on information from PECI and the PAC-E Annual Report of Idaho Demand Side Management Activities. The clothes washer category was modified to reflect new high efficiency standards in 2008: Clothes Washers: 5/31/2006 4/31/2008 (MEF 1.42 in 2006 and in ); Tier 1: 4/1/ current (MEF ); Tier 2: 4/1/2008 current (MEF 2.0+) The Cadmus Group, Inc. / Energy Services 64

67 Billing Analysis Results 34 After screening and matching account attrition, the final analysis group consisted of 258 participants and 1,032 nonparticipants. 35 The final CSA regression model specification below was used to estimate savings from insulation measures: ADC it = + 1CDD it + 2 HDD it + 3 POST t + 4 PARTPOST it + 5 ANNUALPRE i + it Where for customer (i) and month (t): ADC it = average daily kwh consumption HDD it = average daily heating degree-days (base 65) CDD it = average daily cooling degree-days (base 65) POST t = indicator variable that is 1 in the 2009 post period for both participant and nonparticipants, 0 otherwise. PARTPOST it = indicator variable that is 1 in the 2009 post period for participants, 0 otherwise. ANNUALPRE i = the total annual 2006 pre-period kwh usage. The key coefficient determining average insulation savings was 4. This value averaged daily insulation savings per program participant, after accounting for nonparticipant trends. The inclusion of the ANNUALPRE variable was used to create a fixed effects model specification, ensuring no participants or nonparticipants had an undue influence over the final savings estimate; this resulted in a more robust model As described in the Billing Analysis section, the Idaho billing analysis did not yield robust enough savings estimates to accurately assess actual energy savings. It was found that the Pacific Power Washington HES program expected savings and participant climate was a close proxy for the Idaho program. The sample in Idaho consisted of 93 participants and 372 control customers which proved insufficient for the billing analysis. The Cadmus Group, Inc. / Energy Services 65

68 The above pooled model combined nonparticipants (the baseline) and combined participants for the wall, floor, and attic insulation program. Table 18 presents billing analysis results for wall, floor, and attic insulation in Washington 36 as a proxy for Idaho. The billing analysis realization rate for insulation measures was 88%. Presented in Table 18 are savings for insulation measures for smaller-usage homes and higher-usage homes. Smaller homes realized only 14% of the expected savings. Larger homes saved 155% of expected savings. Table 18. HES Attic, Floor, and Wall Insulation Realization Rate Home Type Realization Rate (90% Confidence bounds) Model Savings (Overall) 87.6% (34% - 117%) Model Savings (Low to Medium Usage Homes, under 15,497 kwh) 13.5% (-11% - 38%) Model Savings (Medium to High Usage Homes over 15,497 kwh) 155% (106% - 205%) Model Savings (Average Quartile) 84.3% (47% - 121%) CFL Leakage Analysis Cadmus evaluated leakage of program-subsidized CFLs out of the Rocky Mountain Power service territory. This analysis covered Rocky Mountain Power s Idaho territory during 2006 through While leakage was studied, and is presented in this report, the findings only attempted to measure the number of bulbs for which Rocky Mountain Power provided incentives, which ultimately left its service territory. The study did not measure the number of bulbs bought within the service territory which were covered by incentives from other programs such as BPA and NEEA. The effects of this reverse leakage are likely on par with the effects of the leakage reported herein. Consequently this report assumes that the net leakage is zero for purposes of determining NTG and cost-effectiveness, and additional study is recommended to measure these other impacts as well as to confirm the findings from this initial leakage study. Evaluation results indicated 33.7% of total program CFLs subsidized through the program leaked outside the territory. The overall findings for Idaho, Utah and Washington combined came to 2.76%. The self reported data collected from retailers turned out to be unreliable, due to retailers confusing which bulbs were offered through which programs and a lack of incentive to perform a more thorough lookup. Upon verifying self reported data collected from the residential consumers it was found that on follow-up surveys, consumers provided varying result for the number of bulbs and location purchased. The resulting study sensitivity of the leakage in Idaho was ± 22.8%. Survey Overview Cadmus conducted two surveys. The first targeted households not served by Rocky Mountain Power (noncustomers) within 20 miles of Rocky Mountain Power s service territory. The second targeted retailers participating in the Rocky Mountain Power program. 36 Due to too low participation in Idaho to perform an adequate billing analysis, the billing analysis was used from the Pacific Power HES Washington territory. More detail of the billing analysis is provided in the Billing Analysis section of the report. The Cadmus Group, Inc. / Energy Services 66

69 The retailer survey s purpose was to obtain information on the number of CFLs stores sold since the beginning of 2006, and the percentage of those subsidized by the program. Surveys were completed with all of the 11 stores reported to have sold CFLs subsidized through the program. The household survey was completed with 141 randomly selected households in areas surrounding Rocky Mountain Power s service territory. The surveys asked respondents the number of CFLs purchased by their household since the beginning of 2006, stores from which they purchased those CFLs, and the percentage of purchases from each of those stores. In each region surveyed, the goal was to complete surveys with 70 non-rocky Mountain Power customers to achieve a 90% confidence at a +/- 10% level of precision. Household responses were combined with responses from retailer surveys to determine the number of CFLs households outside the service territory purchased since 2006 and percentage of those which were program bulbs. Calculations, Analysis Process, and Results Table 19 presents the number of program-subsidized CFLs purchased per household, the number of households, the percentage of households that were Rocky Mountain Power customers and the total number of CFLs calculated to have leaked out of Rocky Mountain Power s service territory in each of the two regions surveyed. Table 19. Regional Data Region Sum of Program Bulbs Average Program Bulbs/HH # of HH % of HH that are not PC Customers # of Leaked Program Bulbs n Pocatello Area ,382 93% 13, Idaho Falls Area ,741 23% 3, Sum/Average ,123 63% 17, Table 20 presents primary leakage calculation components used to determine the percentage of program bulbs leaked outside the territory. Table 20. Total Program Bulbs and Leakage Percent Leakage Calculation Component Value Weighted Average Bulbs that leaked by Households 0.47 Total Number of Households (within 20 miles) 66,123 % of Households not in PC Territory 63% Total # of Leaked Program Bulbs 17,732 Total Program Bulbs ( ) 52,621 % of Program leaked outside the territory 33.70% Calculations The following four primary calculations were used in estimating the CFL leakage: 1. Calculate the number of program-financed CFL bulbs purchased per respondent: The Cadmus Group, Inc. / Energy Services 67

70 # of program CFL bulbs purchased by respondent (household) = # of CFLs Purchased X % purchased from participating retailer X Retailer reported % of CFLs sold that were program CFLs 2. Calculate the number of program-financed bulbs purchased per person in each of the surveyed regions: # of program CFLs per household in each region = Average of program CFLs purchased by respondent households in region X # of households in region X % of the region that are not Rocky Mountain Power customers 3. Calculate the number of bulbs leaking out of the service territory in total: # of program CFLs leaking out of territory = Weighted average number of program CFLs per household across all regions (weighted by # of households) X Total number of households in all zip codes within 20 miles of Rocky Mountain Power s service territory X % of households that are not Rocky Mountain Power customers in each zip code 4. Calculate the percent of the program-financed bulbs leaking out of the service territory: % of program CFLs that are leaking out of Rocky Mountain Power s territory = # of program CFLs leaking out of territory # of CFLs that were financed by Rocky Mountain Power Assumptions The evaluation included the following assumptions: Surveyed non-rocky Mountain Power households were demographically and culturally similar to those not surveyed in the same region. The 11 store/retailer responses represented a census and exceeded the confidence and precision goal for likely responses to questions asked (such as the percentage of participating bulbs sold). The percentage of individuals called that were or were not Rocky Mountain Power customers was representative of the breakdown of customers (Rocky Mountain Power vs. other utilities) in the rest of the region. Households surveyed would be able to recall, with a fair degree of certainty; stores from which they purchased CFLs, the numbers of CFLs purchased; and the timeframe in which CFLs were purchased. The Cadmus Group, Inc. / Energy Services 68

71 CFL retailers would be able to accurately recall the number of CFL bulbs sold, and percentage that were part of the Rocky Mountain Power program. Though some respondents were not able to answer all questions, they were still able to provide information in response to a majority of the most critical cases. In these cases, assumptions were made as placeholders for missing data. Specifically: o For households where the interviewed individual was not able to provide the percent purchased from the store, a straight average was used based on the percentage purchased by all other households from all stores. This average was 62%. 37 o For retailers unwilling or unable to complete the interviews, a weighted average of other responses was used for estimating the proportion of total CFLs sold which were program-financed bulbs. The weighted average use was 58%. 38 For residential households, the vast majority of CFL purchases were assumed for residential use. Consequently, while some additional leakage may have occurred outside the residential sector, the nonresidential sector s impact on overall leakage was assumed to be minimal. Additional Details on Analysis Methodology Zip code territory definitions and populations within 20 miles of Rocky Mountain Power service territory were obtained from a GIS database of service territories maintained by Platts, a division of McGraw-Hill. In regions where Platts did not provide a percent of Rocky Mountain Power customer households, survey respondents for zip codes provided by Discovery were used to calculate the percentage of households that were Rocky Mountain Power customers. This assumed a geographically consistent distribution of population across the zip code. Household survey respondents were asked to identify stores from which they purchased CFLs. While matching these reports with participating stores was typically straightforward, some responses were less precise about specific names and locations for stores. In these instances, best efforts were used to determine whether a store being referenced was a participating or a nonparticipating retailer. Methods used to support this determination included Yellow Page listings of stores, Internet searches, and searches on corporate Web sites for store locations. In some rare instances, when multiple locations for a retailer (i.e., a retail chain) matched the location mentioned, it was assumed the respondent referred to the location closest to his or her household To increase the confidence level the average used came from a collective study conducted with the Utah, Idaho, and Washington Rocky Mountain Power territories. 10 regions were mapped out during the phone interview process to cover the three states studied within the Rocky Mountain Power territories. Interviews were conducted with 58 of 96 stores, and 696 randomly selected households in areas surrounding Rocky Mountain Power s service territory. The Cadmus Group, Inc. / Energy Services 69

72 NTG Estimation Cadmus, with its subcontractor Discovery Research, conducted telephone surveys with 429 customers who purchased qualifying measures through the HES Program and 141 non-rocky Mountain Power customers CFL purchasers. The findings were drawn from: a review of gross savings claims for the program during each year; an analysis of program participants freeridership; and an analysis of spillover savings generated by the program. Except for CFLs and insulation, the realization rate was measured through participant surveys asking about current HES measure usage. During the survey, participants were asked what they did with the specific program measure and where it is currently installed. The resulting responses were scored in a matrix to determine the realization rate and the resulting rates for each measure are found in Table 21-Table The insulation realization rate was determined through the billing analysis. Gross Savings Cadmus compared deemed savings values provided by Rocky Mountain Power s program implementation contractor and cost-effectiveness inputs provided by Rocky Mountain Power to determine program gross savings. Except for CFLs and insulation, the realization rate was measured through participant surveys asking about current HES measure usage. The CFLs realization rate is assumed to be one hundred percent 40. The insulation realization rate was determined through the billing analysis 41. The realization rates for each measure are shown in Table 21-Table 23 and the methodology is described in greater detail in the Error! Reference source not found. section. 39 The realization questions are E1, E11, and E18 found in Appendix F: All non-insulation Measures Participant Survey Instrument. Sample question and scoring matrix is found in Appendix N: Realization Rate Matrix. 40 While leakage was studied, and is presented in this report, the findings only attempted to measure the number of bulbs for which Rocky Mountain Power provided incentives, which ultimately left its service territory. The study did not take into account other factors such as the rural nature of the territory, lending itself to a very high reverse leakage effect or other regional programs offered by BPA and NEEA in the same territory which would cut down the number of bulbs sold through the Rocky Mountain Power CFL program. Consequently this 41 report assumes that the net leakage is zero. Due to too low participation in Idaho to perform an adequate billing analysis, the billing analysis was used from the Pacific Power HES Washington territory. More detail of the billing analysis is provided in the Billing Analysis section of the report. The Cadmus Group, Inc. / Energy Services 70

73 Table 21, Table 22, Table 23 and Table 24 present claimed savings and realization rates for each year. Table HES Program Gross Savings and Realization Rate Measure Number Installed Unit Energy Savings Gross Savings (kwh) Realization Rate Realized Gross Savings (kwh) CFL 9, , ,246 Clothes Washer , ,833 Electric Water Heater , ,416 Fixture Insulation: Attic (sq ft) 2, , ,221 Refrigerator , ,376 Total 12, , ,184 Table HES Program Gross Savings and Realization Rate Measure Number Installed Unit Energy Savings Gross Savings (kwh) Realization Rate Realized Gross Savings (kwh) Ceiling Fans Central A/C Equipment CFL 15, , ,555 Clothes Washer , ,110 Dishwasher , ,640 Electric Water Heater , ,885 Evaporative Cooler Fixture , ,588 Insulation: Attic (sq ft) 57, , ,633 Insulation: Floor (sq ft) 2, , ,535 Insulation: Wall (sq ft) 16, , ,660 Refrigerator , ,420 Windows (sq ft) 3, , ,570 Total 96, , ,225 Table HES Program Gross Savings and Realization Rate Measure Number Installed Unit Energy Savings Gross Savings (kwh) Realization Rate Realized Gross Savings (kwh) Ceiling Fans , ,747 Central A/C Equipment CFL 4, , ,165 Clothes Washer , ,856 Clothes Washer-Tier One , ,933 The Cadmus Group, Inc. / Energy Services 71

74 Clothes Washer-Tier Two , ,461 Dishwasher , ,897 Electric Water Heater , ,530 Fixture , ,520 Heat Pump Upgrade Insulation: Attic (sq ft) 180, , ,952 Insulation: Floor (sq ft) 18, , ,499 Insulation: Wall (sq ft) 18, , ,056 Proper CAC Install Proper CAC Sizing Refrigerator , ,991 Windows (sq ft) 11, , ,797 Total 235, , ,572 Table 24. HES Program Total Gross Savings and Realization Rate Program Year Gross Savings (kwh) Realized Gross Savings (kwh) Realization Rate , , % , , % , , % All Years 1,554,972 1,515, % After accounting for realization rates, verified savings identified through survey efforts were consistent with claimed gross savings from program years In addition, savings per measure, used in planning assumptions and in calculating gross savings claims, were compared against savings values for these same measures in the 2006 and 2008 DEER database as well as the RTF 5 th and 6 th Power Plans. This analysis determined savings per measure values used by Rocky Mountain Power were reasonable and consistent when compared to these sources. Freeridership Freeridership, or the percent of savings that would have occurred in the program s absence, was calculated through surveys with program participants. This self-report approach for calculating freeridership is an industry-standard methodology; however, it has some inherent limitations. One factor of particular relevance to the program is a customer self-report methodology is not entirely accurate for capturing market transformational impacts of multiyear programs. For example, a multiyear program may alter the availability of higher-efficiency products in a region through influencing stocking practices at retailers, increasing dealer experience and comfort with more efficient products, or even through impacting demand for efficient products. Customers, when choosing between various makes and models of a given product, may not be aware the selection of measures available has been influenced by a program. Therefore, while the customer may correctly state that, in the program s absence, they would still have chosen between two efficient products, the availability of those products may have been a result of the program. In The Cadmus Group, Inc. / Energy Services 72

75 this case, while the customer would count as a freerider, if the program had not been running, a less-efficient option may have been available to the customer (which he or she might have purchased). In calculating freeridership, Cadmus and Discovery Research surveyed 429 customers participating in 549 program measures over the program years. Participants were selected randomly from populations purchasing measures that contributed the most to program savings and/or accounting for the majority of program costs. The freeridership matrix is described in more detail in Appendix M: Freeridership Matrix. The Cadmus Group, Inc. / Energy Services 73

76 Table 25, Table 26, and Table 27 present freeridership analysis results for each program year, survey population, and confidence bounds based on relative precision calculated at the 90% confidence interval. The average freeridership value was weighted by gross kwh savings across the surveyed measures, and was applied to the total gross number to arrive at the net freeridership number for each year. Table 28 summarizes HES program total freeridership. The Cadmus Group, Inc. / Energy Services 74

77 Table HES Program Freeridership Measure Freeridership (90% Confidence bounds) n Clothes Washer 31% (25% - 37%) 38 Electric Water Heater 0% (0% - 0%) 1 Fixture 25% (0% - 0%) 1 Refrigerator 35% (26% - 44%) 18 Total 30.6% (26% - 36%) 58 Table HES Program Freeridership Measure Freeridership (90% Confidence bounds) n Clothes Washer 25% (20% - 30%) 71 Dishwasher 24% (16% - 32%) 25 Electric Water Heater 13% (0% - 33%) 4 Fixture 13% (0% - 0%) 1 Insulation: Attic 17% (6% - 28%) 13 Insulation: Floor 30% (0% - 97%) 2 Insulation: Wall 12% (0% - 47%) 4 Refrigerator 30% (23% - 37%) 42 Windows 29% (0% - 75%) 3 Total 22% (18% - 26%) 165 Table HES Program Freeridership Measure Freeridership (90% Confidence bounds) n Clothes Washer 32% (26% - 38%) 65 Clothes Washer-Tier One 10% (0% - 30%) 2 Clothes Washer-Tier Two 27% (22% - 32%) 64 Dishwasher 44% (38% - 50%) 54 Electric Water Heater 38% (22% - 54%) 13 Fixture 61% (42% - 80%) 5 Insulation: Attic 11% (7% - 15%) 36 Insulation: Floor 58% (31% - 85%) 5 Insulation: Wall 20% (0% - 48%) 5 Refrigerator 24% (20% - 28%) 59 Windows 21% (16% - 26%) 18 Total 23.3% (21% - 26%) 326 Table 28. HES Program Total Freeridership Program Year Realized Gross Savings Net of FreeFreeriders Freeridership % , , % , , % , , % All Years 1,515,981 1,149, % Spillover In addition to freeridership, Cadmus calculated spillover: the amount of additional savings generated by program participants but not captured by program records. Together, spillover and freeridership yielded the NTG value used in calculating final net savings of a program. The Cadmus Group, Inc. / Energy Services 75

78 Spillover occurs when customers choose to purchase energy-efficient measures or adopt energyefficient practices due to a program, yet choose not to participate or are otherwise unable to participate in a program. Because these customers are not participants, program records typically do not include savings generated by spillover impacts. In this evaluation, spillover was measured by asking participants purchasing a particular measure if, as a result of the program, they would have decided to install another efficient measure or would have undertaken some other efficiency improving activity. Deemed savings values, consistent with those used in calculating the gross savings value, were applied to relevant measures or practices. The sum of these savings values, divided by savings achieved through the program for each relevant measure, yielded spillover savings as a percentage of total savings resulting from program incentives for that measure. 42 Spillover participants indicated an average rating of 7.26 for the program s influence on their decisions to purchase other measures (based on a scale of 0 to 10, with 10 being most influential). In total, 29.2% of respondents indicated they purchased additional efficiency measures because of their participation in the HES Program. Annual findings are presented in Table 29, Table 30, and Table 31, with totals presented in Table 32. Table 29. HES 2006 Program Spillover Measure Survey Respondent Gross Savings (kwh) Survey Respondent Spillover (kwh) Spillover % Clothes Washer 14, % Electric Water Heater % Fixture % Refrigerator 1, % Total 16, % Table 30. HES 2007 Program Spillover Measure Survey Respondent Gross Savings (kwh) Survey Respondent Spillover (kwh) Spillover % Clothes Washer 14, % Dishwasher 2, % Electric Water Heater % Fixture % Insulation: Attic 22, % Insulation: Floor % Insulation: Wall 6, % Refrigerator 3, % Windows % Total 51,465 2, % 42 Detailed data provided by PECI was utilized for Gross Savings kwh in the spillover tables measure detail column. The Cadmus Group, Inc. / Energy Services 76

79 Table 31. HES 2008 Program Spillover Measure Survey Respondent Gross Savings (kwh) Survey Respondent Spillover (kwh) Spillover % Clothes Washer 14, % Clothes Washer-Tier One % Clothes Washer-Tier Two 14, % Dishwasher 1, % Electric Water Heater 1, % Fixture % Insulation: Attic 22,250 1, % Insulation: Floor 2, % Insulation: Wall 1, % Refrigerator 5, % Windows 2, % Total 68,240 4, % Table 32. HES Program Spillover Totals Program Year Realized Gross Savings Net of FreeFreeriders Total Spillover Spillover % , ,645 13, % , ,895 44, % , ,507 33, % All Years 1,515,981 1,149,047 91, % NTG Factor The NTG factor is the ratio of claimed savings, reported by Rocky Mountain Power, to savings realized through the evaluation, including impacts of CFL leakage, freeridership and spillover. Table 33 presents these findings. Across the three program years evaluated, average NTG was measured at 81.8%. Table 33. HES Program NTG Factor Program Year Realized Gross Savings (kwh) Evaluated Net Savings (kwh) NTG Factor , , % , , % , , % All Years 1,515,981 1,240, % The Cadmus Group, Inc. / Energy Services 77

80 6. Cost-Effectiveness Analysis To assess cost-effectiveness, evaluators conducted an analysis of program costs and benefits from five perspectives, using The Cadmus Group s DSM Portfolio Pro model. These perspectives include: a. PacifiCorp Total Resource Cost Test (PTRC): This test examines program benefits and costs from Rocky Mountain Power s and Rocky Mountain Power customers perspectives, combined. On the benefit side, it includes avoided energy costs, capacity costs, and line losses plus a 10% adder to reflect non-quantified benefits. On the cost side, it includes costs incurred by both the utility and participants. b. Total Resource Cost Test (TRC): This test examines program benefits and costs from Rocky Mountain Power s and Rocky Mountain Power customers perspectives, combined. On the benefit side, it includes avoided energy costs, capacity costs, and line losses. On the cost side, it includes costs incurred by both the utility and participants. c. Utility Cost Test (UCT): From Rocky Mountain Power s perspective, benefits are avoided energy and capacity costs and line losses. Costs include any program administration, implementation or incentive costs associated with funding the program. d. Ratepayer Impact (RIM): All ratepayers (participants and nonparticipants) may experience an increase in rates to recover lost revenue. This test includes all Rocky Mountain Power program costs as well as lost revenues. As benefits, this test includes all avoided energy costs, capacity costs, and line losses. e. Participant Cost Test (PCT): From this perspective, program benefits include bill reductions. Costs include any customer contribution to the measure cost. Table 34 summarizes the various components of the five tests. Table 34. Benefits and Costs Included in Various Tests Test Benefits Costs Present Value of Avoided Energy and Capacity Costs with Program Administrative and Marketing Cost + PTRC 10% Adder for Non-quantified Benefits Participant Cost Program Administrative and Marketing Cost + TRC Present Value of Avoided Energy and Capacity Costs Participant Cost UCT Present Value of Avoided Energy and Capacity Costs Program Administrative, Marketing and Incentive Cost RIM Present Value of Avoided Energy and Capacity Costs Program Administrative and Marketing Cost + Present Value of Lost Revenues PCT Present Value of Bill Savings Participant Share of Measure Cost Table 35 provides selected inputs to the cost analysis. These include evaluated energy savings for each year (from Table 33, above), discount rate, line loss, and program costs. Other than energy savings, these values are provided by Rocky Mountain Power. The discount rate is from Rocky Mountain Power s 2008 Integrated Resource Plan. Rocky Mountain Power also provided values for line loss and the program costs. The implementation cost is the amount Rocky Mountain Power paid to PECI, the implementing contractor. The Cadmus Group, Inc. / Energy Services 78

81 Table 35. Selected Cost-Effectiveness Analysis Inputs Input Description Net Program Savings (kwh/year) 227, , ,268 Discount Rate 7.40% 7.40% 7.40% Line Loss 11.39% 11.39% 11.39% Net Participant Costs $65,539 $165,539 $360,706 Program Costs Implementation Cost $63,137 $122,255 $203,679 Incentive Costs $26,641 $128,153 $225,652 Utility Administrative Costs $18,090 $10,636 $21,062 Total Program Costs $107,868 $261,044 $450,393 Program benefits are comprised of energy savings and their associated avoided costs. The energy savings used in the cost-effectiveness analysis are the evaluated kwh from this study. Table 36, Table 37, and Table 38 present the results of the cost-effectiveness analysis for the Program in 2006, 2007, and 2008 respectively. Table 39 summarizes cost-effectiveness analysis for the program for all years ( ). All analyses are based on the Rocky Mountain Power 2008 IRP 60% load factor (LF) eastside residential righting IRP decrement. 43 Detailed incremental costs and measure lives can be found in Appendix B. Table 36. Cost-Effectiveness Summary for 2006 IRP 60% LF Eastside Residential Lighting Decrement Cost Effectiveness Test Levelized $ / kwh Costs Benefits Net Benefits Benefit / Cost Ratio Total Resource + Conservation Adder (PTRC) $0.089 $146,766 $121,436 -$25, Total Resource No Adder (TRC) $0.089 $146,766 $110,396 -$36, Utility (UCT) $0.065 $107,868 $110,396 -$2, Ratepayer Impact (RIM) $0.137 $226,252 $110,396 -$115, Participant (PCT) $0.040 $65,539 $145,025 $79, Lifecycle Revenue Impact ($/kwh) $ Discounted Participant Payback (Years) IRP decrements are detailed in Appendix G of PacifiCorp s 2008 Integrated Resource Plan Volume II Appendices: urce_planning_6.pdf The Cadmus Group, Inc. / Energy Services 79

82 Table 37. Cost-Effectiveness Summary for 2007 IRP 60% LF Eastside Residential Lighting Decrement Cost Effectiveness Test Levelized $ / kwh Costs Benefits Net Benefits Benefit / Cost Ratio Total Resource + Conservation Adder (PTRC) $0.061 $298,430 $385,716 $87, Total Resource No Adder (TRC) $0.061 $298,430 $350,651 $52, Utility (UCT) $0.053 $261,044 $350,651 $89, Ratepayer Impact (RIM) $0.132 $647,746 $350,651 -$297, Participant (PCT) $0.034 $165,539 $514,854 $349, Lifecycle Revenue Impact ($/kwh) $ Discounted Participant Payback (Years) 0.97 Table 38. Cost-Effectiveness Summary for 2008 IRP 60% LF Eastside Residential Lighting Decrement Cost Effectiveness Test Levelized $ / kwh Costs Benefits Net Benefits Benefit / Cost Ratio Total Resource + Conservation Adder (PTRC) $0.118 $585,447 $424,308 -$161, Total Resource No Adder (TRC) $0.118 $585,447 $385,734 -$199, Utility (UCT) $0.091 $450,394 $385,734 -$64, Ratepayer Impact (RIM) $0.176 $873,045 $385,734 -$487, Participant (PCT) $0.073 $360,706 $648,304 $287, Lifecycle Revenue Impact ($/kwh) $ Discounted Participant Payback (Years) 4.12 Table 39. Cost-Effectiveness Summary for IRP 60% LF Eastside Residential Lighting Decrement Cost Effectiveness Test Levelized $ / kwh Costs Benefits Net Benefits Benefit / Cost Ratio Total Resource + Conservation Adder (PTRC) $0.088 $932,184 $848,427 -$83, Total Resource No Adder (TRC) $0.088 $932,184 $771,298 -$160, Utility (UCT) $0.07 $741,392 $771,298 $29, Ratepayer Impact (RIM) $0.151 $1,586,250 $771,298 -$814, Participant (PCT) $0.051 $532,384 $1,186,449 $654, Lifecycle Revenue Impact ($/kwh) $ The Cadmus Group, Inc. / Energy Services 80

83 7. Appendices Appendix A: Available Measures Tariff Information Measure Tariff Effective Start Date Table 40. Available Measures Tariff Information - Appliances Tariff Effective End Date Qualifications From Tariff Ceiling Fan 1/1/2007 current ENERGY STAR qualified Clothes Washer* Clothes Washer* Clothes Washer Clothes Washer Clothes Washer Recycling Dishwasher * 5/31/ /31/200 6 Energy Star qualified. MEF of 1.42 for balance of MEF of for 2007 and beyond (until further Energy Star specification changes) 1/1/2007 4/31/2008 Energy Star qualified. MEF of 1.42 for balance of MEF of for 2007 and beyond (until further Energy Star specification changes) 5/1/2008 current Washing machines: Split measure into two tiers for efficiency and incentives. Tier 1: Modified Energy Factor (MEF) /1/2008 current Washing machines: Split measure into two tiers for efficiency and incentives. Tier 2: Modified Energy Factor (MEF) /31/2006 current Only available to purchasers of new Energy Star Washing machine who participated in the program. Old unit must be operable to qualify 1/1/2007 4/30/2008 Available after January 2007 when revised Energy standards are in effect(energy Star-EF 0.68+) Contractor/ Retailer Incentive Require Program Qualified Contractor Customer Availability Incentive Year round $20 $0 No Year round $75 $0 No Year round $75 $0 No Year round $50 $0 No Year round $100 $0 No Year round $0 $25 Yes Year round $20 $0 No Dishwasher 5/1/2008 current Change.68 Energy Year round $20 $0 No The Cadmus Group, Inc. / Energy Services 81

84 Measure Electric Water Heater* Electric Water Heater Evaporative Cooler* Evaporative Cooler Tariff Effective Start Date Tariff Effective End Date Qualifications From Tariff Factor (EF) to.65 EF to align program eligibility requirements with final Energy Star qualifications of.65. 5/31/2006 4/30/ gallons and EF gallons and EF 5/1/2008 current 40+ gallon tank and EF Availability Customer Incentive Contractor/ Retailer Incentive Require Program Qualified Contractor Year round $50 $0 No Year round $50 $0 No 5/31/2006 4/30/2008 Permanently installed Year round $250 $25 Yes 5/1/2008 current Based on price, availability, ease of installation and experience in other markets, the incentive offer will be adjusted to $100 for the end user, the contractor incentive will be discontinued, and the measure will be promoted at the retail level. Year round $100 $0 Yes Fixtures 5/31/2006 current ENERGY STAR Year round $20 $0 No qualified Refrigerator 5/31/2006 current ENERGY STAR rated Year round $20 $0 No *Program is no longer active Table 41. Idaho Available Measures Tariff Information Home Improvement Measure Insulation- Attic* Insulation- Attic Tariff Effective Start Date Tariff Effective End Date Qualifications From Tariff 5/31/2006 4/30/2008 Per square foot, R-11 in the wall and R-19 increments based on purchased material rating, not rating of overall installed levels. 5/1/2008 current Available only to customers with electric heat systems or ducted unitary cooling equipment service 80% Availability Year round Year round Customer Incentive $1.00/sq. ft. $0.50/sq. ft. Contractor/ Retailer Incentive Require Program Qualified Contractor $0 No $0 No The Cadmus Group, Inc. / Energy Services 82

85 Measure Insulation- Floor* Insulation- Floor Insulation- Wall* Insulation- Wall Tariff Effective Start Date Tariff Effective End Date Qualifications From Tariff of the floor area. Min R- 19 5/31/2006 4/30/2008 Per square foot, R-11 in the wall and R-19 increments based on purchased material rating, not rating of overall installed levels. 5/1/2008 current Available only to customers with electric heat systems or ducted unitary cooling equipment service 80% of the floor area. Min R- 19 5/31/2006 4/30/2008 Per square foot, R-11 in the wall and R-19 increments based on purchased material rating, not rating of overall installed levels. 5/1/2008 current Available only to customers with electric heat systems or ducted unitary cooling equipment service 80% of the floor area. Min R- 11 Windows 5/31/2006 current Available only to customers with electric heat systems or ducted unitary cooling equipment service 80% of the floor area. U- Factor of.32 or better *Program is no longer active Availability Year round Year round Year round Year round Year round Customer Incentive $1.00/sq. ft. $0.50/sq. ft. $1.00/sq. ft. $0.50/sq. ft. $1.50/sq. ft. Contractor/ Retailer Incentive Require Program Qualified Contractor $0 No $0 No $0 No $0 No $0 No The Cadmus Group, Inc. / Energy Services 83

86 Table 42. Idaho Available Measures Tariff Information HVAC Measure CAC Equipment Tariff Effective Start Date Tariff Effective End Date Qualifications From Tariff 1/1/2007 current 15+ SEER/12.5+ EER & TXV Require Availability Customer Incentive Contractor/ Retailer Incentive Program Qualified Contractor Year round $250 $25 Yes CAC Installation 1/1/2007 current 13+ SEER & TXV & Year round $50 $75 Yes CAC Sizing 1/1/2007 current 13+ SEER & TXV & best practices installation (charge and airflow) Year round $50 $25 Yes CAC/Heat Pump Tune-Ups Duct Sealing Heat Pump Conversion Heat Pump Upgrade 5/31/2006 current PTCS requirements performed by program qualified contractor 5/31/2006 current PTCS requirements performed by program qualified contractor 5/1/2008 current Convert an electric heating system to a heat pump 14+ SEER & EER & 8.2+ HSPF 5/1/2008 current Upgrade a baseline heat pump to high efficiency heat pump 14+ SEER & EER & 8.2+ HSPF Year round $100 $25 Yes Year round $150 $50 Yes Year round $350 $25 Yes Year round $250 $25 Yes The Cadmus Group, Inc. / Energy Services 84

87 Table 43. Idaho Available Measures Tariff Information Lighting Tariff Effective Start Date Tariff Effective End Date Qualifications From Measure Tariff Lighting* 5/31/2006 4/30/2008 Screw-in, select common wattages, ENERGY STAR qualified Availability October- March Customer Incentive Contractor/ Retailer Incentive $0 Manufacturer buy-down notto-exceed $1.35 Retail price not-to-exceed $1.50 Require Program Qualified Contractor No Lighting 5/1/2008 current Including specialty bulbs, and respond to changing CFL prices, the Company is proposing to use the markdown/buy-down mid-market incentives to bring the final cost to the customer to $.99 - $2.75 and to offer lighting incentives year round. Year round $0 Manufacturer buy-down notto-exceed $1.35 Retail price not-to-exceed $0.99-$2.75 No *Program is no longer active The Cadmus Group, Inc. / Energy Services 85

88 Appendix B: Cost-Effectiveness Inputs Measure Unit definition Incremental measure cost ($) Measure life (yrs.) 2006 CFL # of CFL Clothes Washer # of Clothes Washer Electric Water Heater # of Electric Water Heater Fixture # of Fixture Insulation: Attic sq. ft. of insulation Refrigerator # of Refrigerator Ceiling Fans # of Ceiling Fans Central A/C Equipment # of Central A/C Equipment CFL # of CFL Clothes Washer # of Clothes Washer Dishwasher # of Dishwasher Electric Water Heater # of Electric Water Heater Evaporative Cooler # of Evaporative Cooler (1,000.00) 15 Fixture # of Fixture Insulation: Attic sq. ft. of insulation Insulation: Floor sq. ft. of insulation Insulation: Wall sq. ft. of insulation Refrigerator # of Refrigerator Windows sq. ft. of Windows Ceiling Fans # of Ceiling Fans Central A/C Equipment # of Central A/C Equipment CFL # of CFL Clothes Washer # of Clothes Washer Clothes Washer-Tier One # of Clothes Washer-Tier One Clothes Washer-Tier Two # of Clothes Washer-Tier Two Dishwasher # of Dishwasher Electric Water Heater # of Electric Water Heater Fixture # of Fixture Heat Pump Upgrade # of Heat Pump Upgrade Insulation: Attic sq. ft. of insulation Insulation: Floor sq. ft. of insulation Insulation: Wall sq. ft. of insulation Proper CAC Install # of Proper CAC Install Proper CAC Sizing # of Proper CAC Sizing Refrigerator # of Refrigerator Windows sq. ft. of Windows The Cadmus Group, Inc. / Energy Services 86

89 Appendix C: Dealer Survey Summary 44 Dealer Survey Quantity Sold Number of Surveyed Dealers Dealer Survey Gross Savings (kwh) Total Quantity Sold Total Gross kwh Savings Total Measure Dealers Clothes Washers-Tier One ,919 1, ,403 Clothes Washers-Tier Two , ,461 Dishwasher , ,537 Electric Water Heater , ,672 Fixture , ,200 Insulation: Attic 204, , , ,265 Insulation: Floor 17, ,712 20, ,167 Insulation: Wall 25, ,256 34, ,891 Refrigerator , ,439 Windows 8, ,699 15, ,367 Central A/C Equipment Total 256, , , , This table excludes CFLs, Evaporative Coolers, Proper CAC Sizing, CAC Installation, CAC/Heat Pump Tune- Ups, Clothes Washer Recycle, Duct Sealing, Room AC Recycle, Room AC Equipment, and Ceiling Fans. These measures were not represented in the dealer survey results. The Cadmus Group, Inc. / Energy Services 87

90 Appendix D: Dealer Sales Perceptions % of Customers Seeking Energy- Efficient Measures % of Residential Sales that are Energy- Efficient Residential Sales as Measure % of Total Sales Clothes Washer 60% 99% 74% $378 Refrigerator 58% 92% 68% $224 Dishwasher 60% 95% 67% $134 Electric Water Heater 14% 99% 53% $169 Windows 95% 100% 100% $36 Insulation 80% 100% 65% $0.67 Central A/C Equipment 20% 50% 40% $1,500 Evaporative Cooler 96% 100% - Room AC New Purchase 12% 95% 47% $60 Incremental Cost of Efficient Measure The Cadmus Group, Inc. / Energy Services 88

91 Appendix E: Attic, Floor and Wall Insulation Participant Survey Instrument Provided under separate cover. The Cadmus Group, Inc. / Energy Services 89

92 Appendix F: All non-insulation Measures Participant Survey Instrument Provided under separate cover. The Cadmus Group, Inc. / Energy Services 90

93 Appendix G: CFL Nonparticipant Leakage Survey Instrument Provided under separate cover. The Cadmus Group, Inc. / Energy Services 91

94 Appendix H: CFL Retailer Survey Instrument Provided under separate cover. The Cadmus Group, Inc. / Energy Services 92

95 Appendix I: Dealer Survey Instrument Provided under separate cover. The Cadmus Group, Inc. / Energy Services 93

96 Appendix J: Implementer Discussion Guide Provided under separate cover. The Cadmus Group, Inc. / Energy Services 94

97 Appendix K: Program Logic Model Figure 53. Rocky Mountain Power Home Energy Savings (HES) Program Logic Model Table 44. Program Theory and Indicators Link Program Theory Indicators The HES program design leads to training third party contractors and PECI staff, marketing and outreach activities, and contracts with manufacturers Trained third party contractors have the ability to properly install measures 7 Contracts negotiated and executed with lighting manufacturers 8 Outreach to retailers and dealers result in stock to support sales of CFLs and ENERGY STAR products 9 Marketing and outreach promote incentives for HES measures Program design Number of training sessions Number of contractors attending sessions Marketing plan established Number of outreach events scheduled Installation paperwork completion Number of trained third party contractors Number of manufacturers contracted Number of bulbs incented Number and distribution of Retailers carrying discounted bulbs and ENERGY STAR products Number of efficient products stocked Number of marketing campaigns/pieces produced Number participants in outreach events The Cadmus Group, Inc. / Energy Services 95