OKLAHOMA GAS and ELECTRIC COMPANY

Transcription

1 OKLAHOMA GAS and ELECTRIC COMPANY 2015 Arkansas Energy Efficiency Program Portfolio Annual Report Section 9: Annual Reporting Requirements, and Order No. 18 in Docket No R. Version 3.0 September 27, 2013 May 1,

2 Table of Contents 1.0 Executive Summary Portfolio Programs OG&E/AOG Weatherization Program Student Energy Education Program (LivingWise ) Commercial Lighting Program Commercial and Industrial Standard Offer Program Arkansas Weatherization Program (AWP) Multi-Family Direct Install Program Supplemental Requirements Staffing Stakeholders Activities Information provided to Customer to Promote EE Appendix A: EM&V Contractor Reports Appendix B: Promotional and Educational Materials 2

3 1.0 Executive Summary Oklahoma Gas and Electric Company ( OG&E or Company ) hereby submits its Energy Efficiency ( EE ) program portfolio Annual Report for Program Year ( PY ) 2015 to the Arkansas Public Service Commission ( APSC or Commission ) pursuant to Order No. 18 in Docket R. This report is required to be filed annually by May 1, in accordance with Order No. 7 filed in Docket U on September 9, HISTORY: OG&E began implementation of EE programs in Arkansas in December 2007 with its Quick Start program portfolio. The Quick Start program continued through December 31, That portfolio contained seven programs in total; five OG&E administered programs and two state administered programs. The OG&E administered programs included; the Livingwise Student Energy Education program, the Residential Energy Audit program, the Commercial Lighting program, the Motor Replacement program, and the Compact Fluorescent Light ( CFL ) program. The two state administered programs included were the Arkansas Weatherization Program ( AWP ), and the Energy Efficiency Arkansas ( EEA ) program. The CFL program was not launched with the other Quick-Start programs and was ultimately discontinued. The Quick-Start portfolio allowed OG&E to build a program delivery framework for its customers in the Arkansas jurisdiction. The initial Comprehensive Energy Efficiency Program ( CEEP ) was approved on February 3, 2010 for an 18-month implementation period ending on June 30, The initial CEEP included the continuation of the two statewide programs, AWP and EEA, and three OG&E programs; Livingwise Student Energy Education, Commercial Lighting, and Motor Replacement programs. The Residential Energy Audit program was renamed the Custom Energy Report ( CER ) program and the new OG&E Weatherization program was introduced. The OG&E Weatherization program was established to offer weatherization for residential customers that would not otherwise qualify for the AWP. The current Comprehensive Portfolio was approved on June 30, 2011 for the remainder of PY The PY s 2012 and 2013 were subsequently approved on December 30, The two statewide programs, AWP and EEA, were continued along with OG&E s Commercial Lighting program and the Livingwise Student Energy Education program. The CER program was discontinued as an EE program, but is still available through OG&E s website. The OG&E Weatherization program was modified to a collaborative program with Arkansas Oklahoma Gas Corporation ( AOG ) to take advantage of administrative efficiencies and cost sharing. The Motor Replacement Program was incorporated into the new Commercial and Industrial Standard Offer Program ( C&I SOP ). In addition, new programs were created for both residential and nonresidential customers. For residential customers, the HVAC tune-up and duct repair program, the Window Unit A/C program, and the Multi- Family program were created to provide a more diverse residential portfolio of programs. After the plan was approved, it was determined the Multi-Family program could not be implemented as designed and 3

4 was discontinued. For nonresidential customers, in addition to the C&I SOP, the Commercial Tune-up program was created to inspect and tune commercial HVAC systems. In January 2013, the APSC opened Docket U to resolve issues related to the development and implementation of the second three year cycle of EE programs in Arkansas. In Order No. 2 of that same Docket, the APSC approved the request of the Parties Working Collaboratively ( PWC ) extending the filing date for the second three year cycle of EE programs from June 1, 2013 to June 1, The Commission also directed that energy savings targets, budgets, and the incentive structure previously approved by the Commission in the TF dockets for use by the Utilities and EE program administrators for the existing PY 2013 shall also be used for PY The exception to this was if the Utilities sought Commission approval of proposed modifications to their EE portfolios. OG&E reviewed its portfolio performance through 2013 and filed an application to modify its existing portfolio to enhance its ability to achieve Commission approved targets for OG&E s interim filing proposed to modify its portfolio by discontinuing three programs, adding one new program, increasing the budget for industrial programs, and aligning its rebate structure with Commission approved targets. The three programs that were discontinued were the Residential HVAC program, the Commercial and Industrial HVAC program, and the Window Unit AC program. The new program added was the Multi- Family Direct Install program. On March 17, 2014, the Commission approved OG&E s modified portfolio. In February 2014, the APSC issued Order No. 15, in Docket U extending for a second year, the filing date for the second three year cycle of programs to June 1, The extension was to allow time to complete efforts to develop a collaborative weatherization program, core C&I programs and complete a Potential Study. In addition the Commission approved a target increase for PY To meet the increased target for 2015, OG&E filed an application to increase the budgets for 3 of its programs to enhance its ability to achieve the new target. On April 1, 2015 the APSC approved OG&E s budget increases for PY Table 1 below summarizes historical annual incremental EE savings achieved by OG&E s previous efforts: Table 1- Historical Annual Incremental EE Savings Achieved Program Year Energy (kwh) Demand (kw) ,434, ,607, ,143,096 1, ,985,328 1, ,595,741 1, ,410,729 2, ,794,070 2,883 4

5 For PY 2015, the savings target increased to 0.90% of 2013 kwh sales. OG&E filed a modified portfolio to address the increased savings target. The program mix remained the same; however, OG&E requested and the APSC approved a 40% increase in budget to achieve the increased savings target. GOALS AND OBJECTIVES: Order No. 15 in Docket U established default energy savings target as a percent of 2010 energy sales. In 2015, the energy savings target increased to 0.90% of 2013 energy sales as adjusted for selfdirects. The annual energy savings targets are shown in Table 2 below. Table 2- Annual Energy Savings Targets Baseline Sales Year Program Year Percent of Sales Energy Savings Targets (kwh) % 6,752, % 11,363, % 16,843, % 16,287, % 19,879,081 OG&E s energy savings target for 2015 was 19,879,081 kwh. The 2015 EE portfolio actual results achieved for energy savings were 20,543,040 kwh or 103% of target. MAJOR ACCOMPLISHMENTS: Since beginning its EE efforts in 2008, OG&E reached its highest level of energy savings in 2015 and for the first time in program history achieved over 100% of target. This was a tremendous accomplishment for OG&E as our target achievement has proven challenging in the past with limited customer base in the Industrial sector. 5

6 PROGRESS ACHIEVED: The portfolio savings continues to steadily climb year over year since The historical annual energy savings to target achievements are depicted in Table 3 below. Table 3- Historical Annual Energy Savings to Target Achievement Program Year Energy Savings Targets (kwh) Energy Savings Achieved (kwh) % of Target Achieved ,752,758 4,985,328 74% ,363,560 7,595,741 67% ,843,560 13,410,729 80% ,287,689 13,794,070 85% ,879,081 20,543, % Figure 1- Energy Savings to Target Achievement In 2015, OG&E began engaging CLEAResult to aid in closing the gaps between achieved energy savings and targets. OG&E s partnership with CLEAResult has proven to be a successful collaboration. The achieved energy savings for 2015 is 103% of the Commission approved target and a 49% increase from OG&E s 2014 achieved energy savings. This continued increase in year over year savings reflects significant enhancements in many program areas and confirms OG&E s commitment to achieve energy savings. 6

7 OG&E s achieved annual incremental savings in gigawatt hours is represented in Table 4 below. Table 4- OG&E s Achieved Annual Incremental Savings Program Year GWh Savings HIGH-LEVEL RECAP: The 2015 portfolio produced 20,543,040 kwh or 103% of the Commission approved target and 109% of OG&E s energy savings goal based on 2013 kwh sales as adjusted for self-direct exemptions. These on-going energy savings will accumulate over the life of the EE measures. The EE portfolio recoverable expenses of $6,075,144 for 2015 were 94% of the approved annual budget of $6,470,885. Customer incentives and rebates account for 68% of the total portfolio expenses. HIGHLIGHTS OF WELL-PERFORMING PROGRAMS: The Commercial and Industrial ( C&I ) programs saw tremendous success in The Commercial Lighting program achieved 99% of its savings goal while remaining under budget at 97% spent. The most successful program accomplishment in 2015 is the Standard Offer Program which achieved 103% of its 2015 energy savings goal, up from 45% in 2014, 35% in 2013, and 15% in The C&I SOP program savings account for 51% of the C&I energy savings as well as 33% of the total Portfolio energy savings. OG&E began implementation of the Multi-Family Direct Install program in The program achieved 139% of its 2015 savings goal and directly installed EE measures in 1,795 multi-family units while only spending 84% of the program budget. This program accounted for 54% of OG&E s residential portfolio energy savings and also penetrates a hard to reach customer segment allowing for more customers to participate and be further educated in the energy management of their home. The Student Energy Education program achieved 113% of its 2015 savings goal by delivering 1,885 customized kits to 6 th grade students across the OG&E Arkansas service territory. The program achieved the energy savings while only spending 94% of the program budget. This program has been very well 7

8 received in the classroom and offers teachers and students a unique avenue for learning about the environment and the importance of energy efficiency. WHAT S WORKING AND WHAT S NOT: The residential portfolio of EE programs is working well. With the addition of the Multi-Family Direct Install program in 2014, OG&E continued to reach additional residential customers throughout The residential portfolio of OG&E administered programs achieved 109% of energy savings goals while spending 91% of the total residential budget. The current EM&V reports validate the impact and process success of OG&E s residential programs. The C&I portfolio of EE programs has seen increasing improvement in 2015 by achieving 101% of energy savings goals while spending 98% of budget. This continued improvement can be directly attributed to the addition of CLEAResult s resources and expertise. PLANNED CHANGES: On June 1, 2015, OG&E filed the 2016 Portfolio Plan which included minimal modifications to the existing 2015 portfolio. Among the modifications was the transfer of resources from the AWP program to the OGE/AOG Weatherization program as part of the Commission ordered wind down of the AWP program. In addition to the winding down of the AWP program, OG&E is incorporating three new measures in 2016 to reflect the Unified Weatherization approach approved by the Commission. Lastly, OG&E adjusted the energy savings goal for the OGE/AOG Weatherization program downward to align with the increased contractor and measure costs as well as the measure baseline shifts which have caused the cost per kwh saved in this program to also increase. The total portfolio costs and energy savings targets remain the same from 2015 to OG&E will continue to engage CLEAResult to assist in implementing existing programs and determining the potential for additional savings. TRAINING ACHIEVEMENTS: OG&E provided training to approximately 807 individuals in To accomplish this, OG&E hosted seminars for weatherization contractors and crews to explain the benefits of the residential programs. OG&E also provided educational sessions with commercial and industrial customers on the benefits of energy efficiency. 8

9 APSC FILED Time: 8/19/2016 2:29:24 PM: Recvd 8/19/2016 2:28:39 PM: Docket TF-Doc. 322 EM&V ACTIVITIES: Applied Energy Group ("AEG"), fonnerly EnerNOC Utility Solutions, was selected to perform the evaluation, measurement, and verification ("EM&V") for all of the EE programs in the portfolio except the AWP and OG&E/AOG Weatherization programs. ADM Associates, Inc. performs the EM&V for both the AWP and OG&E/AOG Weatherization programs. Using the same contractor for both weatherization programs ensures consistency in evaluation. For PY 2015, both EM&V contractors performed process and impact evaluations of the programs delivering measure by measure evaluated net savmgs. The three EM&V reports detail their findings and are included in Appendix A of this annual report. LONG-TERM ENERGY SA VIN GS: The current program portfolio was developed to meet the energy efficiency targets established by the APSC in Order No.31 in Docket U. The expected kw and kwh savings delivered by this portfolio, estimated kw and kwh savings from future portfolios, and the cumulative kw and kwh savings from previous portfolios are included in the Company's load forecast. The Integrated Resource Plan incorporates this information in its planning report. EE OVERVIEW: The following three tables provide an overview of the EE portfolio results for PY 2015: Table 5- Portfolio Summary 2015 Portfo llo Summary Net Energy Savings Co.st Cost-Benefits Actual Performance TRC TRC Demand Energy Expenses LCFC Incentives Net Benefits Ratio MW MWh 3 20,543 $ 6,075, 144 $ 1,522,601 $452,962 $ 10,864,

10 Table 6- Portfolio Costs by Program Summary 2015 % of Budget Actual Budget Program Name Target Sector Program Type ($) ($) Multi-Family Direct Install Residential Market Specific/Hard to Reach 773, ,289 84% OG&E - AOG Weatherization Residential Whole Home 2,351,220 2,191,244 93% Student Energy Education Residential Behavior/Education 88,315 83,359 94% C&I Standard Offer Commercial & Industrial Prescriptive/Standard Offer 1,572,241 1,572, % Commercial Lighting Commercial & Industrial Prescriptive/Standard Offer 1,561,360 1,509,288 97% Arkansas Weatherization Program (AWP) Residential Whole Home 15,730 5,294 34% Energy Efficiency Arkansas (EEA) All Classes Behavior/Education 24,000 18,319 76% Regulatory ,000 43,110 51% Total 6,470,885 6,075,144 94% Table 7- Portfolio Costs by Type Summary EE Portfolio Cost by Program EE Portfolio Summary by Cost Type EE Program Cost Summary 2015 Total Cost % of Budget Actual % of Cost Type Total ($) ($) Total Planning / Design 1% 32,578 23,239 0% Marketing & Delivery 24% 1,527,364 1,361,729 22% Incentives / Direct Install Costs 63% 4,103,287 4,129,181 68% EM&V 6% 365, ,354 3% Administration 6% 357, ,532 5% Regulatory 1% 85,000 43,110 1% 100% 6,470,885 6,075, % Figure 2- Portfolio Costs by Type Summary 10

11 Table 8- Company Statistics 1 Revenue and Expenses Budget Actual Company Statistics Energy Plan Evaluated Program % of % of Portfolio % of Portfolio % of Year Total Annual Net Annual Energy Net Annual Energy Total Revenue Budget Revenue Spending Revenue Energy Sales Savings Sales Savings Sales (a) (b) (c) (d) (e) (f) ($000's) ($000's) (%=b/a) ($000's) (%=b/a) (MWh) (MWh) (%=b/a) (MWh) (%=b/a) 2011 $ 180,406 $ 2, % $ 2, % 2,802,634 6, % 4, % 2012 $ 167,615 $ 3, % $ 3, % 2,743,246 14, % 7, % 2013 $ 179,047 $ 3, % $ 3, % 2,710,927 20, % 13, % 2014 $ 184,882 $ 4, % $ 4, % 2,693,601 14, % 13, % 2015 $ 168,544 $ 6, % $ 6, % 2,604,925 19, % 20, % Figure 3- Company Statistics 1 Total Annual Energy Sales include self-direct customer sales. 11

12 2.0 Portfolio Programs 2.1 OG&E/AOG Weatherization Program Program Description Designed to target residential customers and allow them to participate in the program at no cost, this program provides customers the opportunity to actively manage their energy costs. The program targets residential single-family homes which were built before 2005, specifically those that are severely energy inefficient. Homes that meet these criteria begin with an energy audit utilizing blower door technology on the structure to capitalize on specific weatherization techniques. The program is designed to upgrade and improve the thermal envelope of the dwelling. OG&E serves more than 54,000 residential customers in its Arkansas service territory and has estimated there are as many as 30,000 homes in need of weatherization improvements. OG&E views the weatherization program as a key component in its EE portfolio, and uses three independent contractors: DK Construction, based in Van Buren (Crawford County), Total Home Efficiency and Williams Energy, both based in south Fort Smith (Sebastian County). Each contractor has certified Building Performance Institute ( BPI ) or Residential Energy Services Network ( RESNET ), Home Energy Service Professionals ( HESP ) auditors on staff. OG&E personnel will arrange training to maintain consistent implementation practices across the weatherization program. Contractors are encouraged to attend trainings and receive additional education on weatherization of homes, both online and in classrooms, for improvement in proper home weatherization techniques. Energy saving equipment that is installed or improvements made in the homes include: replacement of glass and/or doors, CFLs, return air cavity sealing, CO detectors, smoke detectors, attic insulation, air infiltration, and water heater pipe wrap. Utilizing blower door technology, the contractors are able to locate and seal larger areas of air infiltration in the homes. OG&E and Arkansas Oklahoma Gas Corporation ( AOG ) continue to work together with contractors to ensure program success. The partnership with AOG has proven to be a successful collaboration for the joint weatherization program. The ability to work together with other utilities is an ongoing effort to combine resources as well as to reach more customers in our over-lapping service territories. OG&E and AOG, along with the efforts of Frontier Associates, continue to fine tune the software package to meet the criteria of the most current Technical Reference Manual ( TRM ). The improvements are to help ensure the software captures more accurate field data, as well as a split payment process for each of the utilities to pay the individual contractors assigned to the program. 12

13 2.1.2 Program Highlights Civic and community presentations highlighting the program were conducted throughout communities served by OG&E promoting the Weatherization program. OG&E weatherized 1,325 homes in The OG&E/AOG Weatherization Program was used as the model for the new Arkansas Unified Weatherization Program Program Budget, Savings and Number of Measures Table 9- OG&E-AOG Weatherization Program Summary OG&E - AOG Weatherization Cost Energy Savings (kwh) Demand Savings (kw) Participants Program Budget Actual % Plan Evaluated % Plan Evaluated % Plan Actual % Program Year 2013 $ 2,302,446 $ 2,302, % 2,994,261 3,655, % 516 1, % 1,620 1, % Program Year 2014 $ 2,231,745 $ 2,231, % 3,497,085 3,679, % 990 1, % 1,620 1,372 85% Program Year 2015 $ 2,351,220 $ 2,191,244 93% 3,497,085 3,000,505 86% % 1,620 1,325 82% Description of Participants Participants of this program fall into two groups. The first group are more mature customers living in single-family homes built more than 15 years ago. Many of these participants are either retired or near retirement and own their own home. They maintain a low-key lifestyle and typically do not have young children living at home. The second group of participants are younger customers with young children living at home. This group of participants include single-family home renters and those that own their home Challenges and Opportunities Working in conjunction with the Community Clearinghouse, OG&E has been able to maintain a steady pace in obtaining and qualifying customers homes in a timely manner for weatherization. As this program matures, long term lead generation will be necessary for future success. The homes that were designated as AWP homes will now be eligible for this program going forward. 13

14 Opportunities will exist for expanding the market of eligible homes when the new program begins in 2016 by modifying the age limit criteria Planned or Proposed Changes to Program and Budget The OG&E/AOG Weatherization Program will be consistent with the new statewide Unified Weatherization Program in

15 2.2 Student Energy Education Program (LivingWise ) Program Description The program provides 6 th grade teachers and their students a curriculum on home EE. At the end of the curriculum a LivingWise education kit provides students the opportunity to participate with their families on energy awareness. The LivingWise education kit contains three CFL bulbs, two faucet aerators, one low-flow showerhead, one LED night light, a thermometer, and a student handbook on EE for the home and community. The students take the LivingWise kit home and install the EE measures with the assistance of their parents. OG&E provides a list of schools each semester to Resource Action Programs ( RAP ) for potential participation in the LivingWise Program. RAP contacts the school, enrolls the teacher and quantifies the number of students. A list of enrolled schools and participation information is sent to OG&E each month. RAP mails the kits to the teachers enrolled in the program. Finally, RAP follows up with teachers on class participation during the curriculum and the students interaction with parents including the installation of the energy savings measures. There was an overwhelming consensus from all participating teachers that it was an informative, easy to understand curriculum. The teachers expressed that with the uncertain environmental and energy situation, the teaching materials were both timely and important Program Highlights The LivingWise Program provided EE and environmental awareness education for 1,885 students from January 2015 through December 31, 2015, targeting 9 school districts in Arkansas. An OG&E customized box was used to improve the generic look for the LivingWise Kits. OG&E utilized Community Coordinators along with key contact personnel for promotion of the program. A report is submitted to OG&E at the end of each semester detailing the activity, the results and the participation level and acceptance of the program. The program achieved 113% of the energy savings goal in

16 2.2.3 Program Budget, Savings and Participants Table 10- Student Energy Education Program Summary Student Energy Education Cost Energy Savings (kwh) Demand Savings (kw) Participants Program Budget Actual % Plan Evaluated % Plan Evaluated % Plan Actual % Program Year 2013 $ 76,298 $ 73,907 97% 152, ,084 83% % 1,840 2, % Program Year 2014 $ 88,694 $ 88, % 288, , % % 1,840 1, % Program Year 2015 $ 88,315 $ 83,359 94% 288, , % % 1,840 1, % Description of Participants This program focuses on 6th grade students in the public school system. The kit provides several easily installed EE products for the home allowing students and parents to have conversations about using energy efficiently. This program promotes EE education to the future home owners so they will understand the impacts of energy conservation Challenges and Opportunities OG&E s success with this program has been through key contacts in each of the school districts. Each of the participating schools within the OG&E service territory have embraced the concept and curriculum provided through RAP. The annual updating of the Arkansas TRM has historically presented challenges to the program s cost-effectiveness Planned or Proposed Changes to Program and Budget OG&E plans to continue its support for the Student Energy Education program through The kits for 2016 will include two CFL light bulbs, two LED light bulbs, two faucet aerators, one low-flow showerhead, one LED night light, a thermometer, and a student handbook on EE for the home and community. 16

17 2.3 Commercial Lighting Program Program Description The purpose of the Commercial Lighting program is to provide incentives to the OG&E Commercial and Industrial customers to install or replace lighting with more efficient equipment. The program targets commercial, public authority, schools, and industrial facilities of all sizes with a focus on the small to medium-sized facilities, where saturation rates and awareness levels of high efficiency lighting are expected to be lower than in larger commercial operations. To encourage commercial customers to participate, incentives were offered for the following upgrades; T-12 to T-8 or T-5 lamps and or LED fixtures, upgrading HID to high efficiency T-8, or T-5 s, installation of sensors, LED exit lighting, incandescent lighting to CFL s or the upgrade of parking lot lighting. The program also encourages new construction to upgrade their lighting above minimum standards and requirements. Incentives were based on kwh reduced on the structure. OG&E personnel, along with CLEAResult representatives, continued to recruit and educate commercial customers on the advantages of upgrading their lighting systems through educational seminars and booth displays at local vendor open houses and lunch and learn opportunities. OG&E personnel utilized many different avenues and strategies to encourage customers to upgrade the lighting in each facility. This includes working with lighting manufacture representatives, conducting walk through audits and detailed audits. The program was very well received with new incentives allowing for quicker reimbursement on enhanced lighting levels in facilities. More of the commercial customers took advantage of the rebate while educating themselves on the benefits of more efficient lighting and lighting controls Program Highlights Presentations were made at lighting supply and distributor warehouses throughout Civic and community presentations promoting the lighting program were conducted throughout each town served by OG&E. CLEAResult performed training on LED performance in the last quarter of 2015, which was very well attended by customers and industry representatives. CLEAResult continued to assist OG&E personnel throughout 2015 in capturing opportunities for lighting replacements with all classifications of C&I consumers. A C&I specific address was created on OG&E s website to receive Program inquiries from customers and contractors. Post installation inspections were performed to ensure the program was implemented as designed and proper documentation was collected. Due to the overwhelming success of the program in 2015, the budget for incentives was reserved and savings goals were projected to be met by the 3 rd quarter. 17

18 The Commercial Lighting program achieved 99% of the energy goal in The average savings per participant increased from prior years Program Budget, Savings and Participants Table 11- Commercial Lighting Program Summary Commercial Lighting Cost Energy Savings (kwh) Demand Savings (kw) Participants Program Budget Actual % Plan Evaluated % Plan Evaluated % Plan Actual % Program Year 2013 $ 514,899 $ 387,722 75% 9,010,145 6,325,111 70% 2, % % Program Year 2014 $ 900,128 $ 958, % 5,162,810 6,525, % 970 1, % % Program Year 2015 $ 1,561,360 $ 1,509,288 97% 6,599,411 6,529,402 99% 976 1, % % Description of Participants Participants in the program included all classifications of C&I customers Challenges and Opportunities Keeping the distributors and contractors updated on the latest program changes, codes and TRM standards. Economic conditions in the Fort Smith market place have started to rebound slowly due to economic conditions in the oil industry. Many commercial lighting customers continue to delay lighting projects due to corporate budget limitations. Market acceptance of LEDs has grown as incremental costs continue to decrease Planned or Proposed Changes to Program and Budget OG&E will discontinue using its online calculator for lighting measures and will continue to use CLEAResult for all calculations in

19 2.4 Energy Efficiency Arkansas Program Program Description The Energy Efficiency Arkansas ( EEA ) program provides information to all customers, of all classes, allowing them to make informed decisions about how they use energy and to look at alternatives to improve their consumption, thereby decreasing demand and energy usage. OG&E has continued its support of the EEA Comprehensive plan, provided by the Arkansas Energy Office ( AEO ), through three components: 1) residential education and information outreach; 2) media promotion; 3) commercial and industrial education and outreach. The AEO provided educational pamphlets, DVDs, and training materials to homeowners throughout the OG&E service territory. Multiple classes were held throughout the State of Arkansas on residential, commercial, and industrial energy efficient usage and design. Area industry plant engineers as well as CEOs, CFOs, and purchasing agents were updated on techniques of how to manage energy consumption in their plants. Courses on refrigeration and compressed-air were held in the Fort Smith area to update individual businesses on EE operations within the C&I segment Program Highlights The AEO provided various methods of reaching all classifications of OG&E customers through radio, print, and seminars. The AEO offered training through Arkansas Manufacturing Solutions throughout the year in the OG&E service territory. Additional information is submitted by the AEO annual report. The comprehensive program began February 3, 2010 and ended on June 30, The EEA program began on July 1, 2011 and continues through December Program Budget, Savings and Participants Table 12- Energy Efficiency Arkansas Program Summary Energy Efficiency Arkansas (EEA) Cost Energy Savings (kwh) Demand Savings (kw) Participants Program Budget Actual % Plan Evaluated % Plan Evaluated % Plan Actual % Program Year 2013 $ 24,000 $ 18,659 78% Program Year 2014 $ 21,600 $ 18,319 85% Program Year 2015 $ 24,000 $ 18,319 76%

20 2.4.4 Description of Participants Residential and C&I customers in Arkansas Challenges and Opportunities OG&E, along with the AEO, has continued to provide updated material to all classifications of consumers throughout the OG&E service territory. Cost effective measures should be implemented in a timely manner to lower utility costs. Education to the customer is essential in stressing the importance of EE in all applications. The AEO collaborated with the PWC to develop a central website for C&I customers as part of the AR Common C&I Approach for All Investor Owned Utilities in Arkansas Planned or Proposed Changes to Program and Budget No changes are planned or proposed to the program budget for OG&E is planning to continue its support for the EEA program in the portfolio cycle. 20

21 2.5 Commercial and Industrial Standard Offer Program Program Description This is a comprehensive long term EE program targeted to C&I Power and Light rate customers. The program provides incentives for the energy savings produced through EE improvements and solutions to meet requirements unique to each facility. The program has proven to be successful in helping to not only manage, but to assist in upgrading existing equipment to a higher efficiency standard. This program has an on-going opportunity to help industrial customers achieve higher efficiency standards while providing incentives to help shorten payback periods. OG&E personnel, along with CLEAResult representatives call on individual C&I customers as well as trade allies to promote the program and facilitate the completion of eligible projects Program Highlights Customers embraced the program by upgrading to higher efficiency equipment. OG&E promoted the program through various civic presentations across OG&E s service territory. OG&E contracted with CLEAResult to assist OG&E personnel in the C&I SOP program delivery. OG&E increased program comprehensiveness with the addition of new measures such as refrigeration door gaskets, strip curtains, and personal computer power management. OG&E successfully implemented direct install measures to increase measure comprehensiveness and cost-effectiveness. The C&I SOP program achieved 103% of the energy goal in The average savings per participant increased from prior years Program Budget, Savings and Participants Table 13- C&I Standard Offer Program Summary C&I Standard Offer Cost Energy Savings (kwh) Demand Savings (kw) Participants Program Budget Actual % Plan Evaluated % Plan Evaluated % Plan Actual % Program Year 2013 $ 537,940 $ 523,591 97% 7,177,710 2,535,967 35% 1, % 18 2, % Program Year 2014 $ 926,250 $ 949, % 3,596,963 1,606,746 45% % % Program Year 2015 $ 1,572,241 $ 1,572, % 6,541,238 6,717, % 1, % % 21

22 2.5.4 Description of Participants Participants in the program included each of the classifications for C&I customers Challenges and Opportunities EE improvements with many industrial customers continue to move at a slow pace due to the customer s budget limitations on capital improvements. Because distributors tend to keep little to no inventory on higher efficiency equipment, in-stock minimum efficiency equipment is often purchased in emergency situations. Many large projects have lead times of up to 18 months or longer from start to finish, which presents a challenge in trying to manage program budgets. Businesses have shown increasing interest in the direct install component. Customers who have successfully completed projects are willing to explore deeper energy savings with other technologies Planned or Proposed Changes to Program and Budget OG&E will discontinue using its online calculator for SOP measures and will continue to use CLEAResult for all calculations. 22

23 2.6 Arkansas Weatherization Program (AWP) Program Description The Arkansas Weatherization Program ( AWP ) was designed to promote EE in homes throughout the OG&E service territory in Arkansas. This program was coordinated by the Arkansas Community Action Agencies Association ( ACAAA ) and was transferred to the Central Arkansas Development Council ( CADC ). The program is targeted to residential customers and allows the customer to participate in programs to assist in managing energy costs and to begin to utilize price response tariffs. This program focuses on customer owned homes that are severely energy inefficient. The program design is to upgrade and improve the thermal envelope of the dwelling and the energy use of appliances Program Highlights The Energy Efficiency Arkansas Weatherization Program was launched on July 1, AWP weatherized 0 homes in 2015 in OG&E s service territory Program Budget, Savings and Participants Table 14- Arkansas Weatherization Program Summary Arkansas Weatherization Program (AWP) Cost Energy Savings (kwh) Demand Savings (kw) Participants Program Budget Actual % Plan Evaluated % Plan Evaluated % Plan Actual % Program Year 2013 $ 85,730 $ 38,714 45% 522,485 54,516 10% % % Program Year 2014 $ 77,157 $ 22,914 30% 100,821 3,141 3% % % Program Year 2015 $ 15,730 $ 5,294 34% 100, % % % Description of Participants This program looks for customers with severely energy inefficient homes and limited disposable income for EE measures. Customers may qualify for federal funds based on income. There were no participants in

24 2.6.5 Challenges & Opportunities The program has been discontinued; however, these customers are eligible to participate in the OGE-AOG Weatherization program as it has been modified to reflect the statewide Uniform Weatherization Approach in Planned or Proposed Changes to Program & Budget This program was discontinued effective January 1,

25 2.7 Multi-Family Direct Install Program Program Description The Multi-Family Direct Install ( MFDI ) program is intended to target multi-family complex owners and/or managers who seek assistance in improving the efficiency of individual units in their complex. The program provides energy saving measures for residential customers living in multi-family housing at no cost to the customer. Replacement measures include, but are not limited to, CFL s, advanced power strips ( APS ), low-flow showerheads, faucet aerators, duct sealing, and air sealing Program Highlights The MFDI program was approved for implementation by this Commission in March of This program was part of an interim filing by OG&E to modify its EE portfolio to ensure it meets energy savings targets. The program reached 1,795 participants in the OG&E service territory and achieved 139% of the energy goal. The program was able to follow up with properties that participated in 2014 and offer additional measures, such as duct sealing, air sealing, and advanced power strips. The program sourced an APS that mounted to the wall with a security screw preventing removal of the measure by the tenant. This wall mounted APS allowed the measure to be provided to more properties and have a greater uptake. Outreach focused on property management companies with multiple properties which maximized opportunities and allowed the program to work directly with decision makers. The program partnered with University of Arkansas-Fort Smith to complete an on-campus apartment-style housing project in which lighting, APS, and water saving measures were installed. The program upgraded several Housing and Urban Development and Section 8 properties, reaching a large number of low income customers. Quality assurance processes were expanded to include live, in-field inspections, as well as postinstallation inspections. The program focused on stakeholder communication throughout all stages of the project which mitigated property management and tenant concerns, maintained high quality installation standards, and increased customer satisfaction. 25

26 2.7.3 Program Budget, Savings and Number of Measures Table 15- Multi-Family Direct Install Program Summary Multi-Family Direct Install Cost Energy Savings (kwh) Demand Savings (kw) Participants Program Budget Actual % Plan Evaluated % Plan Evaluated % Plan Actual % Program Year 2013 $ - $ Program Year 2014 $ 268,893 $ 233,411 87% 1,914,153 1,667,071 87% % 2,050 1,884 92% Program Year 2015 $ 773,019 $ 652,289 84% 2,851,734 3,969, % % 2,832 1,795 63% Description of Participants The participants for the MFDI program are customers living in apartment complexes or other multi-family units and typically rent rather than own their housing. This arrangement requires OG&E to receive permission from the owner of the properties before EE measures are installed. Because of this arrangement, multi-family customers can be considered hard-to-reach when providing education and opportunities for managing energy use Challenges and Opportunities OG&E continues to seek ways to reach this target market where the decision maker for enrolling in an EE program isn t the customer paying the electric bill Planned or Proposed Changes to Program and Budget Due to the success the program has already achieved, there will be no changes in

27 3.0 Supplemental Requirements 3.1 Staffing OG&E has a total of 3 Full Time Employees ( FTEs ); 2 FTEs managing its EE programs, an EM&V Specialist supporting the evaluation of programs, and an Administrative Clerk make up the remaining FTE. The EM&V Specialist and the clerical support have additional responsibilities in OG&E s Oklahoma EE programs as well. New staffing in 2015 included one additional analyst in OG&E s Arkansas service territory that is funded through base rates. The addition of the analyst offers more opportunity to track, monitor, and evaluate the success of the current programs, as well as plan for future program success. 3.2 Stakeholders Activities Training classes fall into three categories. First, training for trade allies who are working with OG&E on the implementation of the program. Their purpose is to educate the installers of the components and operations of the program. Second, training classes for C&I customers provide information on both OG&E programs and educate customers on the benefits of how EE products and processes can assist their energy management efforts. Third, classes are held to train OG&E members, installers, and customers on the technical issues to improve energy efficiencies for customers. Please see training activity included in the annual workbook tables. OG&E participated with the PWC in numerous meetings both in person and over the phone or through webinars. PWC meetings included discussions on TRM updates, Non-Energy Benefits, the Potential Study, Net-to-Gross Studies, a Common C&I Approach, and potential financing opportunities for customers. 3.3 Information provided to Customer to Promote EE Please see Section 5.0 Appendix B for samples of promotional and educational materials used in the program year. 27

28 4.0 Appendix A: EM&V Contractor Reports 28

29 EM&V Contractor Reports OG&E has three different EM&V Reports associated with evaluated savings for PY ADM & Associates provided results for both the AWP Program and OG&E s Weatherization Program while AEG provided results for the remaining programs. Frontier Associates LLC provided the cost benefit analysis and report for PY OG&E is providing these reports in the attached exhibits. Attachments: Attachment A) contains Frontier s Cost Benefit Analysis Attachment B) contains ADM s evaluation of the AWP Program. Attachment C) contains ADM s evaluation of OG&E/AOG s Weatherization Program. Attachment D) contains AEG s evaluation of the remaining programs. 29

30 Attachment A: Frontier s Cost Benefit Analysis

31 Annual Report of Energy Efficiency Programs Program Year 2015 March 2016 Evaluation Conducted by: Frontier Associates

32 Table of Contents TABLE OF CONTENTS... 2 INTRODUCTION... 3 REPORT ORGANIZATION... 3 PROGRAM DESCRIPTIONS... 3 PROGRAM PROJECTIONS AND RESULTS... 5 PROGRAM-RELATED EXPENDITURES... 10

33 Introduction Oklahoma Gas & Electric s Arkansas Energy Efficiency Plan for PY 2015 was approved by the Arkansas Public Service Commission (APSC) on March 2, 2015 in Docket Number TF, Order No. 55. As required by the Conservation and Energy Efficiency Rules, OG&E is submitting its annual report addressing the performance of all approved energy efficiency programs. This report covers program savings and the amount spent per program and total amount spent. It also includes a cost-effectiveness analysis of each program and the portfolio of programs, including all costs and benefits from January 1, 2015 through December 31, Report Organization This report presents the following information, which is based on the Commission s Energy Efficiency Rule, but also includes the results of California Standard Practice Manual cost-benefit tests: 1. Brief description of each program; 2. The most current information available comparing projected savings to reported savings for each of the utility's programs; 3. The results of the standard cost/benefit tests for each program; 4. A statement of funds expended by the utility for program administration. Program Descriptions Student Energy Education (SEE): The Student Energy Education program is an established residential energy efficiency program that uses a school delivery format, in which students are provided with take-home kits containing efficiency devices and are exposed to creative classroom and in-home education techniques which inspire families to adopt new resource usage habits. Students receive a kit of energy and water efficient devices, which are taken home and installed, and the learning experience is shared with family members. They work on subjects required by state learning standards to understand and appreciate the value of natural resources in everyday life. The program aims to shape new behaviors and encourage reduced energy use through a mix of new product installation and resource efficiency knowledge. In OG&E S Arkansas service territory, the program provides the teachers and their classes of 6th grade students a curriculum on home energy efficiency. At the end of the curriculum a SEE education kit, (which includes three CFLs, two faucet aerators, one low-flow shower head, an LED night light, a thermometer, and student handbook on energy efficiency for the home and community), provides the students the opportunity to participate with their families on energy awareness. The students take the kit home and install the energy efficiency measures with the assistance of their parents. 3 OG&E Arkansas Energy Efficiency Report Program Year 2015

34 SEE is a turnkey program managed by Resource Action Programs (RAP) of Modesto, California. In coordination with OG&E, Resource Action Programs performs the marketing and outreach to acquire participation and enrollment in the program. Once schools are enrolled into the program, Resource Action Programs will deliver educational materials directly to participant teachers. Multi-Family Direct Install Program: The Multi-Family Direct Install program is designed to reach residential customers living in multi-family housing. The program provides energy saving fixtures and installation at no cost to the customer. Replacement fixtures include, but are not limited to, Compact Fluorescent Lights (CFLs), low flow shower heads, faucet aerators, air infiltration, duct sealing and Advanced Power Strip (APS) measures. Arkansas Weatherization Program (AWP): This program is implemented by the CADC and targets severely energy inefficient homes. It provides energy efficiency improvements to participants, thereby decreasing demand and energy usage for those customers. The purpose of the AWP is to improve comfort and reduce energy costs by upgrading the thermal envelope and appliances in severely energy inefficient homes. The AWP program is designed to work in partnership with agencies that assist residents occupying severely energy inefficient homes. OG&E partners with the CAP Agencies in Fort Smith, Arkansas. The program helps individuals and families primarily by making their homes more secure from the weather, which helps to conserve energy and reduce energy bills for future years. In addition, homes that are warm in the winter and cool in the summer are more comfortable for individuals. The AWP did not weatherize any homes in OG&E s Arkansas service territory in This program was discontinued effective January 1, OG&E Weatherization Program: This program targets energy inefficient homes no matter the severity. It provides energy efficiency improvements to participants, thereby decreasing demand and energy usage for those customers. The purpose of OG&E s Weatherization Program is to improve comfort and reduce energy costs by upgrading the thermal envelope and appliances in targeted households. This program is delivered in association with the Ft. Smith region gas distribution company, Arkansas Oklahoma Gas (AOG). AOG is contributing resources to be used alongside OG&E s on a per household basis to ensure the most effective application of energy efficiency possible. This program has been modified to be better aligned with the statewide Unified Weatherization Approach in Commercial Lighting Program: The Commercial Lighting Program provides prescriptive rebates for customers that improve the efficiency of lighting systems in existing buildings. This measure is designed to educate, offer performance contracting services, and provide incentives on replacement of inefficient T-12 or T-8 lamps with higher efficiency T-8 or T-5 lamps to commercial and industrial customers. It also promotes replacing less efficient high intensity discharge (HID) lighting with high-bay and low-bay fluorescent lamps, replacing inefficient incandescent lighting with hardwired CFLs, and replacing incandescent exit lighting with LED exit lighting. Additionally, this program provides incentives to OG&E construction commercial and industrial (C & I) customers who purchase and install energy efficient indoor and outdoor lighting, lighting controls, 4 OG&E Arkansas Energy Efficiency Report Program Year 2015

35 occupancy sensors, light emitting diode (LED) fixtures, and exit lights. The measure offers incentives based on the kw and kwh reduction calculated from a lighting survey by a lighting contractor that takes into account the type and quantity of lighting fixtures installed, the building type, and control technologies in place. Commercial and Industrial Standard Offer Program (SOP): The SOP offers financial incentives for the installation of a wide range of measures that reduce customer energy costs, reduce peak demand, and/or save energy in non-residential facilities such as public authority buildings, schools, hospitals, and other industrial customers in OG&E s Arkansas jurisdiction (entities that qualify for the Power and Light rate or the Large power and Light rate). In this program, large individual customers, energy service companies (ESCOs), and qualified contractors are eligible for incentive payments for energy efficiency projects that significantly reduce customer peak demand. The applying entity, whether the customer, ESCO, or other contractor, is a Project Sponsor, and is the responsible party for complying with all program requirements. The SOP also has a small business direct install component for all other non-residential customers, which allows for incentivizing of many measures not covered under other OG&E programs. Program Projections and Results The following tables present program specific information, including forecasted savings, reported savings, the number of participants, participant costs, the economic benefit realized in 2015, and the economic benefits to be expected over the life of the measures. Note that economic benefits are restricted to avoided electricity generation and capacity costs and avoided natural gas costs. Note also the important distinction between the Forecasted Net Savings displayed in this section and the Ex Ante savings stated as Evaluated Savings. The Forecasted Net Savings are the net savings included in OG&E s Direct Testimony and Exhibits of Billy Dean Pollock in Docket No TF. The Ex Ante savings reflect the savings calculated using actual participation data and the deemed savings used to develop the forecasted savings and in continuous tracking of program savings. Assumptions for energy and demand savings, measure lives, and Net-to-Gross Ratios are from the EMV savings analysis for PY Measure cost assumptions are sourced to any cost data captured by OG&E s databases and supplemented by measure cost research performed by Frontier Associates. Discount rates, fuel costs and other inputs are sourced to OG&E Arkansas Energy Efficiency Program Analysis and Plan for PY , filed in Docket Number TF (William L. Brooks testimony, Exhibit WLB-01). The line losses are sourced to an OG&E Line Loss Study conducted in The modeling of the Forecasted Net Savings and Project Net Savings are based on the following assumptions: (a) Forecasted savings are based on the target participation levels for PY 2015 as approved by the APSC in Order No. 59 of Docket No TF on March 2, (b) Program participants are those who participated in the program year (c) The cost per kwh saved is calculated by dividing the total program costs by the lifetime energy saved. The cost per kw-year is calculated by dividing the total program costs by the product of the kw reduction and the approximate average effective useful life (EUL) of measures installed in the program. 5 OG&E Arkansas Energy Efficiency Report Program Year 2015

36 (d) The net present value of the total economic benefits was calculated by taking the discounted value of the annual avoided cost times the annual savings over the useful life of each program measure. (e) The Projected Net Savings for residential programs assume an energy rate based on Rate Arkansas Rate Tariff R-1. Commercial energy rates are assumed to be $.09/kWh for all seasons. Commercial load rates are accounted for in this assumption. The energy rates escalation rates are derived from the avoided costs. The Forecasted Net Savings and Evaluated Savings are presented in Table 1. The Evaluated Savings include electric line losses. 1 Table 1 - Forecasted Net Savings vs. Evaluated Savings 2 Forecasted Net Savings (2015) Evaluated Savings (2015) Program Annual Energy Savings (kwh) Peak Demand Reduction (kw) Participants Annual Energy Savings (kwh) Peak Demand Reduction (kw) Participants 3 SEE Program 288, , , ,919 Multi-Family Direct Install 2,851, ,832 4,342, ,893 AWP 100, AOG-OG&E Weatherization Program 3,497, ,620 3,282,470 1,035 1,325 Commercial Lighting 6,599, ,142,985 1, Commercial and Industrial SOP 6,541,238 1, ,348, TOTAL 19,879,081 3,407 6,778 22,474,437 3,406 10,586 The results of the Total Resource Cost Test show $17,551,288 in present value net benefits for all of 2015, as illustrated in Table 2. Of these benefits, $10,974,485 can be attributed to commercial programs and $6,576,803 is associated with residential programs. 1 Electric line losses used for this cost-effectiveness analysis are 8.59%. This percentage comes from OG&E s 2012 Line Loss study For the SEE, MF Direct Install, Commercial Lighting and C&I SOP, the participant numbers are sourced to the OG&E AR PY 2015 Final EM&V Report February 29, 2016 by AEG. For the SEE program, the number of participants are defined as the number of kits. For MF Direct Install, Commercial Lighting, and the Commercial SOP, the number of participants are defined as the number of projects. For the AOG-OG&E Weatherization Program, the number of participants are sourced the Evaluation of 2015 AOG-OG&E Weatherization Program February 2016 by ADM. AOG-OG&E Weatherization Program participant numbers are defined as the number of homes. 6 OG&E Arkansas Energy Efficiency Report Program Year 2015

37 Table 2 Energy Efficiency Program Total Resource Cost Test Net Benefits Program Name TRC Net Benefits ($000s) Lifetime Energy Savings (kwh) SEE Program ,449,053 Multi-Family Direct Install 1, ,035,073 AWP n/a n/a OG&E Weatherization 1, ,368,968 Commercial Lighting 2, ,858,803 Commercial SOP 4, ,606,988 ALL RESIDENTIAL 3, ,853,094 ALL COMMERCIAL 7, ,465,791 TOTAL 10, ,318,885 Table 3 shows the cumulative results of OG&E s residential energy efficiency programs cost-effectiveness portfolio. The five cost tests deliver a snapshot of the general benefit of the residential energy efficiency programs. The TRC, being above 1, indicates that the residential programs produce an aggregate benefit. Table 3 - ALL Residential Cost/Benefit Tests PCT UCT RIM TRC SCT Benefit/Cost Ratio Net Benefits ($000s) 7, , , , , Total Benefits ($000s) 9, , , , , Total Costs ($000s) 2, , , , , Tables 4 through Table 7 individually show the results of OG&E s residential energy efficiency programs cost-effective portfolio. Table 4 - SEE Program Cost/Benefit Tests PCT UCT RIM TRC SCT Benefit/Cost Ratio Net Benefits ($000s) Total Benefits ($000s) Total Costs ($000s) OG&E Arkansas Energy Efficiency Report Program Year 2015

38 Table 5: Multi-Family Direct Install Cost/Benefit Tests PCT UCT RIM TRC SCT Benefit/Cost Ratio Net Benefits ($000s) 3, , , , , Total Benefits ($000s) 4, , , , , Total Costs ($000s) , Table 6 - AWP Cost/Benefit Tests PCT UCT RIM TRC SCT Benefit/Cost Ratio n/a n/a n/a n/a n/a Net Benefits ($000s) n/a n/a n/a n/a n/a Total Benefits ($000s) n/a n/a n/a n/a n/a Total Costs ($000s) n/a n/a n/a n/a n/a Table 7 - OG&E Weatherization Program Cost/Benefit Tests PCT UCT RIM TRC SCT Benefit/Cost Ratio Net Benefits ($000s) 3, , , , , Total Benefits ($000s) 5, , , , , Total Costs ($000s) 1, , , , , Table 8 Table 8 - All Commercial Cost/Benefit Tests PCT UCT RIM TRC SCT Benefit/Cost Ratio Net Benefits ($000s) 10, , , , , Total Benefits ($000s) 13, , , , , Total Costs ($000s) 2, , , , , OG&E Arkansas Energy Efficiency Report Program Year 2015

39 Table 9 - Commercial Lighting Cost/Benefit Tests PCT UCT RIM TRC SCT Benefit/Cost Ratio Net Benefits ($000s) 4, , , , Total Benefits ($000s) 5, , , , , Total Costs ($000s) 1, , , , , Table 10 - Commercial and Industrial Standard Offer Program Cost/Benefit Tests PCT UCT RIM TRC SCT Benefit/Cost Ratio Net Benefits ($000s) 6, , , , , Total Benefits ($000s) 7, , , , , Total Costs ($000s) , , , , Table 11 shows the cumulative cost-effectiveness results for OG&E s energy efficiency portfolio for program year Table 11 - Portfolio Cost/Benefit Tests PCT UCT RIM TRC SCT Benefit/Cost Ratio Net Benefits ($000s) 18, , , , , Total Benefits ($000s) 22, , , , , Total Costs ($000s) 4, , , , , OG&E Arkansas Energy Efficiency Report Program Year 2015

40 Program-Related Expenditures All program-related expenditures are presented in Table 12 and are separated by administrative costs and inducements. The administrative costs include program planning and design, marketing and delivery, EM&V, and third party implementation costs. Table 12 Program Costs Program Name Administrative and Other Non-Inducement Costs ($) Inducements ($) Total Program Cost ($) SEE Program $9,627 $76,781 $86,408 Multi-Family Direct Install $298,622 $373,139 $671,761 AWP $4,340 $994 $5,334 OG&E Weatherization $303,925 $1,911,946 $2,215,872 Commercial Lighting $699,912 $828,279 $1,528,191 Commercial SOP $659,660 $938,043 $1,597,703 Energy Efficiency Arkansas $18,319 $0 $18,319 Regulatory Costs $50,015 $0 $50,015 TOTAL $2,044,421 $4,129,181 $6,173,603 Planned and actual program costs, including additional regulatory costs and costs associated with the Energy Efficiency Arkansas Program, are compared in Table 13. Table 13 - Planned and Actual Program Costs Program Name Planned Program Cost ($) Actual Program Cost ($) SEE Program $88,315 $86,408 Multi Family Direct Install $773,019 $671,761 AWP $85,730 $5,334 OG&E Weatherization $2,281,220 $2,215,872 Commercial Lighting $1,564,378 $1,528,191 Commercial SOP $1,569,223 $1,597,703 Energy Efficiency Arkansas $24,000 $18,319 Regulatory Costs $85,000 $50,015 TOTAL $6,470,885 $6,173, OG&E Arkansas Energy Efficiency Report Program Year 2015

41 Attachment B: ADM s Evaluation of the AWP Program

42 Evaluation of 2015 Arkansas Weatherization Program Submitted to: Central Arkansas Development Council Arkansas Oklahoma Gas Corporation CenterPoint Energy Arkansas Oklahoma Gas and Electric Southwestern Electric Power Company Empire District Electric Company Entergy Arkansas, Inc. Black Hills Energy March 2016 Prepared by: ADM Associates, Inc.

43 Prepared by: Brian Harold Adam Thomas Kevin Halverson Corporate Headquarters: 3239 Ramos Circle Sacramento, CA Tel: (916) ADM Associates Inc. Energy Research & Evaluation 200 Brown Road Suite 208 Fremont, CA Tel: (510)

44 Acknowledgements We would like to thank the staff at each of the Arkansas Weatherization Program sponsoring utilities, including Arkansas Oklahoma Gas Corporation, CenterPoint Energy Arkansas, Oklahoma Gas and Electric, Southwestern Electric Power Company, Empire District Electric Company, Entergy Arkansas, Inc., and Black Hills Energy for the time and effort they have contributed to the evaluation, measurement, and verification (EM&V) of the Arkansas Weatherization Program. This evaluation required accessible communications with staff at each utility, who actively responded to evaluation inquiries and requests. Additionally, we would like to thank participating customers of the aforementioned utilities, the Central Arkansas Development Council (CADC) and participating community action agencies, and Frontier Associates staff for their cooperation and assistance throughout the evaluation. We would also like to thank Independent Evaluation Monitor staff for their active involvement in providing thorough answers and clarification to the evaluation team when higher-level questions arose during the EM&V effort.

45 TABLE OF CONTENTS Section Title Page 1. Executive Summary Impact Evaluation Findings Process Evaluation Findings Conclusions and Recommendations Appendix A: Participant Survey Instrument... A-1 i

46 LIST OF TABLES Title Page Table 1-1 Key Activities and Program Stages, 2015 Program Year Table 1-2 TRM Sections by Measure Type Table 1-3 Ex Post Net Savings by Electric Utility Table 1-4 Ex Post Net Savings by Gas Utility Table 1-5 Ex Post Net Savings by Measure Type Overall Table 1-6 Gas and Electric Realization Rates by Measure Type Table 1-7 Status of Recommendations from 2014 Program Year Table 1-8 Recommendations from 2015 Program Year Evaluation Table 2-1 Ex Ante Savings by Measure Type Overall Table 2-2 Ex Ante Savings by Electric Utility Table 2-3 Ex Ante Savings by Measure Type AEP-SWEPCO Table 2-4 Ex Ante Savings by Measure Type - EAI Table 2-5 Ex Ante Savings by Measure Type Non-IOU (Electric) Table 2-6 Ex Ante Savings by Gas Utility Table 2-7 Ex Ante Savings by Measure Type CenterPoint Table 2-8 Ex Ante Savings Values by Measure Type Non-IOU (Gas) Table 2-9 TRM Sections by Measure Type Table 2-10 Deemed Savings Values for Air Infiltration Reduction, Zone Table 2-11 Deemed Savings Values for Ceiling Insulation, Zone Table 2-12 Gas and Electric Realization Rates by Measure Type Table 2-13 Ex Post Net Savings by Measure Type Overall Table 2-14 Ex Post Net Savings by Electric Utility Table 2-15 Ex Post Net Savings by Measure Type AEP SWEPCO Table 2-16 Ex Post Net Savings by Measure Type EAI Table 2-17 Ex Post Net Savings by Measure Type Non-IOU (Electric) Table 2-18 Ex Post Net Savings by Gas Utility Table 2-19 Ex Post Net Savings by Measure Type CenterPoint Table 2-20 Ex Post Net Savings by Measure Type Non-IOU (Gas) Table 3-1 Determining Process Evaluation Timing Table 3-2 Determining Process Evaluation Conditions ii

47 Table 3-3 Interview and Survey Data Collection Summary Table 3-4 Status of Recommendations from 2014 Program Year Table 3-5 Key Activities and Program Stages, 2015 Program Year Table 3-6 Total Participation by Community Action Agency Table 3-7 Participation by Associated Utility, Table 3-8 How Participants Learned of the Program Table 3-9 Reasons for Participation Table 3-10 Participants Past Familiarity with Energy Efficiency Improvements Table 3-11 Participants Familiarity with Energy Savings Activities Table 3-12 Increase in Energy Efficiency Knowledge Following AWP Table 3-13 Participants Prior Energy Saving Activities Table 3-14 Participants Current Energy Saving Activities Table 3-15 Participant Satisfaction with Selected Program Elements Table 3-16 Home Construction Dates Table 3-17 Approximate Square Footages of Participant Homes Table 3-18 Number of Bedrooms in Participant Homes Table 3-19 Number of Bathrooms in Participant Homes Table 3-20 Number of Showers in Participant Homes Table 3-21 Number of Residents in Home Year-Round Table 4-1 Recommendations from 2015 Program Year Evaluation iii

48 LIST OF FIGURES Title Page Figure 3-1 Participation Rates by Month, 2015 vs Figure 3-2 Types of Heating Systems in Participant Homes Figure 3-3 Types of Water Heaters in Participant Homes iv

49 1. Executive Summary The purpose of this report is to provide a summary of the methodology and results for the evaluation of the 2015 Arkansas Weatherization Program (AWP). This evaluation was conducted by ADM Associates (referred to in this report as the Evaluators ). This report provides the results of both the impact evaluation and limited process evaluation activities for 2015, the final year of program operation. As there have been few significant modifications to overall program structure and delivery since the prior program year, and this is the final year of operation for the AWP, the process findings are mainly focused on assessing program performance characteristics, any changes in program delivery, and the program s responsiveness to prior evaluation recommendations. A comprehensive process evaluation can be found in the 2012 Arkansas Weatherization Program Evaluation Report. 1.1 Summary of Arkansas Weatherization Program Program design and structure in 2015 remained fairly consistent with the 2014 program year. The following provides a review of program design characteristics and operational procedures, noting any specific updates for The primary change for the 2015 program year was that administration of the program, including coordination of implementation activity and allocation of funding to participating agencies, transitioned to the Central Arkansas Development Council (CADC) from the Arkansas Community Action Agencies Association (ACAAA). As CADC had already been closely involved in program implementation and coordination of agencies in prior years, the effects of this transition were fairly minimal. In 2015, the Arkansas Weatherization Program (AWP) provided residential energy audits and energy efficiency measure installations to homes whose residents are customers of one or more of the following investor owned utilities (IOUs): American Electric Power Southwestern Electric Power Company (AEP- SWEPCO); Entergy Arkansas, Inc. (EAI); and CenterPoint Energy (CenterPoint). The following IOUs are sponsoring utilities for the AWP and have achieved savings through the program in past years but did not have any customers participate in the program during 2015: Empire District Electric Company (EDEC). Oklahoma Gas and Electric (OG&E); Executive Summary 1-1

50 2015 Arkansas Weatherization Program EM&V Report Black Hills Energy; 1 and Arkansas Oklahoma Gas Corporation (AOG). The program is offered in conjunction with the Department of Energy (DOE) Weatherization Assistance Program (WAP), which provides federal assistance to fund the customer co-payment in the AWP for income-qualified households. In Arkansas, the WAP is administered by the Arkansas Energy Office (AEO). If the customer meets the eligibility requirements of the WAP, the weatherization project can be funded by both the WAP and the AWP in order to fully cover the project cost and eliminate the cost to the customer. 2 Customers who are not eligible for the WAP are required to provide their own co-pay in order to participate in the AWP and receive the audit and associated measures. Rather than an income requirement, eligibility for the AWP is based on a set of criteria regarding customer residence energy efficiency. In order to qualify, customer homes must meet specific criteria indicating that the residence is severely energy-inefficient. There were no modifications to these criteria for the 2015 program year. Local community action agencies work with customers to enroll in the program and determine AWP and WAP eligibility. In 2015, qualifying AWP projects were completed by the following agencies: Central Arkansas Development Council (CADC); Crowley s Ridge Development Council (CRDC); and Pine Bluff Jefferson County Economic Opportunities Commission, Inc. (PBJCEOC). After the customer is approved and the in-home audit is performed, optimal energy efficiency measures for AWP (and WAP, for eligible customers) are identified through the use of National Energy Audit Tool (NEAT) or Mobile Home Energy Audit (MHEA) software. The measures implemented in participating homes during 2015 include: Ceiling, floor and wall insulation; Air infiltration reduction; Window replacement and storm window installation; Heating and air conditioning replacement; 1 Formerly Source Gas Arkansas (SGA). 2 Eligibility for the Weatherization Assistance Program (WAP) is based on income thresholds, which increase with the number of residents in the home. A description of the WAP, along with the associated income requirements, can be found here: Executive Summary 1-2

51 2015 Arkansas Weatherization Program EM&V Report Water heater insulation jackets and pipe wrap; Refrigerator replacement; CFL retrofits; and Smart thermostats. 3 The local agencies conduct onsite audits and install the necessary measures using their internal crews or subcontractors. Audit and installation crews record all relevant measure input data and report it to the Central Arkansas Development Council (CADC), who aggregates the information from each agency. Batches of data are then sent to Frontier Associates, the program database provider that manages the EnerTrek software tool. EnerTrek incorporates the onsite data into TRM savings formulas (and NEAT/MHEA values for measures not included in the TRM) to calculate ex ante savings for each measure. The resulting savings are made accessible to program utilities and EM&V contractors, who use EnerTrek database exports to conduct measure implementation and savings verification activities. Table 1-1 identifies core program stages and includes key activities performed throughout the program process. The activities and stages shown for 2015 are fairly consistent with those of 2014 and prior years, with modifications to include additional details and clarifications regarding program procedures, and to reflect CADC s role as the program administrator. Table 1-1 Key Activities and Program Stages, 2015 Program Year Program Stage Program Design Planning Training and Implementation Planning Program Promotion Program Participation Key Activities Utilities set budgets and savings goals for the program year. Frontier Associates and the participating agencies make any necessary modifications to data collection procedures or program delivery based on TRM changes or other program design changes. Agencies plan their program activity based on expected WAP funding levels and planned AWP funding. Community action agencies, contractors, and other program operations staff attend program-relevant training sessions (primarily for new contractor staff) CADC and local agencies discuss implementation and program updates (primarily to comply with TRM changes). Agencies market the program to local customers who may provide a private co-pay. Agencies enroll customers from the WAP wait list. Utilities answer customer inquiries about the AWP or refer customers to their respective agencies. Customers apply for the AWP and home eligibility is determined. WAP eligibility is determined. 3 A complete list of all eligible program measures can be found in ACAAA Docket no TF, Attachment A (AWP Modified Program Design and Description). Executive Summary 1-3

52 2015 Arkansas Weatherization Program EM&V Report Program Stage Data Processing and Monitoring Key Activities Participants receive in-home audits and measures are identified. Contractors install measures that are either stipulated based on NEAT or MHEA software or are agreed upon with the customer (depending on whether or not WAP funds are used for the co-pay). Measure costs and participant tracking data are collected by each agency and reported to CADC. CADC provides periodic cost and participation updates to the utilities. Frontier Associates receives implementation data from CADC and calculates ex ante savings Frontier Associates sends savings data in batches to the utilities. Utilities, CADC, and Frontier Associates have periodic discussions regarding program participation levels and other topics. Starting in 2016, the Arkansas IOUs will be implementing individual and joint weatherization programs that comply with the Consistent Weatherization Approach developed by the Arkansas Parties Working Collaboratively (PWC). The Consistent Weatherization Approach will replace the AWP as the statewide weatherization offering implemented by the Arkansas IOUs, and thus the 2015 program year marks the final year of operation for the AWP. 1.2 Evaluation Objectives The evaluation of the 2015 Arkansas Weatherization Program (AWP) consisted of a program savings impact analysis and a limited process evaluation. These evaluation objectives were primarily focused on savings analysis and verification, as well as program updates and tracking of prior evaluation findings. Specifically, the evaluation activities conducted for the 2015 program year include: Review of deemed savings calculations: The Evaluators used the Arkansas Technical Reference Manual, Version 5.0 (TRM) to verify savings calculations for each implemented measure type in order to ensure that ex ante measure savings were properly calculated according to TRM protocols. 4 Tracking database and documentation review: The Evaluators conducted a comparative assessment of the AWP tracking database in order to evaluate tracking data modifications and improvements since the 2014 program year. On-site field verification: The Evaluators scheduled and conducted site visits to participant homes in order to verify complete and proper measure installation, to conduct post-implementation measurements, and to verify home characteristics such as heating and water heating fuel type. 4 Although EnerTrek calculated savings for the AWP in 2015 using protocols in TRM 4.0, the Evaluators referenced TRM 5.0 for verification purposes as it was the most current version of the TRM at the time of evaluation. Executive Summary 1-4

53 2015 Arkansas Weatherization Program EM&V Report Program staff interviews: Interviews were conducted with program staff. These interviews provided insight into any recent program changes for 2015, updates in specific program processes, and perspectives on closing out the AWP and moving forward with the Consistent Weatherization Approach. Participant surveys: Telephone surveys were conducted with a sample of program participants in order to collect data regarding customer satisfaction, participant characteristics, and to identify any issues with program operation or delivery from the customer perspective. 1.3 Summary of Findings Field Verification Results The Evaluators conducted onsite verification visits to 13 participant homes. These site visits were conducted in order to verify complete and proper measure installation, to conduct post-implementation measurements, and to verify home characteristics such as heating and water heating fuel type. Specific notes regarding the onsite and telephone verification findings include: Contact Information: All residences were located at the addresses provided within the tracking data. Additionally, a high percentage of telephone numbers were found to be accurate, which is an improvement over the 2014 program year. In total, only 4 out of the 54 telephone numbers (7%) dialed by the Evaluators were found to be disconnected or incorrect during the site visit scheduling process. Air Infiltration: The Evaluators conducted blower door testing in nine participant homes for the 2015 program year. Of these homes, the CFM50 value measured during the verification visit was within 10% of the reported value for four homes. The Evaluated CFM50 value was more than 10% greater than the Reported CFM50 value for four homes. Overall, the Evaluated CFM50 value was greater than the Reported CFM50 value for six of the nine homes. Attic Insulation: All reported instances of attic insulation were verified. There were no significant differences between reported pre-installation R values and evaluated pre-installation R values. There were no significant differences between reported square footage and evaluated square footage. Water Heater Jacket and Pipe Wrap: The Evaluators were able to verify all but one instance of water heater jackets. In this case, the Evaluators found that silver bubble wrap had been used to wrap the water heater, rather than standard insulation jackets. The agency responsible for this work reported that the silver bubble wrap is no longer being used. All reported instances of water heater pipe wrap were verified, but in at least two cases the pipe wrap was installed on the Executive Summary 1-5

54 2015 Arkansas Weatherization Program EM&V Report pressure release valve line. The agency responsible for this work reported that the water heater pipe wrap had been installed on the proper lines during the initial inspection of the home. Central Air Conditioner Replacement: All but one reported instance of central air conditioning replacement were verified. One customer did not appear to have had a new unit installed, and their existing unit was not functioning at the time of the site visit. CFLs: All reported instances of CFL installation were verified, in quantities that matched or closely matched program tracking data. All verified CFLs matched the wattage and lumen range reported in the EnerTrek tracking data. Gas Heat Replacement: All reported instances of gas heat replacement were verified. Window Replacement: All reported instances of window replacement were verified. Smart Thermostat: All reported instances of smart thermostat installation were verified. Refrigerator Replacement: All reported instances of refrigerator replacement were verified. Window Air Conditioner: All reported instances of window air conditioners were verified. Wall Insulation: All reported instances of wall insulation were verified. Direct Vent Heater: All reported instances of direct vent heater installation were verified. The results of the verification activity suggest that measures are for the most part installed in the quantities reported in program tracking data. However, the Evaluators found that the quality of work conducted in the two sampled homes serviced by PBJCEOC was fairly poor. This includes the instances of silver bubble wrap and pipe insulation on pressure relief valves mentioned above. Overall, issues identified within these homes include: It appeared that silver bubble wrap was used as water heater insulation, rather than standard insulation jackets; Plenum sealing was done with tape that did not effectively stick to surfaces; Caulk was used in gaps that were too large, where foam sealing should have been used; Water heater pipe insulation was added to the pressure release line; and One customer indicated that the contractor had damaged paint/trim in the home. Executive Summary 1-6

55 2015 Arkansas Weatherization Program EM&V Report These findings were submitted to CADC and PBJCEOC staff responded, indicating that the silver water heater wrap is no longer being used and that the water heater pipe insulation had been added to the proper lines when the home was inspected. However it may be beneficial to conduct further training with PBJCEOC staff in order to ensure that they are complying with industry quality standards, and that they are providing adequate customer service to participants Summary of Ex Post Net Savings For measures implemented through the 2015 program, savings verification was performed according to methodologies described in TRM V5.0. The following table identifies the sections in the TRM that were used for verification of measure-level savings under the AWP: 5 Table 1-2 TRM Sections by Measure Type Measure TRM Version Section in TRM Air Infiltration Ceiling Insulation Central AC Replacement ENERGY STAR Windows Floor Insulation Gas Furnace Replacement Heat Pump Replacement Lighting Efficiency ENERGY STAR Refrigerator Smart Thermostat Storm Windows 5.0 Appendix H Direct Vent Heaters Wall Insulation Water Heater Jackets Water Heater Pipe Insulation Window AC Table 1-3 and Table 1-4 present ex post net savings for electric utilities and gas utilities, respectively. Table 1-5 presents the ex post net savings by measure, including measurelevel realization rates (RR). The net-to-gross ratio for the AWP is 1, meaning that net savings are equal to gross savings. 5 The savings for storm windows were calculated through NEAT/MHEA, and these measures do not have an associated savings algorithm section in the TRM. However, Appendix H in TRM V5.0 specifies that NEAT/MHEA peak savings for storm windows should be multiplied by a deemed demand factor. This approach was used for the 2015 evaluation. Executive Summary 1-7

56 2015 Arkansas Weatherization Program EM&V Report Table 1-3 Ex Post Net Savings by Electric Utility Electric Utility # of Homes Peak Demand Savings (kw) Annual Savings (kwh) Lifetime Savings (kwh) AEP-SWEPCO , , EAI , ,146, OG&E Non-IOU , , Total , ,178, Table 1-4 Ex Post Net Savings by Gas Utility Gas Utility # of Homes Peak Demand Savings (Therms) Annual Savings (Therms) Lifetime Savings (Therms) AOG CenterPoint , , Black Hills Non-IOU , , Total , , Executive Summary 1-8

57 2015 Arkansas Weatherization Program EM&V Report Table 1-5 Ex Post Net Savings by Measure Type Overall Measure Peak Demand Savings (kw) Annual Savings (kwh) Lifetime Savings (kwh) Peak Demand Savings (Therms) Annual Savings (Therms) Lifetime Savings (Therms) Air Infiltration , ,033, , , Ceiling Insulation , ,324, , , Central AC Replacement , , Direct Vent Heater , , Duct Sealing Installation Floor Insulation , (0.09) Gas Central Replacement , Heat Pump Replacement , , Refrigerator Replacement , , Residential Lighting , , (0.31) (2.35) Smart Thermostat - 7, , , Storm Windows Wall Insulation , , Water Heater Insulation Water Heater Pipe Insulation , Window AC , Window Replacement , , , Total , ,178, , , Executive Summary 1-9

58 2015 Arkansas Weatherization Program EM&V Report Table 1-6 presents annual therms and kwh realization rates by measure category. These realization rates are presented at the program level, and individual utility realization rates may vary from those presented in this table. Table 1-6 Gas and Electric Realization Rates by Measure Type Measure kwh Realization Rate kw Realization Rate Therms Realization Rate Peak Therms Realization Rate Air Infiltration 100% 100% 100% 100% Ceiling Insulation 101% 149% 102% 101% Central AC Replacement 71% 161% - - Direct Vent Heater % 162% Duct Sealing Installation Floor Insulation 75% 68% 100% - Gas Central Replacement % 108% Heat Pump Replacement 49% 52% - - Refrigerator Replacement 178% 178% - - Residential Lighting 110% 118% 230% - Smart Thermostat 1909% - 129% - Storm Windows 100% 125% 100% 146% Wall Insulation 104% 131% 107% 0% Water Heater Insulation 0% 0% 100% 100% Water Heater Pipe Insulation 96% 98% 100% 102% Window AC 148% 100% - - Window Replacement 100% 100% 100% 100% Overall 103% 118% 103% 101% Summary of Savings Verification Findings Ex post savings were calculated through TRM verification of EnerTrek inputs and ex ante savings values. Any instances of discrepancies between ex ante and ex post savings were due to one of three issues: Difference in TRM: EnerTrek calculated measure savings in 2015 using TRM 4.0, and the Evaluators conducted savings verification using TRM 5.0. There were differences in input assumptions, measure parameters, and savings equations between the two TRM versions for some measures. Calculation Error: Any difference in interpretation of TRM protocols, mathematical errors, or data entry errors may cause ex ante savings to be higher or lower than ex post (verified) savings. On-site Verification Issues: Measures that were unable to be verified during verification visits at participant s homes received zero savings. Executive Summary 1-10

59 2015 Arkansas Weatherization Program EM&V Report The realization rate for most measures was close to 100%, and the Evaluators found that the majority of discrepancies between ex ante and ex post savings were due to differences between TRM V4.0 and TRM V5.0 rather than due to calculation errors. The following list identifies measure categories where there were significant differences between ex ante and ex post savings, and specifies whether this was due to differences in TRM versions or due to calculation errors: Ceiling Insulation o Difference in TRM: High overall electric and gas realization rates were due to differences in TRM versions. TRM V5.0 provides deemed saving tables for both R-38 and R-49 and allows for linear interpolation for insulation that is between the two R-values. TRM V4.0 only provides a deemed savings table for R-38 insulation. Floor Insulation o Calculation Error: Low overall electric realization rates are due to differences in TRM versions. Deemed savings in TRM V5.0 included negative energy savings values depending on weather zone and HVAC equipment configuration. The simulation procedures used for this measure in TRM V5.0 identified negative electric savings, likely caused by the floor insulation acting as a barrier to ground cooling effects. This would cause the home temperature to be higher during cooling months, likely resulting in increased air conditioner usage. The ex ante savings calculation did not allow for homes to receive negative energy savings for floor insulation while realized savings incorporated this increased energy usage. Wall Insulation o Difference in TRM: High overall electric and gas realization rates were due to differences in TRM versions. TRM V5.0 provides deemed saving tables for both R-13 and R-23 and allows for linear interpolation for insulation that is between the two R-values. TRM V4.0 only provides a deemed savings table for R-13 insulation. Residential Lighting o Difference in TRM/Calculation Error: High overall electric and gas realization rates were due to differences between TRM versions and possible EnerTrek calculation issues. CFL annual kwh savings in EnerTrek may have been calculated as an increment of lifetime savings, which takes into account future baseline changes that should not affect first-year kwh savings. Additionally, the interactive effects factor to account for gas heating penalties was updated in TRM V5.0. Refrigerator Replacement Executive Summary 1-11

60 2015 Arkansas Weatherization Program EM&V Report o Difference in TRM/Calculation Error: High overall electric realization rates were due to differences between TRM versions and possible EnerTrek calculation issues. Refrigerator annual kwh savings in EnerTrek may have been calculated as an increment of lifetime savings, which takes into account future baseline changes that should not affect first-year kwh savings. ADM also assigned TRM V5.0 refrigerator types based on the model number of the efficient refrigerator provided in the tracking data. Central AC Replacement o Difference in TRM: The low kwh realization rate and high kw realization rate is due to differences in TRM versions. TRM V4.0 provides a deemed savings table while TRM V5.0 provides a savings algorithm requiring additional measure specific inputs. Direct Vent Heater o Calculation Error: High overall electric realization rates were due to differences between TRM versions and possible EnerTrek calculation issues. Direct vent heater annual kwh savings in EnerTrek may have been calculated as an increment of lifetime savings, which takes into account future baseline changes that should not affect first-year kwh savings. In addition, in cases where existing unit exceeded the age of eligibility that can be claimed as early retirement according to Section 1.8 of TRM V5.0, replacement on burnout methodology was used. Gas Central Replacement o Difference in TRM/Calculation Error: High overall gas realization rates were due to differences in TRM versions and possible EnerTrek calculation issues. Gas Central Replacement annual kwh savings in EnerTrek may have been calculated as an increment of lifetime savings, which takes into account future baseline changes that should not affect first-year kwh savings. In addition, in cases where existing unit exceeded the age of eligibility that can be claimed as early retirement according to section 1.8 of TRM V5.0, replacement on burnout methodology was used. Finally, heating load value and equivalent full hours were updated in TRM V5.0. Heat Pump Replacement o Difference in TRM/Calculation Error: The low electric realization rates are due to differences in TRM versions. TRM V4.0 provides a deemed savings table while TRM V5.0 provides a savings algorithm requiring additional measure specific inputs. Smart Thermostat Executive Summary 1-12

61 2015 Arkansas Weatherization Program EM&V Report o Difference in TRM: The high electric and gas realization rates are due to differences in TRM versions. Smart Thermostats were added for TRM V5.0. Previous evaluations relied on NEAT savings values. Storm Windows o Difference in TRM: The high peak savings realization rates are due to differences in TRM versions. TRM V5.0 provides updated multipliers used estimating peak demand reductions. Water Heater Pipe Insulation o Difference in TRM/On-site Verification Issues: The low electric realization rates are due to an adjustment made by the Evaluators based on on-site verification of this measure. This is described in more detail in Section 2.6. The high peak gas realization rates are due to updates to TRM V5.0. Water Heater Tank Wrap o On-site Verification Issues: The 0% electric realization rate is due to an adjustment made by the Evaluators based on on-site verification of this measure. This is described in more detail in Section 2.6. Window AC o Difference in TRM: The high overall electric realization rates are due to differences in TRM versions. The room adjustment factor was updated for TRM V5.0 causing an increase in savings. Detailed savings verification findings can be found in Sections 2.8 and 2.9 of this report Responsiveness to Prior Year Recommendations Table 1-7 summarizes the status of issues and recommendations identified in the 2014 process evaluation and impact evaluation of the Arkansas Weatherization Program. Some issues such as tracking data errors have been addressed, but several of the issues have persisted through the 2015 program year. Executive Summary 1-13

62 2015 Arkansas Weatherization Program EM&V Report Table 1-7 Status of Recommendations from 2014 Program Year Issue Consequences Recommendation Program Response Many AWP operational and performance issues are related to WAP operations and WAP requirements for community action agencies. There were minor tracking data errors such as missing ex ante savings, calculation errors, and other missing fields in some cases. Some utilities provide or link to program documents that are out-of-date. Most of the participating agencies do not discuss the AWP on their websites, and frame weatherization as an income-qualified service. Restricts agency participation in AWP May create inconsistencies in data collection, leading to potential errors for the AWP Potentially lost savings Skewed measure-level realization rates Customers may gain inaccurate information regarding service providers and other details. May reduce program interest from private copay customers. CADC should continue to make efforts to work with the AEO in developing a mutually beneficial working relationship, and maintain consistency between the two programs where feasible. Resolve these tracking data issues for the 2015 program year. The utilities should review their website materials and provide links to updated program documentation if possible. The agencies should provide information regarding the AWP on their websites, and explain that the program does not have an income level requirement. There does not appear to have been an improvement in agency activity or improved cooperation between WAP and AWP mechanisms. Most tracking issues have been resolved, only minor issues remain No longer relevant for most utilities, and AWP marketing has ceased due to program ending after Status of Issue Persists Resolved Reviewed and Rejected/No Longer Applicable Some data are not available due to being only in hardcopy form or decentralized from the CADC. Potential lost data Potential delays in data transfer if additional data are needed Agencies should maintain electronic records of all collected audit, implementation, and verification data. Sufficient electronic data exist to satisfy TRM requirements, but some data remain in hard copy only Partially Addressed Executive Summary 1-12

63 2015 Arkansas Weatherization Program EM&V Report Issue Consequences Recommendation Program Response Periodic program activity updates to the utilities do not include measure level cost data or measure counts. The reported air infiltration leakage rates appear skewed downward, based on the Evaluators site visits. Limits utility ability to plan for annual reporting Limits utility awareness of program performance Possible issues with measure implementation or data collection Possible discrepancies between implementation and verification that will lead to skewed realization rates. Include more details in the periodic reports that are sent to utilities, including measure counts/descriptions, customer names, etc. 1: Include itemized air infiltration measures in the tracking data so that the Evaluators are able to verify individual measure elements 2: Include any field notes related to the blower door test in the tracking data so that the Evaluators may more accurately recreate the testing conditions 3: Discuss air infiltration testing procedures with the Evaluators in order to ensure that the testing methodologies are consistent among agencies, their contractors, and the Evaluators. The level of detail in monthly and quarterly reports to the utilities from CADC and other agencies has not increased. Measure counts and specific participant information have not been included. There has been no change in the level of detail of air infiltration measures, and no clarification of methodologies Status of Issue Persists Persists Executive Summary 1-13

64 2015 Arkansas Weatherization Program EM&V Report Summary of Conclusions and Recommendations After reviewing the Arkansas Weatherization Program for 2015, the Evaluators highlight the following conclusions: Minimal Effects from CADC Transition: The transition from ACAAA to CADC as the program administrator does not appear to have mitigated the AWP s operational or performance issues. It appears that CADC made efforts to work with the AEO and increase agency involvement with the program, but any beneficial effects of these efforts were for the most part overshadowed by the program s decreased activity overall. Quality of Work Issues: The Evaluators visited two participating homes that received weatherization services from PBJCEOC during The Evaluators identified issues with the quality of work performed, and one of the homeowners indicated that the agency contractors had chipped paint in their home and had not been considerate of the home in general. The Evaluators found that silver bubble wrap had been used as water heater tank insulation in one home and that water heater pipe wrap was not correctly in place at the time of the visit. It is unclear whether these quality of work issues are limited to this agency or are indicative of a larger problem, and agency staff responded to the findings indicating that the silver bubble wrap is no longer being used and that the pipe wrap had been installed properly. However it may be beneficial to conduct further training with agency staff in order to ensure that they are complying with industry quality standards, and that they are providing adequate customer service to participants. Minor Improvements in Data Accuracy: Tracking data errors have been for the most part resolved in 2015, and the current version of the tracking database within EnerTrek contains the necessary information to comply with TRM V4.0 requirements. Overall Frontier Associates has been very responsive to data requests and provided the utilities with fairly accurate batches of data throughout the program year. There were fewer tracking data issues in 2015 as compared to Continued WAP Reliance Issues: As with prior years, program staff acknowledged the challenges that have emerged and persisted due to the AWP s relationship with the Weatherization Assistance Program (WAP). Ideally, this arrangement would use utility funds to efficiently leverage federal funding and substantially increase the number of weatherization projects that the agencies are able to perform. However, the AWP s inherent link to the WAP has continued to result in performance issues due to federal funding reductions. Additionally, the participating agencies were directed to prioritize LIHEAP funding over AWP funding when implementing weatherization projects, which is a key barrier to AWP program activity. Decreasing Program Activity: The number of participants and the resulting savings levels for the AWP have steadily decreased since the 2011 program year. A major contributing factor to this decline in program activity is likely the fact that the program was winding down in 2015 and the IOUs were already focusing efforts on the Consistent Weatherization Approach. However other issues including variable agency engagement Executive Summary 1-14

65 2015 Arkansas Weatherization Program EM&V Report in weatherization services, inconsistent availability of WAP funding, and insufficient interest from private co-pay customers have historically limited the program s performance. Upcoming Consistent Weatherization Approach: The new weatherization framework developed by the utilities and other stakeholders has established statewide weatherization procedures and services, and will be implemented beginning in Utility staff reported that they anticipate that this Consistent Weatherization Approach will be a more effective method of meeting the state s weatherization needs. Additionally, utility staff noted that the collaborative relationship among utilities has improved during the development of the new framework and that the core framework will hopefully lead to a more coordinated approach to weatherization in the state. Although 2015 marks the final year of operation for the AWP as it currently stands, the Evaluators provide the following recommendations that the utilities or agencies may consider when moving forward with weatherization services under the Consistent Weatherization Approach framework: Mitigate Quality of Work Issues: The quality of work issues identified by the Evaluators during on-site verifications during 2015 suggest that additional training or verification may be needed for contractors in the Pine Bluff region. Overall, the Evaluators suggest that additional quality assurance and training be conducted with any new contractors who are brought onto the IOUs Consistent Weatherization Approach offerings. Record and Report Air Infiltration Details: As with prior years, the Evaluators identified discrepancies between reported air infiltration leakage rates and verified air infiltration leakage rates. Although only nine homes received blower door testing as part of the 2015 evaluation, the majority of these homes showed verified infiltration rates that were higher than reported infiltration rates. Moving forward, the Evaluators recommend that the IOUs and contractors collect and report the itemized air infiltration measures that are installed. As it is very difficult to reliably replicate blower door results during a site visit, having this additional information will allow program staff or their EM&V contractors to verify that the work was performed properly. Executive Summary 1-15

66 2015 Arkansas Weatherization Program EM&V Report The Evaluators identified significant issues with the quality of work in two homes during the on-site verification efforts Table 1-8 Recommendations from 2015 Program Year Evaluation Issue Consequences Recommendation Some measures were not eligible for savings The reported air infiltration leakage rates appear skewed downward, based on the Evaluators site visits. Negatively affects customer satisfaction Possible issues with measure implementation or data collection Possible discrepancies between implementation and verification that will lead to skewed realization rates. Contractors joining Consistent Weatherization Approach offerings, namely in the Pine Bluff area, should receive additional training and undergo quality control procedures that ensure sufficient customer service and installation of measures. Record and report itemized air infiltration measures in the tracking data so that it is possible to verify individual measure elements. Also, Include any field notes related to the blower door test in the tracking data so that testing conditions can be more accurately replicated. Executive Summary 1-16

67 2015 Arkansas Weatherization Program EM&V Report 1.4 Report Organization The report is organized as follows: Chapter 2 presents the impact findings and discusses the methods used for, and the results obtained from, estimating gross and net savings for the program; Chapter 3 presents the results of the process evaluation tasks and additional program findings; and Chapter 4 presents key conclusions and recommendations from the evaluation of the program. Appendix A presents the participant survey instrument. Executive Summary 1-17

68 2. Impact Evaluation Findings This chapter presents the results of the measure verification and savings calculations for the Arkansas Weatherization Program (AWP) in the 2015 program year. 2.1 Glossary of Terms As a first step to detailing the evaluation methodologies, the Evaluators provide a glossary of terms to follow: Ex Ante Savings Energy savings as determined and reported by program implementers/sponsoring utilities prior to evaluation by EM&V contractor Ex Post Gross Savings Energy savings as determined by the EM&V contractor through engineering analysis, statistical analysis, and/or onsite verification o Gross Realization Rate Ratio of Ex Post Gross Savings / Ex Ante Savings Ex Post Net Savings Ex Post Gross savings x Net-to-Gross Ratio o Net-to-Gross Ratio (NTGR) = (1 Free-Ridership % + Spillover %), also defined as Ex Post Net Savings / Ex Post Gross Savings Summary of Ex Ante Savings The Arkansas Weatherization Program is designed to use both electric and gas utility funds to assist customers with the cost of the in-home audit and energy efficient measures. Table 2-1 presents the overall ex ante savings by measure. These values are based on the claimed savings values within the EnerTrek software tool. Exports of these data were provided to the Evaluators for analysis and verification purposes. 6 The net-to-gross ratio for the AWP in 2015 is 1, thus 100% of gross savings are counted as net savings. Impact Evaluation Findings 2-1

69 2015 Arkansas Weatherization Program EM&V Report Table 2-1 Ex Ante Savings by Measure Type Overall Measure Peak Demand Savings (kw) Annual Savings (kwh) Peak Demand Savings (Therms) Annual Savings (Therms) Air Infiltration , , Ceiling Insulation , , Central AC Replacement , Direct Vent Heater Duct Sealing Installation Floor Insulation Gas Central Replacement Heat Pump Replacement , Refrigerator Replacement , Residential Lighting , (0.13) - Smart Thermostat Storm Windows Wall Insulation Water Heater Insulation Water Heater Pipe Insulation Window AC Window Replacement , Total , , Ex Ante Savings for Electric Utilities The electric utilities with participating homes in the AWP during 2015 were AEP- SWEPCO and EAI. Table 2-2 presents the ex ante savings of the electric IOUs for the 2015 program year. Table 2-2 Ex Ante Savings by Electric Utility Electric Utility # of Homes Peak Demand Annual Savings Savings (kw) (kwh) AEP-SWEPCO , EAI , OG&E Non-IOU , Total , Impact Evaluation Findings 2-2

70 2015 Arkansas Weatherization Program EM&V Report Table 2-3 and Table 2-4 summarize the ex ante savings by measure for the two electric utilities that had participating homes during The Non-IOU category refers to savings that were achieved as a result of program services, but were not attributable to the investor-owned utilities (IOUs) that fund the Arkansas Weatherization Program. Table 2-3 Ex Ante Savings by Measure Type AEP-SWEPCO Measure Peak Demand Savings (kw) Annual Savings (kwh) Air Infiltration , Ceiling Insulation , Central AC Replacement , Direct Vent Heater - - Duct Sealing Installation - - Floor Insulation - - Gas Central Replacement - - Heat Pump Replacement , Refrigerator Replacement Residential Lighting , Smart Thermostat Storm Windows - - Wall Insulation - - Water Heater Insulation - - Water Heater Pipe Insulation Window AC - - Window Replacement , Total , Impact Evaluation Findings 2-3

71 2015 Arkansas Weatherization Program EM&V Report Table 2-4 Ex Ante Savings by Measure Type - EAI Measure Peak Demand Savings (kw) Annual Savings (kwh) Air Infiltration , Ceiling Insulation , Central AC Replacement , Direct Vent Heater - - Duct Sealing Installation - - Floor Insulation Gas Central Replacement - - Heat Pump Replacement , Refrigerator Replacement , Residential Lighting , Smart Thermostat Storm Windows Wall Insulation Water Heater Insulation Water Heater Pipe Insulation Window AC Window Replacement , Total , Impact Evaluation Findings 2-4

72 2015 Arkansas Weatherization Program EM&V Report Table 2-5 presents the ex ante electric savings that were not associated with any IOU. These ex ante savings are attributable to municipal utilities, co-op utilities, or other energy providers. These savings are not attributed to any specific program sponsoring utility, and are provided for reference only. Table 2-5 Ex Ante Savings by Measure Type Non-IOU (Electric) Measure Peak Demand Savings (kw) Annual Savings (kwh) Air Infiltration , Ceiling Insulation , Central AC Replacement , Direct Vent Heater - - Duct Sealing Installation - - Floor Insulation - - Gas Central Replacement - - Refrigerator Replacement - - Residential Lighting Smart Thermostat , Storm Windows Wall Insulation Water Heater Insulation - - Water Heater Pipe Insulation - - Window AC Window Replacement - - Total , Ex Ante Savings for Gas Utilities The only IOU with participating homes during 2015 was CenterPoint. Table 2-6 presents the ex ante savings of each IOU for the 2015 program year. Table 2-6 Ex Ante Savings by Gas Utility Gas Utility # of Homes Peak Demand Savings (Therms) Annual Savings (Therms) AOG CenterPoint , Black Hills Non-IOU , Total , Impact Evaluation Findings 2-5

73 2015 Arkansas Weatherization Program EM&V Report Table 2-7 summarizes the ex ante savings by measure CenterPoint, the only gas IOU with participating homes in Table 2-7 Ex Ante Savings by Measure Type CenterPoint Measure Peak Demand Savings (Therms) Annual Savings (Therms) Air Infiltration , Ceiling Insulation , Central AC Replacement - - Direct Vent Heater Duct Sealing Installation - - Floor Insulation Gas Central Replacement Refrigerator Replacement - - Residential Lighting - - Smart Thermostat - (0.12) Storm Windows Wall Insulation Water Heater Insulation Water Heater Pipe Insulation Window AC Window Replacement - - Total , Impact Evaluation Findings 2-6

74 2015 Arkansas Weatherization Program EM&V Report Table 2-8 presents the ex ante gas savings that were not associated with any AWP IOU. The non-iou ex ante gas savings may represent propane customers or other types of customers that do not receive gas service from an IOU. Therefore, Table 2-8 is a reflection of the non-iou ex ante gas savings that are claimed within the tracking system, and these savings are not applicable to any specific service provider. Table 2-8 Ex Ante Savings Values by Measure Type Non-IOU (Gas) Measure Peak Demand Annual Savings Savings (Therms) (Therms) Air Infiltration , Ceiling Insulation , Central AC Replacement - - Direct Vent Heater Duct Sealing Installation - - Floor Insulation Gas Central Replacement Refrigerator Replacement - - Residential Lighting - - Smart Thermostat - (0.13) Storm Windows Wall Insulation Water Heater Insulation Water Heater Pipe Insulation Window AC Window Replacement - - Total , Gross Savings Calculation Methodology For measures implemented through the 2015 program, savings verification was performed according to methodologies described in TRM V5.0. Table 2-9 identifies the sections in the TRM that were used for verification of measure-level savings under the AWP. 7 7 The savings for storm windows were calculated through NEAT/MHEA, and these measures do not have an associated savings algorithm section in the TRM. However, Appendix H in TRM V5.0 specifies that NEAT/MHEA peak savings for storm windows should be multiplied by a deemed demand factor. This approach was used for the 2015 evaluation. Impact Evaluation Findings 2-7

75 2015 Arkansas Weatherization Program EM&V Report Table 2-9 TRM Sections by Measure Type Measure TRM Version Section in TRM Air Infiltration Ceiling Insulation Central AC Replacement ENERGY STAR Windows Floor Insulation Gas Furnace Replacement Heat Pump Replacement Lighting Efficiency ENERGY STAR Refrigerator Smart Thermostat Storm Windows 5.0 Appendix H Direct Vent Heaters Wall Insulation Water Heater Jackets Water Heater Pipe Insulation Window AC Air infiltration reduction and ceiling insulation accounted for the majority of both gas and electric savings for the AWP in Residential lighting (CFL installation) also accounted for a substantial portion of electric savings. The calculation methodologies for these three measures are detailed in the following sections. The deemed savings algorithms in TRM 5.0 for air infiltration reduction were developed through simulation modeling in BEopt, a residential building simulation modeling platform that uses the DOE EnergyPlus simulation engine. Multiple equipment configurations were simulated in each of the four Arkansas weather zones in developing savings values denominated in deemed savings per CFM50 of air leakage rate reduction. The following table summarizes the deemed savings values for Weather Zone 7. Table 2-10 Deemed Savings Values for Air Infiltration Reduction, Zone 7 Equipment Type kwh Savings / CFM50 (ESF) kw Savings / CFM50 (DSF) Therm Savings / CFM50 (GSF) Peak Therms / CFM50 (GPSF) Electric AC with Gas Heat Gas Heat Only (no AC) Elec. AC with Resistance heat n/a N/A N/A Heat Pump N/A N/A Impact Evaluation Findings 2-8

76 2015 Arkansas Weatherization Program EM&V Report The following example considers a residence in Weather Zone 7 with electric AC and gas heat. If the residence had a leakage rate of 16,100 CFM50 before air infiltration reduction and a leakage rate of 7,220 CFM50 after, then the residence would have an annual gross savings of 1,687 kwh. kwh Savings Air Infiltration Savings = (16,100 CFM CFM 50 pre 7,220 CFM 50 post ) 50 Air Infiltration Savings = 1,687 kwh TRM V5.0 also specifies Minimum Final Ventilation Rates (MVR) and Maximum Preinstallation Infiltration Rates in order to ensure that air infiltration work is performed in accordance with health and safety guidelines and that infiltration reduction is not attempted on homes with prohibitively severe leakage levels. TRM V5.0 specifies that the MVR must comply with current Arkansas building codes, which present three options for calculating MVR. However, as per Protocol E2 of TRM V5.0, the enforcement date for a code or standard update is the end of the current program year if the effective date of the code or standard update is before July 1. Therefore the Evaluators applied the MVR guidelines from TRM V4.0 for the 2015 evaluation. TRM V4.0 specifies MVR as follows: The MVR specifies the minimum post-installation air infiltration value that can be applied to the deemed savings calculation. If a home s final CFM50 value is below the MVR, the deemed savings calculation for air infiltration reduction on the home is calculated using the MVR rather than the actual post-installation leakage value. The MVR for a given home is calculated as follows: Where: Min CFM50 = [0.01 x Afloor x (BR + 1)] X N Min CFM50 = Minimum final ventilation rate (CFM50) AFloor = Floor area (ft 2 ) BR= Number of bedrooms (must be at least 1) N = N factor (deemed value based on type of wind shielding and number of stories in home) With regard to Maximum Pre-installation Infiltration Rate, TRM 5.0 specifies that in order to avoid incentivizing homes with severe building envelope issues that cannot be remedied with typical air infiltration procedures, the baseline pre-installation infiltration rate should be based on a maximum air change rate of 3.0. With this baseline in effect, the maximum allowable pre-installation CFM50 value is calculated as follows: Impact Evaluation Findings 2-9

77 2015 Arkansas Weatherization Program EM&V Report Where: CFM 50,pre /ft 2 = ACH Nat,pre h N 60 CFM50,pre /ft 2 = Per square foot pre-installation infiltration rate (CFM50/ft 2 ) ACHNat,pre = Maximum pre-installation air change rate (ACHNat) = = Constant to convert from minutes to hours h = Ceiling height (ft) = 8.5 (default) N = N factor (deemed value based on type of wind shielding and number of stories in home) If a home s pre-installation infiltration rate exceeds the rate calculated above, the Maximum Pre-installation Infiltration Rate is used for deemed savings calculations. Additionally, TRM 5.0 specifies a maximum CFM50 per-square-foot value. For deemed savings calculations, pre-installation leakage rates cannot exceed these values Ceiling Insulation Savings Calculations The deemed savings algorithms in TRM 5.0 for ceiling insulation were developed through simulation modeling in BEopt, a residential building simulation modeling platform that uses the DOE EnergyPlus simulation engine. Multiple equipment configurations were simulated in each of the four Arkansas weather zones in developing savings values denominated in deemed savings per square footage of ceiling area. Table 2-11 summarizes the deemed savings values for R-38 insulation for participants located in Weather Zone 8. Table 2-11 Deemed Savings Values for Ceiling Insulation, Zone 8 Ceiling Insulation Base R-value AC/Gas Heat kwh Gas Heat (no AC) kwh Gas Heat (no AC) Therms AC/Electric Resistance kwh Heat Pump kwh AC Peak Savings (kw) Peak Gas Savings 8 (therms) (/ sq. ft.) (/ sq. ft.) (/ sq. ft.) (/ sq. ft.) (/ sq. ft.) (/ sq. ft.) (/ sq. ft.) 0 to to to to to Data in table are for Blytheville peak. Other Zone 8 peaks can be calculated by multiplying Blytheville peak by the appropriate factor, m. For Jonesboro, m=0.890 (0-1), m = (2 to 4), (5 to 8), (9 to 14), (15 to 22). For Fort Smith, m=0.859 (0-1), m = (2 to 4), (5 to 8), (9 to 14), (15 to 22). Impact Evaluation Findings 2-10

78 2015 Arkansas Weatherization Program EM&V Report The following example considers a residence that had R-38 insulation installed in Weather Zone 8 with a heat pump, and a pre-retrofit R-value of ceiling insulation in the range of 9 to 14. If the residence has a ceiling area of 1,200 sq. ft., then the residence would have an annual gross savings of 1,100 kwh. Ceiling Insulation Savings = kwh ft 2 (1,200 ft2 ) = 1,100 kwh TRM 5.0 specifies an efficiency standard of R-38, meaning that in order to qualify for deemed savings the combined R-value of existing and added insulation should be at least R Compact Fluorescent Lamps (CFLs) Savings Calculations The deemed savings for compact fluorescent lamps can be calculated by using the following equation. kwh savings = ((Watts base Watts post )/1,000) x Hours x ISR x IEF E The inputs, which assume the following prerequisite knowledge, can be found in Section of TRM V5.0: The quantity, lumens, and wattages post fixtures; Whether or not the retrofits were time of sale or direct install (this defines the inservice rate); and The heating type of the residence. For example, if in March 2015 (5) 13W and 800 lumen CFLs were directly installed in a residence with gas heating, the residence would have an annual gross savings of 128 kwh. kwh savings = (( )/1, = 128 kwh TRM 5.0 includes specifications for heating penalties from CFLs in natural gas heated homes, calculated as follows: Where: Therms penalty = ((W base W post )/1000) x ISR x IEF G IEFg = Interactive Effects Factor to account for gas heating penalties TRM V5.0 also accounts for future changes in lighting baselines as per EISA 2007 guidelines. Specifically, TRM V5.0 specifies that the 1 st Tier EISA 2007 baselines come into effect in January 2014, and that the 2 nd Tier EISA 2007 baselines come into effect in January These baseline changes affect lifetime savings calculations for CFLs. As per Protocol E2 of TRM V5.0, the enforcement date for a code or standard update is the end of the current program year if the effective date of the code or standard update is Impact Evaluation Findings 2-11

79 2015 Arkansas Weatherization Program EM&V Report before July 1. Thus, the Evaluators calculated 2015 first-year savings using the 1 st Tier EISA baseline. 2.4 Net Savings Determination As with prior program years, the Evaluators applied a net-to-gross ratio of 1 for savings achieved through the program in The context for and explanation of this determination, which appeared in prior AWP evaluation reports, is provided below. The Evaluators conducted a net-to-gross assessment of the program in 2012 in order to determine the likelihood of significant free-ridership or savings spillover. Feedback obtained from customers, community action agencies, and utility staff indicates that the likelihood for program free-ridership is very low. As a high percentage of AWP participants qualified for and participated in the income-qualified statewide Weatherization Assistance Program (WAP), they are unlikely to be candidates for free-ridership in the AWP. The promotional structure of the AWP targets customer groups who would be very unlikely to pursue these weatherization projects in the absence of the program, and who would likely not seek out an energy audit at their own cost. Additionally, participants who were visited by the Evaluators field staff were asked a series of questions related to program savings spillover, and none of these customers identified any potential spillover savings. Due to these factors, the Evaluators determined the net-to-gross ratio for the Arkansas Weatherization Program to be 1, or 100% of gross savings, for the 2012 program year. This determination has been carried over and applied to the 2015 program year, and 2015 AWP gross savings are equal to net savings. 2.5 On-site Verification Procedure In addition to TRM verification, the Evaluators conducted on-site field verification of a sample of participant homes. This process involved reviewing tracking information and inspecting the completeness and accuracy of the implemented measures. This section provides a summary of the methodology used by the Evaluators to conduct the verification activity Verification Sampling Methodology The Evaluators conducted a random sample of participants for the ex-post verification process. The sample size for verification was calculated to meet 90% confidence and 10% precision (90/10) for the overall 2015 program population at the time of the on-site verification activity. The main purpose of the verification activity was to determine whether measures were properly installed in the quantities reported in program tracking data. Thus, the coefficient of variation (CV) used for sampling was not based on participant savings but was assumed to be 0.5, which is a commonly assumed CV value for residential program evaluations. The resulting sample size is estimated as: Impact Evaluation Findings 2-12

80 2015 Arkansas Weatherization Program EM&V Report Where, CV n 0 = ( ) RP = Z Score for 90% confidence interval in a normal distribution CV = Coefficient of Variation RP = Required Precision, 10% in this evaluation With 10% required precision (RP), this calls for a sample of 68 for programs with a sufficiently large population. However, for programs with lower levels of participation, a finite population correction is used to maintain cost-effective verification while meeting precision goals. For the AWP, the Evaluators applied a finite population correction factor as follows: Where n = n0 = Sample Required for Large Population N = Size of Population n = Corrected Sample n n 0 N During 2015, 94 residences received measures through the AWP. After applying the population correction factor, the program calls for a sample size of 40 participants. In total, the Evaluators attempted to schedule 28 site visits but due to the limited number of participants in the program population, as well as cancellations and customer absences, Evaluator field staff members were able to conduct on-site visits for 13 program participants. This does not meet the sampling requirement specified above. To supplement the verification effort, the Evaluators conducted participant satisfaction surveys with 24 customers which also served to confirm that they had participated in and received measures through the program. Of these 24 survey respondents, 9 were also part of the group of 13 customers who received on-site verification visits from the Evaluators. Thus, 13 customers received on-site measure verification, and 15 additional customers received program participation verification through the participant survey. This was supplemented by 12 brief telephone verification calls to bring the total number of unique customer verifications to 40. Although the participant satisfaction survey did not verify individual measure installation, the Evaluators were able to confirm that all 40 sampled customers had participated in the program. The sample achieved for the evaluation is not representative of the population 2 Impact Evaluation Findings 2-13

81 2015 Arkansas Weatherization Program EM&V Report at the measure level, but it is representative at the project level in verifying customer participation On-site Verification Procedure The primary goal of field verification was to ensure that the reported measures were installed and operating correctly in participant homes. Participants were given Walmart gift cards for their time; these were in the amount of $25. During the on-site visits, the Evaluators field technicians accomplished the following: Verified the implementation status of the measures; verified that the measures were installed, that they were installed correctly, and were functioning properly. Photographs were taken of most of the installed measures. Data collected at each site focused on obtaining more specific information regarding the characteristics of the home where the measures were implemented. A field visit form was completed for each visited site in order to document measure quantities, home characteristics, and any needed additional commentary regarding the visit. Specifically, the field form included the following fields: Home Characteristics: The field engineer documented the type of home (i.e. single story vs. multi-story), number of bedrooms, number of bathrooms, total conditioned area, and heating type. Measure Quantity Verification: The engineer documented reported vs. actual quantities of each measure type (e.g. CFLs, water heater measures) and any applicable notes regarding burnt out bulbs or non-operational equipment. Insulation Assessment: The form includes fields for insulation square footage, the R-value or inches of insulation, and the type of insulation (e.g. blown cell). Infiltration Assessment: For homes receiving air infiltration measures, the field engineer conducted a blower door test and recorded ex-post leakage for comparison with reported leakage values. Supplemental Notes: The field engineer recorded any notable comments provided by the customer regarding the work that was performed, and identified any verification issues that had occurred during the visit (e.g. if the attic was not accessible). 2.6 Onsite Verification Results As described in Section 2.5 of this report, the Evaluators conducted onsite verification visits to 13 participant homes. These site visits were conducted in order to verify complete and proper measure installation, to conduct post-implementation measurements, and to Impact Evaluation Findings 2-14

82 2015 Arkansas Weatherization Program EM&V Report collect information regarding residence characteristics such as square footage and heating type. The field and telephone verification activity showed that the weatherization measures had for the most part been installed in the quantities reported within program tracking data, although quality of work issues were identified in two homes. This section summarizes the verification findings by measure category. Specific notes regarding the onsite and telephone verification findings include: Contact Information All residences were located at the addresses provided within the tracking data. Additionally, a high percentage of telephone numbers were found to be accurate, which is an improvement over the 2014 program year. In total, only 4 out of the 54 telephone numbers (7%) dialed by the Evaluators were found to be disconnected or incorrect during the site visit scheduling process Air Infiltration The Evaluators conducted blower door testing in nine participant homes for the 2015 program year. Of these homes, the CFM50 value measured during the verification visit was within 10% of the reported value for four homes. The Evaluated CFM50 value was more than 10% greater than the Reported CFM50 value for four homes. Overall, the Evaluated CFM50 value was greater than the Reported CFM50 value for six of the nine homes. As mentioned in prior evaluation reports for this program, there are several factors that may cause the Evaluated CFM50 value to be greater than the Reported CFM50 value, including customer actions following the weatherization work (such as removing door sweeps), methodological differences between contractor blower door testing and Evaluator blower door testing, and environmental or weather effects. Without additional information regarding air sealing and leakage testing procedures conducted by contractors for each home, it is not possible to determine the reason for these measurement discrepancies. The Evaluators have previously recommended that itemized air infiltration measures be included in program tracking data, but this may be associated with a database programming cost. Similarly, field notes regarding how the initial blower door test was conducted (such as whether a fireplace flue was open or closed) may be useful for the verification process but generating a report of this information may require additional EnerTrek programming. Overall, increasing the level of tracking data detail and minimizing methodological differences among contractors would help to distinguish data entry and implementation errors from situational and procedural differences. As this is the final year of program operation, moving forward this issue may only be relevant to IOUs who will continue to Impact Evaluation Findings 2-15

83 2015 Arkansas Weatherization Program EM&V Report use community action agency contractors to provide services under the Consistent Weatherization Approach framework Attic Insulation All reported instances of attic insulation were verified. There were no significant differences between reported pre-installation R values and evaluated pre-installation R values. There were no significant differences between reported square footage and evaluated square footage CFLs All reported instances of CFL installation were verified, in quantities that matched or closely matched program tracking data. All verified CFLs matched the wattage and lumen range reported in the EnerTrek tracking data Water Heater Jacket and Pipe Wrap The Evaluators were able to verify all but one instance of water heater jackets. In this case, the Evaluators found that silver bubble wrap had been used to wrap the water heater, rather than standard insulation jackets. The agency responsible for this work reported that the silver bubble wrap is no longer being used. All reported instances of water heater pipe wrap were verified, but in at least two cases the pipe wrap was installed on the pressure release valve line. The agency responsible for this work reported that the water heater pipe wrap had been installed on the proper lines during the initial inspection of the home Central Air Conditioner Replacement All but one reported instance of central air conditioning replacement were verified. One customer did not appear to have had a new unit installed, and their existing unit was not functioning at the time of the site visit Gas Heat Replacement All reported instances of gas central replacement were verified Window Replacement All reported instances of window replacement were verified Smart Thermostat All reported instances of smart thermostat installation were verified Wall Insulation All reported instances of wall insulation were verified Refrigerator Replacement All reported instances of refrigerator replacement were verified. Impact Evaluation Findings 2-16

84 2015 Arkansas Weatherization Program EM&V Report Direct Vent Heater All reported instances of direct vent heater installation were verified. The results of the verification activity suggest that measures are for the most part installed in the quantities reported in program tracking data. However, the Evaluators found that the quality of work conducted in the two sampled homes serviced by PBJCEOC was fairly poor. This includes the instances of bubble wrap and pipe insulation on pressure relief line mentioned above. Overall, issues identified within these homes include: It appeared that silver bubble wrap was used as water heater insulation, rather than a standard insulation jacket; Plenum sealing was done with tape that did not effectively stick to surfaces; Caulk was used in gaps that were too large, where foam sealing should have been used; Water heater pipe insulation was added to the pressure release line; and One customer indicated that the contractor had damaged paint/trim in the home. These findings were submitted to CADC and PBJCEOC staff responded, indicating that the silver water heater wrap is no longer being used and that the water heater pipe insulation had been added to the proper lines when the home was inspected. However it may be beneficial to conduct further training with PBJCEOC staff in order to ensure that they are complying with industry quality standards, and that they are providing adequate customer service to participants. As a result of the field verification activity, the Evaluators made the following adjustments to program savings: No water heater tank wrap savings were attributed to the home where silver bubble wrap was used instead of standard tank insulation; No water heater pipe wrap savings were attributed to the two homes where the pipe wrap was installed on the pressure relief valve; and No central air conditioner savings were attributed to the home where the central air conditioner did not appear to have been replaced. Rather than extrapolating these savings adjustments to the program population based on the limited sample of 13 site visits that were conducted, these adjustments only affected the individual homes where the verification issue occurred. 2.7 Review of EnerTrek Input Assumptions Although the EnerTrek system calculated savings for the AWP using protocols in TRM V4.0, some of the measure inputs required by the TRM were not collected by program contractors during In order to calculate savings, Frontier Associates developed input assumptions for individual measure types. The Evaluators reviewed these assumptions and attempted to validate or supplement specific assumptions during the Impact Evaluation Findings 2-17

85 2015 Arkansas Weatherization Program EM&V Report verification activity. The assumptions applied to individual measure calculations for some homes in 2015 include: ENERGY STAR Refrigerator Replacement: For measures where an early retirement savings methodology can be applied, but a unit age is not provided, savings were calculated using the Replace on Burnout methodology. Central AC Replacement: For measures where an early retirement savings methodology can be applied, but a unit age is not provided, savings were calculated using the Replace on Burnout methodology. Direct Vent Heaters: For measures where an early retirement savings methodology can be applied, but a unit age is not provided, savings were calculated using the Replace on Burnout methodology. Gas Central Replacement: Assume replace on burnout. Heat Pump Replacement: Assume replace on burnout. Water Heater Pipe Insulation: Assume water heater is located in a conditioned space. Data collected by the Evaluators during the verification activity indicated that the assumptions for CFLs and window replacements were reasonable and consistent with actual measure characteristics. Overall, following a review of program tracking data and field verification findings, the Evaluators determined that all of the listed assumptions were reasonable for measures implemented during Ex Post Net Savings by Measure Ex post savings were calculated through TRM verification of EnerTrek inputs and ex ante savings values. Any instances of discrepancies between ex ante and ex post savings were due to one of three issues: Difference in TRM: EnerTrek calculated measure savings in 2015 using TRM V4.0, and the Evaluators conducted savings verification using TRM V5.0. There were differences in input assumptions, measure parameters, and savings equations between the two TRM versions for some measures. Calculation Error: Any difference in interpretation of TRM protocols, mathematical errors, or data entry errors may cause ex ante savings to be higher or lower than ex post (verified) savings. On-site Verification Issues: Measures that were unable to be verified during verification visits at participant s homes received zero savings. Impact Evaluation Findings 2-18

86 2015 Arkansas Weatherization Program EM&V Report Table 2-12 presents electric and gas realization rates by measure category. These realization rates are presented at the program level, and individual utility realization rates may vary from those presented in this table. Individual utility realization rates are presented in Section 2.10 and Section Table 2-12 Gas and Electric Realization Rates by Measure Type Measure kwh Realization Rate kw Realization Rate Therms Realization Rate Peak Therms Realization Air Infiltration 100% 100% 100% 100% Ceiling Insulation 101% 149% 102% 101% Central AC Replacement 71% 161% - - Direct Vent Heater % 162% Duct Sealing Installation Floor Insulation 75% 68% 100% - Gas Central Replacement % 108% Heat Pump Replacement 49% 52% - - Refrigerator Replacement 178% 178% - - Residential Lighting 110% 118% 230% - Smart Thermostat 1909% - 129% - Storm Windows 100% 125% 100% 146% Wall Insulation 104% 131% 107% 0% Water Heater Insulation 0% 0% 100% 100% Water Heater Pipe Insulation 96% 98% 100% 102% Window AC 148% 100% - - Window Replacement 100% 100% 100% 100% Overall 103% 118% 103% 101% The Evaluators found that the majority of discrepancies between ex ante and ex post savings were due to differences between TRM V4.0 and TRM V5.0 rather than due to calculation errors. The following list identifies measure categories where there were significant differences between ex ante and ex post savings, and specifies whether this was due to differences in TRM versions or due to calculation errors: Rate Ceiling Insulation o Difference in TRM: High overall electric and gas realization rates were due to differences in TRM versions. TRM V5.0 provides deemed saving tables for both R-38 and R-49 and allows for linear interpolation for insulation that Impact Evaluation Findings 2-19

87 2015 Arkansas Weatherization Program EM&V Report is between the two R-values. TRM V4.0 only provides a deemed savings table for R-38 insulation. Floor Insulation o Calculation Error: Low overall electric realization rates are due to differences in TRM versions. Deemed savings in TRM V5.0 included negative energy savings values depending on weather zone and HVAC equipment configuration. The simulation procedures used for this measure in TRM V5.0 identified negative electric savings, likely caused by the floor insulation acting as a barrier to ground cooling effects. This would cause the home temperature to be higher during cooling months, likely resulting in increased air conditioner usage. The ex ante savings calculation did not allow for homes to receive negative energy savings for floor insulation while realized savings incorporated this increased energy usage. Wall Insulation o Difference in TRM: High overall electric and gas realization rates were due to differences in TRM versions. TRM V5.0 provides deemed saving tables for both R-13 and R-23 and allows for linear interpolation for insulation that is between the two R-values. TRM V4.0 only provides a deemed savings table for R-13 insulation. Residential Lighting o Difference in TRM/Calculation Error: High overall electric and gas realization rates were due to differences between TRM versions and possible EnerTrek calculation issues. CFL annual kwh savings in EnerTrek may have been calculated as an increment of lifetime savings, which takes into account future baseline changes that should not affect first-year kwh savings. Additionally, the interactive effects factor to account for gas heating penalties was updated in TRM V5.0. Refrigerator Replacement o Difference in TRM/Calculation Error: High overall electric realization rates were due to differences between TRM versions and possible EnerTrek calculation issues. Refrigerator annual kwh savings in EnerTrek may have been calculated as an increment of lifetime savings, which takes into account future baseline changes that should not affect first-year kwh savings. ADM also assigned TRM V5.0 refrigerator types based on the model number of the efficient refrigerator provided in the tracking data. Central AC Replacement o Difference in TRM: The low kwh realization rate and high kw realization rate is due to differences in TRM versions. TRM V4.0 provides a deemed Impact Evaluation Findings 2-20

88 2015 Arkansas Weatherization Program EM&V Report savings table while TRM V5.0 provides a savings algorithm requiring additional measure specific inputs. Additionally, it did not appear that this measure was in place in one of the sites visited during on-site verification, and that instance of this measure was not attributed with savings. Direct Vent Heater o Calculation Error: High overall electric realization rates were due to differences between TRM versions and possible EnerTrek calculation issues. Direct vent heater annual kwh savings in EnerTrek may have been calculated as an increment of lifetime savings, which takes into account future baseline changes that should not affect first-year kwh savings. In addition, in cases where existing unit exceeded the age of eligibility that can be claimed as early retirement according to Section 1.8 of TRM V5.0, replacement on burnout methodology was used. Gas Central Replacement o Difference in TRM/Calculation Error: High overall gas realization rates were due to differences in TRM versions and possible EnerTrek calculation issues. Gas Central Replacement annual kwh savings in EnerTrek may have been calculated as an increment of lifetime savings, which takes into account future baseline changes that should not affect first-year kwh savings. In addition, in cases where existing unit exceeded the age of eligibility that can be claimed as early retirement according to Section 1.8 of TRM V5.0, replacement on burnout methodology was used. Finally, heating load value and equivalent full hours were updated in TRM V5.0. Heat Pump Replacement o Difference in TRM/Calculation Error: The low electric realization rates are due to differences in TRM versions. TRM V4.0 provides a deemed savings table while TRM V5.0 provides a savings algorithm requiring additional measure specific inputs. Smart Thermostat o Difference in TRM: The high electric and gas realization rates are due to differences in TRM versions. Smart Thermostats were added for TRM V5.0. Previous evaluations relied on NEAT savings values. Storm Windows o Difference in TRM: The high peak savings realization rates are due to differences in TRM versions. TRM V5.0 provides updated multipliers used estimating peak demand reductions. Water Heater Pipe Insulation Impact Evaluation Findings 2-21

89 2015 Arkansas Weatherization Program EM&V Report o Difference in TRM/On-site Verification Issues: The low electric realization rates are due to an adjustment made by the Evaluators based on on-site verification results. This is described in more detail in Section 2.6. The high peak gas realization rates are due to updates to TRM V5.0. Water Heater Pipe Insulation o The 0% electric realization rate is due to an adjustment made by the Evaluators based on on-site verification results. This is described in more detail in Section 2.6. Window AC o Difference in TRM: The high overall electric realization rates are due to differences in TRM versions. The room adjustment factor was updated for TRM V5.0 causing an increase in savings. 2.9 Overall Ex Post Net Savings Table 2-13 presents the savings results of the evaluation of the 2015 Arkansas Weatherization Program, by measure. Total savings summarizes the savings calculations performed as per TRM V5.0 protocols for the AWP. As previously noted, the net-to-gross ratio for the 2015 program year is 1. Table 2-13 Ex Post Net Savings by Measure Type Overall Measure Peak Demand Savings (kw) Annual Savings (kwh) Lifetime Savings (kwh) Peak Demand Savings (Therms) Annual Savings (Therms) Lifetime Savings (Therms) Air Infiltration , ,033, , , Ceiling Insulation , ,324, , , Central AC Replacement , , Direct Vent Heater , , Duct Sealing Installation Floor Insulation , (0.09) Gas Central Replacement , Heat Pump Replacement , , Refrigerator Replacement , , Residential Lighting , , (0.31) (2.35) Smart Thermostat - 7, , , Storm Windows Wall Insulation , , Water Heater Insulation Water Heater Pipe Insulation , Window AC , Window Replacement , , , Total , ,178, , , Impact Evaluation Findings 2-22

90 2015 Arkansas Weatherization Program EM&V Report 2.10 Ex Post Net Savings for Electric Utilities The participating electric IOUs with homes achieving savings through the 2015 program were AEP-SWEPCO and EAI. Table 2-14 presents the ex post net savings results of the evaluation of the 2015 AWP for electric utilities. Table 2-14 Ex Post Net Savings by Electric Utility Electric Utility # of Homes Peak Demand Savings (kw) Annual Savings (kwh) Lifetime Savings (kwh) AEP-SWEPCO , , EAI , ,146, OG&E Non-IOU , , Total , ,178, Impact Evaluation Findings 2-23

91 2015 Arkansas Weatherization Program EM&V Report Table 2-15 and Table 2-16 summarize the ex post net savings and net realization rates by measure for the two electric IOUs that had participating homes during Table 2-15 Ex Post Net Savings by Measure Type AEP SWEPCO Measure Peak Demand Savings (kw) Annual Savings (kwh) Lifetime Savings (kwh) kwh Realization Rate kw Realization Rate Air Infiltration , , % 100% Ceiling Insulation , , % 148% Central AC Replacement , , % 182% Direct Vent Heater Duct Sealing Installation Floor Insulation Gas Central Replacement Heat Pump Replacement , , % 52% Refrigerator Replacement , % 148% Residential Lighting , , % 115% Smart Thermostat - 1, , % - Storm Windows Wall Insulation Water Heater Insulation Water Heater Pipe Insulation % 102% Window AC Window Replacement , , % 100% Total , , % 121% Impact Evaluation Findings 2-24

92 2015 Arkansas Weatherization Program EM&V Report Table 2-16 Ex Post Net Savings by Measure Type EAI Measure Peak Demand Savings (kw) Annual Savings (kwh) Lifetime Savings (kwh) kwh Realization Rate kw Realization Rate Air Infiltration , , % 101% Ceiling Insulation , , % 146% Central AC Replacement , , % 138% Direct Vent Heater Duct Sealing Installation Floor Insulation , % 68% Gas Central Replacement Heat Pump Replacement , , % 52% Refrigerator Replacement , , % 194% Residential Lighting , , % 117% Smart Thermostat - 4, , % - Storm Windows % 578% Wall Insulation , % 131% Water Heater Insulation % 0% Water Heater Pipe Insulation , % 97% Window AC , % 100% Window Replacement , , % 100% Total , ,146, % 116% Impact Evaluation Findings 2-25

93 2015 Arkansas Weatherization Program EM&V Report Table 2-17 presents the electric savings that were not associated with any AWP IOU. These savings are attributable to municipal utilities, co-op utilities, or other energy providers. Thus, the savings are not attributed to any specific program sponsoring utility. Table 2-17 Ex Post Net Savings by Measure Type Non-IOU (Electric) Measure Peak Demand Savings (kw) Annual Savings (kwh) Lifetime Savings (kwh) kwh Realization Rate kw Realization Air Infiltration , , % 100% Ceiling Insulation , , % 161% Central AC Replacement , % 182% Direct Vent Heater Duct Sealing Installation Floor Insulation Gas Central Replacement Heat Pump Replacement Refrigerator Replacement , % 53% Residential Lighting , , % 125% Smart Thermostat , % - Storm Windows % 94% Wall Insulation Water Heater Insulation Water Heater Pipe Insulation % 109% Window AC Window Replacement , , % 100% Total , , % 121% 2.11 Ex Post Net Savings for Gas Utilities The only gas IOU with participating homes during 2015 was CenterPoint. Table 2-18 presents the savings results of the evaluation of the 2015 AWP for CenterPoint and for non-iou sources. Table 2-19 summarizes the ex post net savings and net realization rate by measure for CenterPoint. Gas Utility Table 2-18 Ex Post Net Savings by Gas Utility # of Homes Peak Demand Savings (Therms) Annual Savings (Therms) Lifetime Savings (Therms) AOG CenterPoint , , Black Hills Non-IOU , , Total , , Rate Impact Evaluation Findings 2-26

94 2015 Arkansas Weatherization Program EM&V Report Table 2-19 Ex Post Net Savings by Measure Type CenterPoint Measure Peak Peak Annual Lifetime Therms Demand Therms Savings Savings Realization Savings Realization (Therms) (Therms) Rate (Therms) Rate Air Infiltration , , % 100% Ceiling Insulation , , % 102% Central AC Replacement Direct Vent Heater , , % 163% Duct Sealing Installation Floor Insulation (0.09) % - Gas Central Replacement , % 108% Heat Pump Replacement Refrigerator Replacement Residential Lighting - (0.28) (2.13) 230% - Smart Thermostat , % - Storm Windows % 146% Wall Insulation , % 0% Water Heater Insulation % 100% Water Heater Pipe Insulation % 102% Window AC Window Replacement , % 100% Total , , % 101% Impact Evaluation Findings 2-27

95 2015 Arkansas Weatherization Program EM&V Report Table 2-20 presents the ex post net gas savings that were not associated with any AWP IOU. The Non-IOU ex post savings may represent propane customers or other types of customers that do not receive gas service from an IOU. Table 2-20 Ex Post Net Savings by Measure Type Non-IOU (Gas) Measure Peak Demand Savings (Therms) Annual Savings (Therms) Lifetime Savings (Therms) Therms Realization Rate Peak Therms Realization Air Infiltration , % 100% Ceiling Insulation , % 100% Central AC Replacement Direct Vent Heater , % 154% Duct Sealing Installation Floor Insulation Gas Central Replacement Heat Pump Replacement Refrigerator Replacement Residential Lighting - (0.03) (0.22) 233% - Smart Thermostat % - Storm Windows % - Wall Insulation Water Heater Insulation Water Heater Pipe Insulation Rate % 101% Window AC Window Replacement % 100% Total , , % 102% Impact Evaluation Findings 2-28

96 3. Process Evaluation Findings This chapter presents the key findings from the limited process evaluation that the Evaluators conducted in This includes a status assessment of recommendations from prior program evaluations and a summary of updates to program operation and delivery. Additionally, the chapter presents findings from in-depth interviews with program staff, provides a review of customer surveys conducted by the participating community action agencies, and addresses the checklist factors for portfolio comprehensiveness. 3.1 Process Evaluation Considerations The Evaluators conducted a formal process evaluation of the AWP in 2012 and conducted limited process evaluations in 2013 and These process evaluation efforts resulted in several recommendations and identified program strengths and weaknesses, as well as existing opportunities. TRM V5.0 Protocol C addresses the criteria used to determine the timing and conditions needed for a process evaluation, and the following tables summarize the AWP in the context of these requirements. Table 3-1 Determining Process Evaluation Timing Component New and Innovative Components No Previous Process Evaluation New Vendor or Contractor Determination No. The overall program design has not been significantly modified in recent years. No. A full process evaluation was conducted in 2012, and limited process evaluations were conducted in 2011, 2013, and Yes. The program continued to be funded by the Arkansas IOUs and implemented by the Arkansas community action agencies and their contractor, but the program administrator is now one of the agencies (CADC) rather than ACAAA. Process Evaluation Findings 3-1

97 2015 Arkansas Weatherization Program EM&V Report Table 3-2 Determining Process Evaluation Conditions Component Are program impacts lower or slower than expected? Are the educational or informational goals not meeting program goals? Are the participation rates lower or slower than expected? Are the program s operational or management structure slow to get up and running or not meeting program administrative needs? Is the program s cost-effectiveness less than expected? Do participants report problems with the programs or low rates of satisfaction? Is the program producing the intended market effects? Determination Yes. Program activity has decreased over time and the program did not meet its savings goals for any of the IOUs in No. Program awareness is sufficient and participants have reported substantial increases in energy efficiency and home maintenance awareness. Yes. Program activity has decreased over time, and the program did not meet its savings goals for any of the IOUs in Yes. The community action agencies have struggled to expend utility funds towards weatherization projects. Cost-effectiveness scores for the program vary significantly by IOU. No. Participants in past years reported very high levels of satisfaction with their participation and with the quality of work performed. Possibly. Overall weatherization activity, including development of additional weatherization programs, has increased since the initiation of the AWP, although attribution to the AWP has not been formally established. As 2015 marks the final year of operation for the AWP before the IOUs begin implementing weatherization services under the Consistent Weatherization Approach framework, a full process evaluation is not needed. Instead, the Evaluators conducted a limited process evaluation focusing on the program s response to prior recommendations, current participant feedback and satisfaction, and identifying issues that may be relevant to the agencies or IOUs moving forward. In order to address these areas, the Evaluators conducted the following research tasks: Tracking database and documentation review; Interviews with program staff; and Participant surveys. Additionally, the Evaluators gained insight into savings performance through the impact evaluation. Results from the TRM verification provided insight into ex ante vs. ex post savings discrepancies and overall measure savings estimates. Table 3-3 below summarizes the survey and interview data collection for the process evaluation activities, including data collection type, number of respondents, and additional details. The Evaluators invited staff from each of the seven sponsoring IOUs to participate in in-depth interviews for the 2015 program evaluation. This request was ultimately accepted by staff representing four of the IOUs. Additionally the Evaluators discussed the Process Evaluation Findings 3-2

98 2015 Arkansas Weatherization Program EM&V Report program with CADC staff during the 2015 program year, but CADC did not respond to requests for a formal in-depth interview. Table 3-3 Interview and Survey Data Collection Summary Component Activity N Details AOG Program Manager and Staff OG&E Program Manager and Staff Interview Interview 1 1 The program manager and operational staff are responsible for managing reimbursements to local agencies, planning SWEPCO Program Manager Interview 1 for overall program activity and savings expectations, and communicating with EAI Program Manager Interview 1 agency and ACAAA staff as necessary throughout the program year. CADC Staff Mid-year Discussion Participating Customers Surveys Response to Program Recommendations 1 CADC serves as the lead community action agency and coordinates program implementation, quality assurance, and data reporting processes. Participating residential utility customers received weatherization services through the program in Table 3-4 summarizes the status of issues and recommendations identified in the 2014 process evaluation and impact evaluation of the Arkansas Weatherization Program. Some issues such as tracking data errors have been addressed, but several of the issues have persisted through the 2015 program year. Process Evaluation Findings 3-3

99 2015 Arkansas Weatherization Program EM&V Report Table 3-4 Status of Recommendations from 2014 Program Year Issue Consequences Recommendation Program Response Many AWP operational and performance issues are related to WAP operations and WAP requirements for community action agencies. There were minor tracking data errors such as missing ex ante savings, calculation errors, and other missing fields in some cases. Some utilities provide or link to program documents that are out-of-date. Most of the participating agencies do not discuss the AWP on their websites, and frame weatherization as an income-qualified service. Restricts agency participation in AWP May create inconsistencies in data collection, leading to potential errors for the AWP Potentially lost savings Skewed measure-level realization rates Customers may gain inaccurate information regarding service providers and other details. May reduce program interest from private copay customers. CADC should continue to make efforts to work with the AEO in developing a mutually beneficial working relationship, and maintain consistency between the two programs where feasible. Resolve these tracking data issues for the 2015 program year. The utilities should review their website materials and provide links to updated program documentation if possible. The agencies should provide information regarding the AWP on their websites, and explain that the program does not have an income level requirement. There does not appear to have been an improvement in agency activity or improved cooperation between WAP and AWP mechanisms. Most tracking issues have been resolved, only minor issues remain No longer relevant for most utilities, and AWP marketing has ceased due to program ending after Status of Issue Persists Resolved Reviewed and Rejected/No Longer Applicable Some data are not available due to being only in hardcopy form or decentralized from the CADC. Potential lost data Potential delays in data transfer if additional data are needed Agencies should maintain electronic records of all collected audit, implementation, and verification data. Sufficient electronic data exist to satisfy TRM requirements, but some data remain in hard copy only Partially Addressed Process Evaluation Findings 3-4

100 2015 Arkansas Weatherization Program EM&V Report Issue Consequences Recommendation Program Response Periodic program activity updates to the utilities do not include measure level cost data or measure counts. The reported air infiltration leakage rates appear skewed downward, based on the Evaluators site visits. Limits utility ability to plan for annual reporting Limits utility awareness of program performance Possible issues with measure implementation or data collection Possible discrepancies between implementation and verification that will lead to skewed realization rates. Include more details in the periodic reports that are sent to utilities, including measure counts/descriptions, customer names, etc. 1: Include itemized air infiltration measures in the tracking data so that the Evaluators are able to verify individual measure elements 2: Include any field notes related to the blower door test in the tracking data so that the Evaluators may more accurately recreate the testing conditions 3: Discuss air infiltration testing procedures with the Evaluators in order to ensure that the testing methodologies are consistent among agencies, their contractors, and the Evaluators. The level of detail in monthly and quarterly reports to the utilities from CADC and other agencies has not increased. Measure counts and specific participant information have not been included. There has been no change in the level of detail of air infiltration measures, and no clarification of methodologies Status of Issue Persists Persists Process Evaluation Findings 3-5

101 2015 Arkansas Weatherization Program EM&V Report 3.3 Program Structure Overview Program design and structure in 2015 remained fairly consistent with the 2014 program year. The following provides a review of program design characteristics and operational procedures, noting any specific updates for The primary change for the 2015 program year was that administration of the program, including coordination of implementation activity and allocation of funding to participating agencies, transitioned to the Central Arkansas Development Council (CADC) from the Arkansas Community Action Agencies Association (ACAAA). As CADC had already been closely involved in program implementation and coordination of agencies in prior years, the effects of this transition were fairly minimal. In 2015, the Arkansas Weatherization Program (AWP) provided residential energy audits and energy efficiency measure installations to homes whose residents are customers of one or more of the following investor owned utilities (IOUs): American Electric Power Southwestern Electric Power Company (AEP- SWEPCO); Entergy Arkansas, Inc. (EAI); and CenterPoint Energy (CenterPoint). The following IOUs are sponsoring utilities for the AWP and have achieved savings through the program in past years but did not have any customers participate in the program during 2015: Empire District Electric Company (EDEC). Oklahoma Gas and Electric (OG&E); Black Hills Energy; and Arkansas Oklahoma Gas Corporation (AOG). The program is offered in conjunction with the Department of Energy (DOE) Weatherization Assistance Program (WAP), which provides federal assistance to fund the customer co-payment in the AWP for income-qualified households. In Arkansas, the WAP is administered by the Arkansas Energy Office (AEO). If the customer meets the eligibility requirements of the WAP, the weatherization project can be funded by both the WAP and the AWP in order to fully cover the project cost and eliminate the cost to the Process Evaluation Findings 3-6

102 2015 Arkansas Weatherization Program EM&V Report customer. 9 Customers who are not eligible for the WAP are required to provide their own co-pay in order to participate in the AWP and receive the audit and associated measures. Rather than an income requirement, eligibility for the AWP is based on a set of criteria regarding customer residence energy efficiency. In order to qualify, customer homes must meet specific criteria indicating that the residence is severely energy-inefficient. There were no modifications to these criteria for the 2015 program year. Local community action agencies work with customers to enroll in the program and determine AWP and WAP eligibility. In 2015, qualifying AWP projects were completed by the following agencies: Central Arkansas Development Council (CADC); Crowley s Ridge Development Council (CRDC); and Pine Bluff Jefferson County Economic Opportunities Commission, Inc. (PBJCEOC). After the customer is approved and the in-home audit is performed, optimal energy efficiency measures for AWP (and WAP, for eligible customers) are identified through the use of National Energy Audit Tool (NEAT) or Mobile Home Energy Audit (MHEA) software. The measures implemented in participating homes during 2015 include: Ceiling, floor and wall insulation; Air infiltration reduction; Window replacement and storm window installation; Heating and air conditioning replacement; Water heater insulation jackets and pipe wrap; Refrigerator replacement; CFL retrofits; and Smart thermostats. 10 The local agencies conduct onsite audits and install the necessary measures using their internal crews or subcontractors. Audit and installation crews record all relevant measure input data and report it to the Central Arkansas Development Council (CADC), who 9 Eligibility for the Weatherization Assistance Program (WAP) is based on income thresholds, which increase with the number of residents in the home. A description of the WAP, along with the associated income requirements, can be found here: 10 A complete list of all eligible program measures can be found in ACAAA Docket no TF, Attachment A (AWP Modified Program Design and Description). Process Evaluation Findings 3-7

103 2015 Arkansas Weatherization Program EM&V Report aggregates the information from each agency. Batches of data are then sent to Frontier Associates, the program database provider that manages the EnerTrek software tool. EnerTrek incorporates the onsite data into TRM savings formulas (and NEAT/MHEA values for measures not included in the TRM) to calculate ex ante savings for each measure. The resulting savings are made accessible to program utilities and EM&V contractors, who use EnerTrek database exports to conduct measure implementation and savings verification activities. Table 3-5 identifies core program stages and includes key activities performed throughout the program process. The activities and stages shown for 2015 are fairly consistent with those of 2014 and prior years, with modifications to include additional details and clarifications regarding program procedures, and to reflect CADC s role as the program administrator. Table 3-5 Key Activities and Program Stages, 2015 Program Year Program Stage Program Design Planning Training and Implementation Planning Program Promotion Program Participation Data Processing and Monitoring Key Activities Utilities set budgets and savings goals for the program year. Frontier Associates and the participating agencies make any necessary modifications to data collection procedures or program delivery based on TRM changes or other program design changes. Agencies plan their program activity based on expected WAP funding levels and planned AWP funding. Community action agencies, contractors, and other program operations staff attend program-relevant training sessions (primarily for new contractor staff) CADC and local agencies discuss implementation and program updates (primarily to comply with TRM changes). Agencies market the program to local customers who may provide a private co-pay. Agencies enroll customers from the WAP wait list. Utilities answer customer inquiries about the AWP or refer customers to their respective agencies. Customers apply for the AWP and home eligibility is determined. WAP eligibility is determined. Participants receive in-home audits and measures are identified. Contractors install measures that are either stipulated based on NEAT or MHEA software or are agreed upon with the customer (depending on whether or not WAP funds are used for the co-pay). Measure costs and participant tracking data are collected by each agency and reported to CADC. CADC provides periodic cost and participation updates to the utilities. Frontier Associates receives implementation data from CADC and calculates ex ante savings Frontier Associates sends savings data in batches to the utilities. Utilities, CADC, and Frontier Associates have periodic discussions regarding program participation levels and other topics. Process Evaluation Findings 3-8

104 2015 Arkansas Weatherization Program EM&V Report Starting in 2016, the Arkansas IOUs will be implementing individual and joint weatherization programs that comply with the Consistent Weatherization Approach developed by the Arkansas Parties Working Collaboratively (PWC). The Consistent Weatherization Approach will replace the AWP as the statewide weatherization offering implemented by the Arkansas IOUs, and thus the 2015 program year marks the final year of operation for the AWP. 3.4 Arkansas Weatherization Program 2015 Participation In 2015, the Arkansas Weatherization Program conducted energy audits and installed measures in 94 homes. This is a substantial reduction in participation from each of the prior program years (168 homes serviced in 2014, 291 in 2013, 641 in 2012, and 810 in 2011). Table 3-6 displays total participation disaggregated by the community action agency associated with the participant. As with prior years, CADC was the most active agency within the program, completing 73% of projects (CADC completed 76% of AWP projects during the 2014 program year). Table 3-6 Total Participation by Community Action Agency Agency Name Percentage of Participating Homes Central Arkansas Development Council (CADC) 73% Crowley's Ridge Development Council (CRDC) 11% Pine Bluff-Jefferson County Economic Opportunities Commission (PBJCEOC) 16% N 94 The AWP is offered in all investor-owned utility service territories and is funded by participating gas utilities and electric utilities throughout Arkansas. Depending on the location of customers and the fuel sources used in their homes, services for each customer are funded by one gas utility, one electric utility, or both a gas and an electric utility. EAI and SWEPCO were the only electric IOUs with participating homes during 2015, and CenterPoint was the only gas IOU with participating homes. Table 3-7 crosstabulates participation by the gas and/or electric utility associated with the participant. N/A represents projects performed in homes with only one utility source or with a utility service provider that is not part of the AWP. Table 3-7 Participation by Associated Utility, 2015 Electric Utility Gas Utility CenterPoint N/A EAI SWEPCO 9 4 N/A 13 - Process Evaluation Findings 3-9

105 Number of Homes Serviced APSC FILED Time: 5/2/2016 2:38:38 PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc Arkansas Weatherization Program EM&V Report Figure 3-1 displays a comparison between 2015 and 2013 in terms of participation rates by month, based on the installation date included in program tracking data. 11 The number of weatherization projects per month in 2015 was lower than 2014 for all months other than February and December. The most active month of 2015 was February, with 16 participants Program Year (N = 91) 2014 Program Year (N = 159) Figure 3-1 Participation Rates by Month, 2015 vs Participant Survey Results This section highlights key findings from participant surveys for the 2015 program year of the Arkansas Weatherization Program (AWP). The surveys were administered individually to program participants over the phone, and each program participant was given the same survey. The survey primarily focused on participants satisfaction with the program and the perceived benefits of participating. A similar participant survey was administered as part of the 2012 AWP evaluation, and these results from 2015 may be used to compare participant satisfaction with the program over time. In total, 24 program participants responded to the survey. The data collected from this survey provides insight into participants overall program experience, specifically addressing: Customer motivations and awareness of the program; Customer familiarity with energy efficiency; Customer satisfaction; and Customer characteristics. 11 The installation date was not listed for three participants. Process Evaluation Findings 3-10

106 2015 Arkansas Weatherization Program EM&V Report This section highlights key findings related to the above categories, and draws comparisons between the results from the 2012 program evaluation and the current evaluation where appropriate Participant Removal of Measures In order to assess whether the measures reported in program tracking data were still installed at the time of the survey, respondents were asked whether they had removed or replaced any of the equipment or energy efficiency improvements that had been installed through the program. Only one respondent initially indicated that they had removed a measure, stating that they had replaced their new windows with old windows. However, the Evaluators conducted a follow up call with this respondent and found that this had been a miscommunicated response and that the customer had not actually removed any of their measures. Thus, the survey found a measure removal rate of 0% among the participant survey sample Participant Motivations and Familiarity with Energy Efficiency This section details findings related to how participants learned about the Arkansas Weatherization Program and discovering the extent of their prior experience with energy efficiency practices. Table 3-8 illustrates that the majority of participants (54%) heard about the program from friends, family, or other personal acquaintances. This was also the top result for this question during the 2012 evaluation, and suggests that the program has continued to receive positive word-of-mouth marketing within the customer base. Only three respondents (13%) stated that they learned of the program through their local agency, and only two respondents indicated learning of the program online. Process Evaluation Findings 3-11

107 2015 Arkansas Weatherization Program EM&V Report How did you learn of the Arkansas Weatherization Program? Table 3-8 How Participants Learned of the Program Percentage Response of Respondents (N = 24) Information that came in the mail 8% Newspaper or magazine article/ad 4% Contractor 0% Word of mouth from friends, relatives, or others 54% TV ad 0% Radio ad 0% Utility bill message 0% Utility website 8% Other website 0% Local community action agency 13% Other 13% Don't know 4% Respondents were able to provide more than one response for this question. Percentages displayed are percentages of respondents rather than percentages of responses. Therefore, the total exceeds 100%. Participants were then asked about their reasons for participating, and the results are shown in Table 3-9. Although respondents were provided with a list of response options, the majority of respondents selected the option of Other and provided an open-ended response indicating that they participated in the program because their home needed specific improvements. These improvements included insulation, making the house warmer, replacing the air conditioning unit, and reducing air leakage. Among the remaining listed response options, respondents most commonly selected to reduce my monthly electric bill or to reduce my monthly gas bill. Reduction of utility bills was the most common reason for participating found during the 2012 evaluation. Process Evaluation Findings 3-12

108 2015 Arkansas Weatherization Program EM&V Report What is the main reason you decided to participate in the program? Table 3-9 Reasons for Participation Response Percentage of Respondents (N = 24) To reduce my monthly electric bill 17% To reduce my monthly gas bill 21% Save energy 13% AWP paid for some or all of the improvements 13% Recommendation from a friend, relative, neighbor 8% [The house needed specific improvements] 54% Help save the environment 0% Contractor recommendation 0% Community Action Agency 0% Recommended Other 8% It is the right thing to do 0% Respondents were able to provide more than one response for this question. Percentages displayed are percentages of respondents rather than percentages of responses. Therefore, the total exceeds 100%. In order to further understand participants reasons for participating in the AWP, the survey asked a series of questions relating to their understanding of the concept of energy efficiency prior to participation in the program. For these questions, participants responded on a scale of 1 to 5, where 5 is very familiar, 1 is very unfamiliar, and 3 is neutral, i.e. neither familiar or unfamiliar. The findings suggest that the majority of participants consider themselves to have had some familiarity with energy efficiency prior to the start of the program. Table 3-10 shows that the majority of respondents (63%) stated that they were at least somewhat familiar with the installation of various energy efficiency home improvements while Table 3-11 shows that three-quarters of respondents considered themselves to be at least somewhat familiar with various household energy saving activities, such as washing clothes with cold water, changing light bulbs, and adjusting heating systems. These results are similar to those found during the 2012 evaluation. 12 As with the 2012 program year, a greater percentage of respondents reported being familiar with energy saving activities rather than energy saving purchases, which may suggest that these customers are more likely to take no-cost or low-cost actions when attempting to reduce their energy consumption. 12 During the 2012 survey, 52% of respondents stated that they were at least somewhat familiar with energy efficiency improvements, and 67% stated that they were at least somewhat familiar with energy saving behaviors. Process Evaluation Findings 3-13

109 2015 Arkansas Weatherization Program EM&V Report Table 3-10 Participants Past Familiarity with Energy Efficiency Improvements Prior to the audit, how familiar were you with the benefits of installing various energy efficiency improvements similar to those offered by the Arkansas Weatherization Program? Percentage of Response Respondents (N = 24) Very Familiar 21% Somewhat Familiar 42% Neither Familiar or Unfamiliar 0% Somewhat Unfamiliar 12% Very Unfamiliar 17% Don't Know 8% Table 3-11 Participants Familiarity with Energy Savings Activities Prior to the audit, how familiar were you with various household energy saving activities such as washing with cold water, reducing your use of light fixtures, and adjusting heating system settings? Percentage of Response Respondents (N = 24) Very Familiar 33% Somewhat Familiar 42% Neither Familiar or Unfamiliar 0% Somewhat Unfamiliar 17% Very Unfamiliar 0% Don't Know 8% Following this, respondents were asked whether they are now more knowledgeable about energy efficiency than they were before participating in the program. As shown in Table 3-12, approximately two-thirds (67%) of respondents indicated that they are now much more knowledgeable than they were before participating in the program. Only two respondents (8%) stated that they are no more knowledgeable about energy efficiency and energy efficient options than they were previously. These results suggest that although participants considered themselves to be fairly knowledgeable about energy efficiency before participating in the AWP, they also credit the AWP with further increasing this level of knowledge. Table 3-12 Increase in Energy Efficiency Knowledge Following AWP As a result of your experience with the AWP, would you say you are more knowledgeable about energy efficiency and energy efficient options for your home? Response Percentage of Respondents (N = 24) Yes, much more knowledgeable than before participating 67% Yes, somewhat more knowledgeable than before participating 13% Yes, slightly more knowledgeable than before participating 8% No, not more knowledgeable than before participating 8% Don't Know 4% Process Evaluation Findings 3-14

110 2015 Arkansas Weatherization Program EM&V Report Participant Perspectives on Energy Efficiency Upon establishing a baseline of understanding about participants familiarity with energy efficiency, the survey asked participants about their previous, current and potential future involvement with implementation of energy efficiency improvements in their homes. Table 3-13 shows that 46% of respondents claimed to be performing energy-saving activities, which is a similar percentage to that found during the 2012 evaluation. 13 Respondents who reported that they had previously performed energy saving activities were asked to identify these activities, with common responses including turning off lights, washing with cold water, and turning down the thermostat. Table 3-13 Participants Prior Energy Saving Activities Prior to the audit, did you perform any common household energy saving activities? Response Percentage of Respondents (N = 24) Yes 46% No 42% Don't Know 12% Participants were then asked whether they now take additional energy saving actions in their home as a result of participating in the program. As shown in Table 3-14, 88% of program participants stated that they now do this. When asked to elaborate on these activities, participants primarily cited common, no-cost improvements such as washing with cold water, turning down the thermostat, and turning off lights when not in use. The majority of these respondents (53%) reported implementing more than one energy saving action in their home. Table 3-14 Participants Current Energy Saving Activities As a result of your experience with the program, do you now take additional action to save energy in your home? Participant Satisfaction Response Percentage of Respondents (N = 24) Yes 88% No 12% Don't Know 0% This section presents the findings from survey questions geared toward understanding participants satisfaction with the program. Participants were asked about various elements of the program s functioning; the results can be found in Table These elements include the information provided by the agency, the quality of installation work, 13 During the 2012 survey, 54% of respondents indicated that they had performed energy saving activities prior to participating in the program. Process Evaluation Findings 3-15

111 2015 Arkansas Weatherization Program EM&V Report the performance of the equipment installed, and the savings on utility bills. The vast majority of responses show that participants were very satisfied. Other than their overall program experience, respondents provided the most ratings of very satisfied for their improvement in home comfort. Table 3-15 Participant Satisfaction with Selected Program Elements Program Element Information provided by the community action agency The quality of installation work by the contractor The performance of the equipment installed The savings on your monthly utility bills The effort required for the application process The wait-time to receive services Information provided by utilities on how to reduce your utility bill Improvement in home comfort Usefulness of the energy audit Overall program experience Very satisfied Somewhat satisfied Neither satisfied nor dissatisfied Somewhat dissatisfied Very dissatisfied Don't know 67% 21% 4% 4% 4% 0% 24 63% 25% 4% 0% 4% 4% 24 79% 13% 0% 0% 4% 4% 24 67% 17% 8% 4% 0% 4% 24 63% 21% 8% 0% 4% 4% 24 42% 21% 21% 0% 12% 4% 24 63% 25% 8% 0% 4% 0% 24 83% 13% 0% 4% 0% 0% 24 67% 17% 4% 0% 4% 8% 24 88% 8% 0% 0% 4% 0% 24 N As was the case with the 2012 program evaluation, the program element with the lowest average satisfaction was wait time. The three respondents who indicated that they were either somewhat or very dissatisfied with the wait time explained that they had waited either two or three years to receive services through the AWP. This is consistent with program staff comments during the current and prior evaluations, and suggests that wait times have not improved over the course of the program. Overall, the results suggest that respondents are highly satisfied with each element of the program experience, with the exception of a few respondents. It should be noted that the majority of somewhat dissatisfied and very dissatisfied responses were provided by a single respondent, and only three respondents total indicated dissatisfaction with any program elements. Process Evaluation Findings 3-16

112 2015 Arkansas Weatherization Program EM&V Report When asked whether they would recommend the AWP to a friend or family member, all but one respondent (96%) stated that they would do this. The respondent who stated that they would not recommend the program did not provide a specific reason for this response other than stating that they would have to think about it. This further suggests that participants are highly satisfied with the program overall, and that participant satisfaction levels have been maintained over the course of the program since Participant Characteristics This section presents the results from survey questions intended to provide insight into participant and home characteristics, including the age, square footage, heating type, and water heating type of participating homes. Additionally, respondents were asked about the number of bedrooms, bathrooms, showers, and total residents in their homes. Table 3-16 Home Construction Dates When was your home built? Response Percentage of Respondents (N = 24) Before % 1970's 13% 1980's 13% % % % Don't know 16% Refused 0% Table 3-17 Approximate Square Footages of Participant Homes What is the approximate square footage of your home? Response Percentage of Respondents (N = 24) Less than 1,000 17% 1,001-1,500 17% 1,501-2,000 4% 2,001-2,500 4% Greater than 2,500 4% Don't know 50% Refused 4% Process Evaluation Findings 3-17

113 2015 Arkansas Weatherization Program EM&V Report Table 3-18 Number of Bedrooms in Participant Homes How many bedrooms are there in your home? Response Percentage of Respondents (N = 24) 1 4% 2 50% 3 38% 4 8% Don't Know/Refused to Answer 0% Table 3-19 Number of Bathrooms in Participant Homes How many bathrooms are there in your home? Response Percentage of Respondents (N = 24) 1 75% 2 25% 3 0% Don't Know/Refused 0% Table 3-20 Number of Showers in Participant Homes How many showers are there in your home? Response Percentage of Respondents (N = 24) 0 4% 1 75% 2 21% 3 0% Don't Know/Refused 0% Table 3-21 Number of Residents in Home Year-Round How many people live in your home year round, including yourself? Response Percentage of Respondents (N = 24) 1 59% 2 29% 3 4% 4 8% 5 0% 7 0% Don't Know/Refused 0% Process Evaluation Findings 3-18

114 2015 Arkansas Weatherization Program EM&V Report What type of heating system do you have in your home? 0% 4% 25% 71% Natural gas heating Electric heating Combination of types Other (N = 24) Figure 3-2 Types of Heating Systems in Participant Homes What type of water heater do you have in your home? 4% 42% 54% Natural gas water heater Electric water heater Other (N = 24) Figure 3-3 Types of Water Heaters in Participant Homes Process Evaluation Findings 3-19

115 2015 Arkansas Weatherization Program EM&V Report 3.6 Program Staff Interviews As part of the evaluation of the 2015 Arkansas Weatherization Program, the Evaluators conducted in-depth interviews with utility staff representing four of the participating IOUs. These interviews primarily served to assess the status of previous evaluation conclusions and recommendations, as well as to identify notable changes in program operation, delivery, and performance. As 2015 marks the final year of AWP operation before the Arkansas IOUs implement weatherization services as part of the Consistent Weatherization Approach framework, the interviews did not focus on further changes to the AWP but program staff provided feedback on the upcoming core framework. This section presents key findings and issues identified through these interviews and through mid-year discussions with CADC staff Program Efficiency and Performance Minimal Effects from CADC Transition: When asked about the transition from ACAAA to CADC as the program administrator, utility staff explained that the effects of this change have been minimal. One staff member noted that from their perspective, nothing about the operational structure had changed during the 2015 program year. Staff generally did not have much feedback regarding CADC s performance as the program administrator, but noted that CADC had continued to be the most active agency in performing weatherization for the program during Continued Prioritization of WAP Funding: As with prior years staff acknowledged the challenges that have emerged and persisted due to the AWP s relationship with the Weatherization Assistance Program (WAP). The Arkansas Energy Office (AEO), which administers the WAP, has directed the agencies to follow a specific set of rules in order to comply with WAP procedures. The participating agencies have continually been directed to prioritize LIHEAP funding over AWP funding when implementing weatherization projects; remaining LIHEAP funds were set to expire on September 30 th, This has continued to work as a barrier to the AWP and was not resolved during Advancing Funds to the Agencies: CADC noted that some agencies have been unable to complete jobs through the AWP because they were concerned about being reimbursed for the work performed. As CADC typically provides funds to the agencies after the work is completed, some agencies preferred to conduct work under the WAP only. In order to address this, CADC staff requested that the utilities allow CADC to advance funds to the agencies. The purpose of this would be to provide the agencies with immediate funds that they could use to provide services, without having to rely on reimbursements from either the state or the AWP utilities. The utilities explored this possibility with CADC but ultimately it did not have any significant effects on agencies ability to complete jobs under the AWP. Process Evaluation Findings 3-20

116 2015 Arkansas Weatherization Program EM&V Report Continued Program Decline: When asked about program performance in 2015 as compared with prior years, utility staff emphasized the fact that participation rates have steadily decreased during recent years and that very few significant program improvements had been made over time. Several utility staff members reported that their expectations for AWP performance were low or non-existent for 2015, and at least one utility received commission approval to move its funds away from the program in order to focus its other program offerings. Overall, utility staff noted that they have already shifted their focus to the Consistent Weatherization Approach, and stated that there seemed to be an overall lack of interest in the AWP on the part of program staff for Data Quality and Availability Minor Improvements in Data Accuracy: When asked about the quality of data provided to them in 2015, utility staff noted that the quality had either remained the same as 2014 or had only improved slightly. One utility staff member noted that there appeared to be fewer errors in the data during 2015 but that this could be due to the small participant population, as fewer errors would exist in a smaller data set. The level of detail in periodic reporting from CADC did not increase significantly, but one utility staff member noted that they did receive information about costs and the location of participant homes each month. Additionally, the utilities received periodic batches of ex ante measure-level savings data from Frontier throughout the year. Overall fewer data revisions and error corrections occurred during the 2015 program year Communication and Collaborative Efforts Agency Roles in Upcoming Programs: Utility staff noted that as the Consistent Weatherization Approach framework allows the utilities to use community action agencies as audit and installation contractors, they may continue to work with the agencies moving forward. Two utility staff members explained that they have already begun recruiting the agencies into their weatherization programs. One staff member stated that they have reached out to six agencies and that one of the agencies has expressed interest in participating, while the other has had two agencies (Crawford Sebastian and CADC) attend their contractor kick-off meeting. Consistent Weatherization Approach Development: When asked about communications surrounding the Consistent Weatherization Approach, utility staff stated that this process has been beneficial and that it has provided a mechanism by which the utilities are able to corroborate. Staff noted that the development of the Consistent Weatherization Approach framework has resulted in additional joint program partnerships and will hopefully lead to a more coordinated approach to weatherization. Process Evaluation Findings 3-21

117 2015 Arkansas Weatherization Program EM&V Report 3.7 Tracking Database Review As with prior years, Frontier Associates develops and maintains EnerTrek, the software tool that is used to store participant data and to calculate measure level savings based on collected inputs and TRM formulas. EnerTrek includes a full list of all participants, the measures that were installed in their homes, and the kwh and Therms savings associated with each measure. During the 2015 program year, the Evaluators received periodic tracking data updates as well as final tracking exports. The Evaluators previously reviewed program tracking data in 2014 in order to assess its compliance with Protocol A of TRM V5.0, which specifies that tracking data should be checked for: Participating Customer Information; Measure Specific Information; Vendor Specific Information; Program Tracking Information; Program Costs; and Marketing & Outreach Activities. The Evaluators conducted a review of each of the above factors within the 2015 program tracking data with the exception of marketing and outreach activities as these are outside the scope of EnerTrek reporting. Each of these factors was assessed individually based on the guidelines stated in TRM V5.0. Overall, the Evaluators conclude the following regarding tracking data completeness: The tracking data contained names, addresses, and contact information for all participants, and contained contact information for all but one participant. All participants were listed with a Job ID number. Additional participant information present in the tracking data included gas and electric utility provider designations and utility account numbers. All participant records included the name of the agency that implemented the weatherization services, and all but three records included the date of measure installation. The tracking data included project level costs for each home. The exports received by the Evaluators did not include measure-level costs. Premise characteristics such as home heating type, cooling type, construction date, baseline measurements, and attic square footage were present for all participants where necessary. Process Evaluation Findings 3-22

118 2015 Arkansas Weatherization Program EM&V Report 3.8 Comprehensiveness Factors The Arkansas Public Service Commission has in place a set of criteria in order to determine whether a DSM portfolio qualifies as Comprehensive. This section provides updates to the review of the Arkansas Weatherization Program that was conducted by the Evaluators in prior years in relation to each factor. As the AWP is one component of the larger utility energy efficiency program portfolios, a broader perspective is necessary in order to determine how well it is serving its intended role in those groups of programs. Utility annual reports and portfolio evaluations may present the AWP within the context of these broader energy efficiency portfolios. This section focuses on the comprehensiveness factors as they relate to the AWP on the program-level. Additionally, as there were few changes to program design and operation during the 2015 program year, this review uses the prior comprehensiveness findings as a baseline and provides updates where appropriate. Factor 1: Whether the programs and/or portfolio provide, either directly or through identification and coordination, the education, training, marketing, or outreach needed to address market barriers to the adoption of cost-effective energy efficiency measures; o Assessment of Education The AWP has continued to implement educational efforts towards its prospective participants and other customers, although these efforts declined substantially during Educational efforts offered by the program have included: Providing educational materials (energy audit, brochures, demonstrations) Providing education targeted to specific market barriers (emphasizing increased comfort and safety levels as a benefit of energy efficiency) The program did not excel in the following components: Providing outreach through multiple channels. A few agencies continued to promote the program but overall outreach declined substantially during Providing coordinated education from multiple entities. Each agency and some utilities provide this, but based on interviews with agency and utility staff, the coordination could be improved substantially. Lack of coordination during 2015 is most likely attributable to the fact that several of the IOUs and agencies did not focus on the AWP as a major method of achieving energy savings. Process Evaluation Findings 3-23

119 2015 Arkansas Weatherization Program EM&V Report o Assessment of Training The active community action agencies have continued to participate in multiple training courses throughout the year. This includes training related to program updates and data requirements, as well as training that leads to residential audit and installation certifications. These courses maintain contractor skill levels and ensure that agency services comply with up-todate audit and installation requirements. However, due to issues identified during the on-site verifications, it appears that there are significant opportunities to train some staff members on proper installation techniques and proper materials to use when performing this work. o Marketing and Outreach Marketing for the program during 2015 was very minimal. Due to continual waiting list issues and the program ending after 2015, marketing was not a priority for the program and increased marketing may not have improved program performance. The program therefore did not excel in the following criteria: Performed through several channels. Overall outreach declined substantially during Promoted by trade allies (agencies and their contractors). Program promotion was minimal during 2015, with most agencies prioritizing the WAP over the AWP. Address specific barriers. One major barrier to AWP participation has been that customers who are able to provide their own co-payment do not commonly participate. With minimal outreach conducted, this barrier persisted throughout Factor 2: Budgetary, Management, and Program Delivery Resources Although utility budget allocations to the AWP have historically been sufficient to fund the targeted number of homes, the AWP has continually experienced challenges in meeting program goals due to organizational and program delivery issues. As utilities began shifting their funds away from the AWP, the program was left with fewer operational resources. Additionally, funding issues within the WAP have constrained the AWP s participation potential and effectively reduced program resources. Factor 3: Addressing Major End-Uses The measure list available to the AWP did not change in The AWP offers a wide range of measures, which are chosen based on cost-effectiveness testing Process Evaluation Findings 3-24

120 2015 Arkansas Weatherization Program EM&V Report through NEAT and MHEA. The list of eligible program measures covers all major end-uses for targeted customer homes, including: o HVAC systems; o Equipment tune-ups; o Hot water measures; o Appliances (refrigerators); o Safety measures (smoke detectors); o Lighting; and o Building envelope measures. 14 The whole house approach to participant home improvements is conducive to providing a comprehensive set of measures in each home. Factor 4: Comprehensively Addressing Customer Needs The AWP is designed to comprehensively address the major needs of its participants by providing the following benefits: o Technical assistance through in-home audits; o Energy and monthly bill savings through measure installation; and o Increased comfort and/or safety for participants. Although the AWP is able to provide these benefits to customers who participate in the program, there remain a large number of utility customers who are in need of such services but whose participation has been delayed due to the program s operational issues. Participants who provide their own private co-pay for the audit and energy efficiency measures may choose to receive a less comprehensive set of services as they are allowed to select individual measures. These participants are encouraged to install the full set of recommended items, but comprehensiveness within measure installation is not required by the program in these cases. Factor 5: Targeting Market Sectors & Leveraging Opportunities The AWP focuses on a specific market of utility residential customers whose homes are severely energy inefficient. The AWP also involves utility partnerships and is intended to provide cross-fuel coordination rather than focusing only on gas or electric savings in isolation. This program is intended to amplify the benefits of the statewide Weatherization Assistance Program (WAP) in order to provide additional services to customers who have substantial weatherization needs. 14 A complete list of eligible AWP measures can be found in program filing and planning documentation such as Attachment A, (AWP Modified Program Design and Description), of ACAAA Docket no TF. Process Evaluation Findings 3-25

121 2015 Arkansas Weatherization Program EM&V Report Thus, in theory, the program leverages WAP resources and is delivered through the same channels as the WAP. Factor 6: Cost-Effectiveness of Energy Efficiency There have been no significant improvements to program cost-effectiveness for Although the program is designed to cost-effectively generate net savings and meet the stated annual program goals, it has been unable to meet the annual goals thus far. Cost-effectiveness has varied widely among utilities in prior years. The AWP has successfully met industry standards for net-to-gross levels, as the Evaluators have determined that it calls for a net-to-gross ratio of 1. However, in terms of cost-effectiveness and savings goals, the AWP has not excelled. Factor 7: Adequacy of EM&V Procedures The AWP was reviewed for EM&V procedures in the following areas: o QA/QC and EM&V procedures conducted by utility staff; o QA/QC and EM&V procedures conducted by installation contractor staff; and o QA/QC and EM&V procedures conducted by the Evaluators. The onsite QA/QC procedures currently conducted by utility staff and agency staff are adequate in most cases. During onsite field verification visits during 2015, the Evaluators found that the reported installation data was fairly accurate and matched actual observed conditions for the majority of measures, but that there were significant quality of work issues with two homes. It is unclear whether this is an indication of systematic problems, but sufficient training and quality control should have identified or prevented these issues. The issues that were identified during these site visits are detailed in Section 2.6. Tracking data errors have been for the most part resolved in 2015, and the current version of the tracking database within EnerTrek contains the necessary information to comply with TRM V4.0 requirements. 15 The Arkansas Weatherization Program meets several of the comprehensiveness requirements, but the program has struggled to achieve success and was not able to fully resolve any of its major operational issues during the past program cycle or recent bridge years. As previously noted, utility annual reports and other portfolio-level assessments may provide a more comprehensive view of how the AWP fits into the larger context of the sponsoring utilities energy efficiency program portfolios. 15 See Section 2.8 of this report for detailed information regarding the program tracking data review. Process Evaluation Findings 3-26

122 4. Conclusions and Recommendations After reviewing the Arkansas Weatherization Program for 2015, the Evaluators highlight the following conclusions: Minimal Effects from CADC Transition: The transition from ACAAA to CADC as the program administrator does not appear to have mitigated the AWP s operational or performance issues. It appears that CADC made efforts to work with the AEO and increase agency involvement with the program, but any beneficial effects of these efforts were for the most part overshadowed by the program s decreased activity overall. Quality of Work Issues: The Evaluators visited two participating homes that received weatherization services from PBJCEOC during The Evaluators identified issues with the quality of work performed, and one of the homeowners indicated that the agency contractors had chipped paint in their home and had not been considerate of the home in general. The Evaluators found that silver bubble wrap had been used as water heater tank insulation in one home and that water heater pipe wrap had been installed incorrectly in both homes. It is unclear whether these quality of work issues are limited to this agency or are indicative of a larger problem, and agency staff responded to the findings indicating that the silver bubble wrap is no longer being used and that the pipe wrap had been installed properly. However it may be beneficial to conduct further training with agency staff in order to ensure that they are complying with industry quality standards, and that they are providing adequate customer service to participants. Minor Improvements in Data Accuracy: Tracking data errors have been for the most part resolved in 2015, and the current version of the tracking database within EnerTrek contains the necessary information to comply with TRM V4.0 requirements. Overall Frontier Associates has been very responsive to data requests and provided the utilities with fairly accurate batches of data throughout the program year. There were fewer tracking data issues in 2015 as compared to Continued WAP Reliance Issues: As with prior years, program staff acknowledged the challenges that have emerged and persisted due to the AWP s relationship with the Weatherization Assistance Program (WAP). Ideally, this arrangement would use utility funds to efficiently leverage federal funding and substantially increase the number of weatherization projects that the agencies are able to perform. However, the AWP s inherent link to the WAP has continued to result in performance issues due to federal funding reductions. Additionally, the participating agencies were directed to prioritize LIHEAP funding over AWP funding when implementing weatherization projects, which is a key barrier to AWP program activity. Decreasing Program Activity: The number of participants and the resulting savings levels for the AWP have steadily decreased since the 2011 program year. A major contributing factor to this decline in program activity is likely the fact that the program was Conclusions and Recommendations 4-1

123 2015 Arkansas Weatherization Program EM&V Report winding down in 2015 and the IOUs were already focusing efforts on the Consistent Weatherization Approach. However other issues including variable agency engagement in weatherization services, inconsistent availability of WAP funding, and insufficient interest from private co-pay customers have historically limited the program s performance. Upcoming Consistent Weatherization Approach: The new weatherization framework developed by the utilities and other stakeholders has established statewide weatherization procedures and services, and will be implemented beginning in Utility staff reported that they anticipate that this Consistent Weatherization Approach will be a more effective method of meeting the state s weatherization needs. Additionally, utility staff noted that the collaborative relationship among utilities has improved during the development of the new framework and that the core framework will hopefully lead to a more coordinated approach to weatherization in the state. Although 2015 marks the final year of operation for the AWP as it currently stands, the Evaluators provide the following recommendations that the utilities or agencies may consider when moving forward with weatherization services under the Consistent Weatherization Approach framework: Mitigate Quality of Work Issues: The quality of work issues identified by the Evaluators during on-site verifications during 2015 suggest that additional verification and training may be needed for contractors in the Pine Bluff region. Overall, the Evaluators suggest that additional quality assurance and training be conducted with any new contractors who are brought onto the IOUs Consistent Weatherization Approach offerings. Record and Report Air Infiltration Details: As with prior years, the Evaluators identified discrepancies between reported air infiltration leakage rates and verified air infiltration leakage rates. Although only nine homes received blower door testing as part of the 2015 evaluation, the majority of these homes showed verified infiltration rates that were higher than reported infiltration rates. Moving forward, the Evaluators recommend that the IOUs and contractors collect and report the itemized air infiltration measures that are installed. As it is very difficult to reliably replicate blower door results during a site visit, having this additional information will allow program staff or their EM&V contractors to verify that the work was performed properly. Conclusions and Recommendations 4-2

124 2015 Arkansas Weatherization Program EM&V Report The Evaluators identified significant issues with the quality of work in two homes during the on-site verification efforts Table 4-1 Recommendations from 2015 Program Year Evaluation Issue Consequences Recommendation Some measures were not eligible for savings The reported air infiltration leakage rates appear skewed downward, based on the Evaluators site visits. Negatively affects customer satisfaction Possible issues with measure implementation or data collection Possible discrepancies between implementation and verification that will lead to skewed realization rates. Contractors joining Consistent Weatherization Approach offerings, namely in the Pine Bluff area, should receive additional training and undergo quality control procedures that ensure sufficient customer service and installation of measures. Record and report itemized air infiltration measures in the tracking data so that it is possible to verify individual measure elements. Also, Include any field notes related to the blower door test in the tracking data so that testing conditions can be more accurately replicated. Conclusions and Recommendations 4-3

125 Appendix A: Participant Survey Instrument Arkansas Weatherization Program Participant Telephone Survey ID No. Customer Name: Date of interview: Date data entered... Hello. May I please speak with [CONTACT NAME]: )? Hello. My name is and I m calling from [Surveying Company Name] on behalf of the Arkansas gas and electric utilities about the Arkansas Weatherization Program your household participated in during Through this program your home received items such as attic insulation, air sealing, light bulbs, and other energy saving measures. Are you the person who is most familiar with your household s participation in this program? (IF NOT RIGHT PERSON) May I please speak to the person who would know the most about your household s participation in this program? REPEAT INTRODUCTION AND CONTINUE (IF RIGHT PERSON) We are conducting a study to evaluate the Arkansas Weatherization Program, known as the AWP. AWP and community agency staff will use the results of this evaluation to determine the effectiveness of the program and to make improvements. We would like to include your opinions about the program in our evaluation. The interview will take approximately 10 minutes. May I ask you some questions about the work performed? Your responses will remain completely confidential. Q-1 Our records indicate that you participated in the Arkansas Weatherization Program in 2015 by having an energy audit completed and receiving several energy efficient items installed in your home. Do you recall participating in this program? Yes [SKIP TO Q-4] No [GO TO Q-2] Don t know [GO TO Q-2] Q-2 Is there anyone else in your household who may be familiar with your household s participation in the program? Yes [GO TO Q-3] No [THANK RESPONDENT AND TERMINATE INTERVIEW] Don t know [THANK RESPONDENT AND TERMINATE INTERVIEW] Appendix A: Participant Survey Instrument A-1

126 2015 Arkansas Weatherization Program EM&V Report Q-3 May I speak with that person? Yes [RETURN TO Q-1 AND BEGIN QUESTIONS WITH NEW RESPONDENT] No [THANK RESPONDENT AND TERMINATE INTERVIEW] Don t know [THANK RESPONDENT AND TERMINATE INTERVIEW] RESPONDENT BACKGROUND As a reminder, your responses to this survey will be kept completely confidential. I ll begin with a few questions about your decision to participate in the program. Q-4 How did you learn of the Arkansas Weatherization Program? [SELECT ALL THAT APPLY] Information that came in the mail Newspaper or magazine article/ad Contractor Word of mouth from friends, relatives, or others TV ad Radio ad Utility bill message Utility website Other website Local community action agency Other (Specify) Don t know [DO NOT READ] Q-5 What is the main reason you decided to participate in the program? [SELECT ALL THAT APPLY] To reduce my monthly gas bill To reduce my monthly electric bill The AWP paid for some or all of the improvements Contractor recommendation Utility recommendation or information (Specify which utility) Recommendation from a friend, relative, neighbor Community action agency recommendation It is the right thing to do Help save the environment Save energy Other (Specify) Q-5A Of the things you mentioned, which was the most important? To reduce my monthly gas bill To reduce my monthly electric bill Appendix A: Participant Survey Instrument A-2

127 2015 Arkansas Weatherization Program EM&V Report The AWP paid for some or all of the improvements Contractor recommendation Utility recommendation or information (Specify which utility) Recommendation from a friend, relative, neighbor Community action agency recommendation It is the right thing to do Help save the environment Save energy Other (Specify) MEASURE INSTALLATION Next, I have some questions about the work that was performed in your home through the Arkansas Weatherization Program. Q-6 Since the work was performed, have you removed or replaced any of the equipment or energy efficiency improvements implemented in your home through the program? Yes (Please specify which items have been removed or replaced): No Don t know [IF Q-6 = Yes, Ask Q-7. Otherwise skip to Q-8] Q-7 Why did you remove or replace these items? [SELECT ALL THAT APPLY] They were no longer working properly I purchased new items that I liked better I liked my old items better so I reinstalled them I performed some remodeling or maintenance that required the removal of these items Other: Don t know OVERALL ENERGY EFFICIENCY DECISION MAKING Q-8 Based on your experience with the Arkansas Weatherization Program, would you recommend the program to a friend or family member? Yes [SKIP TO Q-9] No [ASK Q-8A] Q-8A Why wouldn t you recommend the Arkansas Weatherization Program to a friend or family member? [OPEN-ENDED] Appendix A: Participant Survey Instrument A-3

128 2015 Arkansas Weatherization Program EM&V Report Q- 9 Prior to the audit, how familiar were you with the benefits of installing various energy efficiency improvements similar to those offered by the Arkansas Weatherization Program? 5: Very familiar 4: Somewhat familiar 3: Neither familiar nor unfamiliar 2: Somewhat unfamiliar 1: Very unfamiliar 99: Don t know Q-9A Prior to the audit, how familiar were you with various household energy saving activities such as washing with cold water, reducing your use of light fixtures, and adjusting heating system settings? 5: Very familiar 4: Somewhat familiar 3: Neither familiar nor unfamiliar 2: Somewhat unfamiliar 1: Very unfamiliar 99: Don t know Q-9B Prior to the audit, did you perform any common household energy saving activities? If so, which activities? Yes (please explain): No Don t know Q-10 As a result of your experience with the Arkansas Weatherization Program, how much more knowledgeable would you say you are about energy efficiency and energy efficient options for your home? Much more knowledgeable than before participating Somewhat more knowledgeable than before participating Slightly more knowledgeable than before participating No more knowledgeable than before participating Don t know Q-11 As a result of your experience with the program, do you now take additional action to save energy in your home, such as wash with cold water, reduce your use of light fixtures, and adjust heating system settings? Yes (please explain): No Don t know PROGRAM SATISFACTION Appendix A: Participant Survey Instrument A-4

129 2015 Arkansas Weatherization Program EM&V Report Now I d like to ask you about your satisfaction with several aspects of this program. Q-12 On a scale of 1 to 5, where 5 is very satisfied and 1 is very dissatisfied, and a 3 is neutral, how would you rate your satisfaction with the following? [RECORD AS 99 IF DON T KNOW] Element of Program Experience Information provided by the community action agency The quality of installation work The performance of the equipment installed The savings on your monthly utility bills The effort required for the application process The wait-time to receive services Information provided by utilities on how to reduce your utility bill Improvement in home comfort Usefulness of the energy audit Overall program experience Very Satisfied Somewhat Satisfied Neither Satisfied or Dissatisfied Somewhat Dissatisfied Very Dissatisfied Don't Know [5] [4] [3] [2] [1] [99] Q-13 (If any item in Q-12 rated 2 or 1) Why were you dissatisfied with [Program Element]? [VERBATIM]: Q-14 Are there any changes or improvements you would like to see for the Arkansas Weatherization Program? [VERBATIM]: DEMOGRAPHICS Finally, I have a few questions about your household. As a reminder, your responses will remain confidential. Appendix A: Participant Survey Instrument A-5

130 2015 Arkansas Weatherization Program EM&V Report Q-15 When was your home built? [IF RESPONDENT DOES NOT GIVE VERBATIM ANSWER, READ OFF YEAR RANGES UNTIL RESPONDENT INDICATES ONE] Verbatim Before 1970 s 1970 s 1980 s or newer Don t know [DON T READ] Refused Q-16 What is the approximate square footage of your home? [IF RESPONDENT DOES NOT GIVE VERBATIM ANSWER, READ OFF SIZE RANGES UNTIL RESPONDENT INDICATES ONE] Verbatim Less than 1,000 1,001-1,500 1,501-2,000 2,001-2,500 Greater than 2,500 Don t know [DON T READ] Refused Q-17 How many bedrooms are there in your home? Quantity: Don t know [DON T READ] Refused Q-18 What type of heating system do you have in your home? Natural gas heating Electric heating Combination of types (Specify): Other (Specify): Don t know [DON T READ] Q-19 What type of water heater do you have in your home? Natural gas water heater Electric water heater Other (Specify): Don t know [DON T READ] Q-20 How many bathrooms are there in your home? Quantity: Appendix A: Participant Survey Instrument A-6

131 2015 Arkansas Weatherization Program EM&V Report Don t know [DON T READ] Refused Q-21 How many showers are there in your home? Quantity: Don t know [DON T READ] Refused Q-22 Including yourself, how many people currently live in your home year-round? Quantity: Don t know [DON T READ] Refused Q-23 Do you have any other comments that you would like to relay to Arkansas Weatherization Program staff about energy efficiency in residences or about these programs in general? [VERBATIM] This completes the survey. Your input is greatly appreciated.thank you very much for your time! Appendix A: Participant Survey Instrument A-7

132 Attachment C: ADM s Evaluation of the OG&E/AOG Weatherization Program

133 Evaluation of 2015 AOG/OG&E Weatherization Program Submitted to: Arkansas Oklahoma Gas Corporation Oklahoma Gas and Electric March 2016 Prepared by: ADM Associates, Inc.

134 Prepared by: Brian Harold Adam Thomas Kevin Halverson Corporate Headquarters: 3239 Ramos Circle Sacramento, CA Tel: (916) ADM Associates Inc. Energy Research & Evaluation 200 Brown Road Suite 208 Fremont, CA Tel: (510)

135 Acknowledgements We would like to thank the staff at both Arkansas Oklahoma Gas (AOG) and Oklahoma Gas and Electric (OG&E) for their time and effort in contributing to the evaluation, measurement, and verification (EM&V) of the AOG/OG&E Weatherization Program. This evaluation required regular communications with staff at AOG and OG&E, who were very responsive to evaluation requests and actively participated in interviews and other discussions. Additionally, we would like to thank AOG and OG&E participating customers, program contractors, and program tracking database management staff for their cooperation and assistance throughout the evaluation. We would also like to thank Independent Evaluation Monitor staff for their active involvement in providing thorough answers and clarification to the evaluation team in response to higher-level questions that arose throughout the EM&V effort.

136 TABLE OF CONTENTS Section Title Page 1. Executive Summary Impact Evaluation Findings Process Evaluation Findings Conclusions and Recommendations Appendix A: Participant Survey Instrument... A-1 i

137 LIST OF TABLES Title Page Table 1-1 Gross Realized Gas Savings, AOG Table 1-2 Gross Realized Electricity Savings, OG&E Table 1-3 Gross Realized Savings, Non-Program and AOG Homes Paid by OG&E Table 1-4 Overall Gross Realization Rates by Measure Table 1-5 Net Realized Gas Savings, AOG Table 1-6 Net Realized Electricity Savings, OG&E Table 1-7 Recommendations from 2015 Program Year Evaluation Table 2-1 Ex Ante Savings by Measure, AOG Table 2-2 Ex Ante Savings by Measure, OG&E Table 2-3 Ex Ante Savings by Measure Non-Program and AOG Homes Paid by OG&E Table 2-4 TRM Sections by Measure Type Table 2-5 Deemed Savings Values for Air Infiltration Reduction, Zone Table 2-6 Deemed Savings Values for R-38 Ceiling Insulation, Zone Table 2-7 Gross Realized Gas Savings, AOG Table 2-8 Gross Realized Electricity Savings, OG&E Table 2-9 Gross Realized Savings, Non-Program and AOG Homes Paid by OG&E Table 2-10 Overall Gross Realization Rates by Measure Table 2-11 Measures Eligible for Spillover Savings Table 2-12 Net Realized Gas Savings Attributable to AOG Table 2-13 Net Realized Electricity Savings Attributable to OG&E Table 2-14 Net Realized Gas Savings, Non-Program and AOG Homes Paid by OG&E Table 2-15 Net Realized Electricity Savings, Non-Program Table 3-1 Determining Process Evaluation Timing Table 3-2 Determining Process Evaluation Conditions Table 3-3 Interview and Survey Data Collection Summary ii

138 Table 3-4 Status of 2014 Evaluation Recommendations Table 3-5 Participation by Associated Utility Table 3-6 Total Implementations by Measure Table 3-7 How Participants Learned of the Program Table 3-8 Motivations for Participating in the Program Table 3-9 Prior Customer Awareness of Energy Efficiency Measures Table 3-10 Prior Customer Awareness of Energy Saving Behaviors Table 3-11 Prior Customer Involvement with Energy Saving Behaviors Table 3-12 Post-participation Familiarity with EE Measures and Behaviors Table 3-13 Post-participation Energy Efficiency Behaviors Table 3-14 Participant Satisfaction with Selected Program Elements Table 3-15 Reported Age of Participant Homes Table 3-16 Reported Square Footage of Participant Homes Table 3-17 Other Reported Participant Residence Characteristics Table 4-1 Recommendations from 2015 Program Year Evaluation iii

139 LIST OF FIGURES Title Page Figure 3-1 Homes Serviced by Month, Figure 3-2 Reported Participant Residence Heating Type Figure 3-3 Reported Participant Residence Water Heating Type iv

140 1. Executive Summary The purpose of this report is to provide the methodology and results of the evaluation effort for the 2015 AOG/OG&E Weatherization Program. This evaluation was conducted by ADM Associates (referred to in this report as the Evaluators). This report provides the results of both the impact evaluation and process evaluation activities, presenting verified savings results and tracking program performance and changes in program delivery since the prior program year. While this report provides a review of previous program findings and recommendations and additional conclusions and recommendations based on the limited process evaluation conducted for the current program year, a full process evaluation of the program can be found in the 2012 AOG/OG&E Weatherization Program Evaluation Report. 1.1 Overview of AOG/OG&E Weatherization Program The overall structure and delivery of the AOG/OG&E Weatherization Program has remained fairly constant throughout This section provides a summary of current program design features and procedures, noting any differences between 2015 and prior years. Program procedures and participation stages have remained unchanged since the 2014 program year. Participating homes were evaluated in order to determine cost-effective energy efficiency measures that would improve overall building efficiency and reduce residential energy usage. The program provided funds for the installation of various measures, including insulation, lighting, air infiltration, and water heater jacket and pipe wrap. Weatherization of participating homes is funded by AOG and/or OG&E, depending on the home s utility service provider, in this co-funded program. In 2015, the utilities covered up to $3,500 of services in participant homes, an increase over the $3,000 limit for 2014 and prior years. Eligible OG&E customers include homeowners or leaseholders of a single family home, duplex condos, townhouses or mobile home constructed prior to Participants must meet three of the following eligibility criteria: 2 Attic insulation less than or equal to R-22; Wall insulation equal to or less than R-4; 1 The home age requirement will be updated to a rolling 10 year age minimum beginning in Eligibility requirements are taken from AOG informational materials. Obtained from: Executive Summary 1-1

141 2015 AOG/OG&E Weatherization Program EM&V Report Floor insulation equal to R-0; Single pane windows with no storm windows attached; Heating system less than or equal to 78% AFUE; Cooling system with SEER of 10 or less; and Air infiltration problems identified through either a pre-blower door test or visual inspection procedures. 3 These criteria are designed to target severely energy inefficient residences; this helps to ensure that each participating home has the potential to generate a substantial amount of energy savings through the program. In 2015 AOG and OG&E sought to provide services to larger homes in order to achieve a higher level of savings per home. During 2015 approximately 5% of participating homes were greater than 3,000 square feet, and the average square footage of participant homes was 1,753. The enrollment procedures for 2015 were consistent with prior years. Customers who are interested in participating in the program contact program staff members to sign up for the in-home audit. Additionally, prospective participants may learn about the program and apply for enrollment through a local community clearinghouse. As with prior program years, the program uses three installation contractors who perform the weatherization and measure implementation services. After the measures are installed, utility staff members perform post-inspections in order to verify that all measures have been properly implemented. 1.2 Evaluation Objectives The evaluation of the 2015 AOG/OG&E Weatherization Program consisted of a program savings impact analysis and a limited process evaluation. The primary objectives of this evaluation were to verify reported program savings through TRM verification, and to track program performance characteristics and operational changes since the prior program year. The evaluation activities conducted for the 2015 program year include: Review of deemed savings calculations. The Evaluators used the Arkansas Technical Reference Manual, Version 5.0 (TRM) to verify savings calculations for each implemented measure type in order to ensure that ex ante measure savings were properly calculated according to TRM protocols. 3 Measured air infiltration is measured at 50 Pascal (Pa) and must be greater than 2,200 CFM for households of five or fewer occupants, or greater than 2,700 CFM for households of more than five occupants. Executive Summary 1-2

142 2015 AOG/OG&E Weatherization Program EM&V Report Tracking database and documentation review. The Evaluators conducted a tracking database review according to the guidelines defined in Protocol A of the TRM. On-site field verification. The Evaluators scheduled and conducted site visits to participant homes in order to verify complete and proper measure installation, to conduct post-implementation measurements, and to verify home characteristics such as heating and water heating fuel type. Program staff interviews. Interviews were conducted with utility staff who are responsible for designing and managing the program. These interviews primarily served to assess the status of previous evaluation conclusions and recommendations, as well as to identify notable changes in program operation, delivery, and performance. Participant surveying. Telephone surveys were conducted with a sample of program participants in order to collect data regarding customer satisfaction, participant characteristics, and to identify any issues with program operation or delivery from the customer perspective. 1.3 Summary of Findings This section summarizes the key findings from the on-site verification, impact evaluation, and limited process evaluation Measure Verification Findings The Evaluators conducted onsite verification visits to 58 participant homes, supplemented by 10 telephone verifications. These site visits were conducted in order to verify complete and proper measure installation, to conduct post-implementation measurements, and to collect data regarding participant residence characteristics. The onsite field verification showed that the weatherization measures had for the most part been installed in the quantities reported within program tracking data. Specific notes illustrating the accuracy of program tracking data include: Contact information: All residences were located at the addresses provided within the tracking data. Although the majority of telephone numbers were found to be accurate during the appointment scheduling and field visit activities, the Evaluators identified a few telephone numbers during the appointment scheduling process that were disconnected or did not belong to a program participant. However this was the case for only nine percent of attempted calls, which is an improvement over the approximately 30% of disconnected or incorrect numbers during the 2014 program evaluation. Executive Summary 1-3

143 2015 AOG/OG&E Weatherization Program EM&V Report Air infiltration: For the 23 homes receiving blower door testing for air infiltration during verification site visits, the reported CFM leakage value was within 10% of the measured leakage value in approximately 60% of cases. The reported CFM leakage value was within 25% of the measured leakage value in approximately 80% of cases. There were three instances where measured leakage was more than 50% greater than reported leakage. Measured leakage was greater than reported leakage in approximately 65% of cases. Ceiling insulation: All reported instances of ceiling insulation were verified. Any identified discrepancies between reported insulation levels and measured insulation levels were very minor and infrequent. Additionally, there were no instances where the reported insulation square footage differed significantly from the observed insulation square footage. CFLs: All reported instances of CFL installation were verified. There were no significant differences between the reported quantity and verified quantity of CFLs installed, after accounting for the 97% in-service-rate specified in the TRM. Water heater measures: All reported instances of water heater jacket and pipe insulation were verified. As with the 2014 and 2013 program years, the measure implementation data reported by the installation contractors were found to be fairly accurate and few discrepancies were identified. However, as the measured CFM leakage value was greater than the reported CFM leakage value in a majority of cases, there may be some differences between the Evaluators and the contractors blower door testing methodology. Alternatively, some customers may have removed door sweeps or other air sealing items after they were installed, and reliable duplication of blower door testing results is difficult due to the number of variables involved. As stated in prior reports, including an itemized list of air infiltration measures would assist in verifying that the work was performed as reported Ex Post Gross and Net Savings Results After reviewing the tracking data and inputs for savings calculations, the Evaluators provided ex post gross savings according to protocols from the TRM. Ex post gross savings were within 0.1% of ex ante estimates for air infiltration, ceiling insulation, and water heater measures. The savings calculated in this evaluation differed slightly from EnerTrek calculations for CFLs. As with the 2014 program year, EnerTrek calculated annual (first-year) kwh savings for CFLs in 2015 by first calculating lifetime savings and then dividing by the estimated useful life (EUL). Thus, the annual kwh values calculated by EnerTrek incorporate future CFL baseline changes that will occur as per EISA 2007 guidelines. As the Evaluators calculated annual kwh savings as first-year savings rather than as an increment of lifetime savings, ex post CFL savings are on average greater than ex ante CFL savings. Executive Summary 1-4

144 2015 AOG/OG&E Weatherization Program EM&V Report Table 1-1 presents the gross savings achieved from participating homes receiving gas utility service from AOG. These savings do not include the 226 AOG homes whose project cost was fully paid by OG&E. Table 1-1 Gross Realized Gas Savings, AOG # of homes Gross Peak Demand Savings (Therms) Gross Annual Savings (Therms) Gross Lifetime Savings (Therms) Gross Realization Rate 1,003 4, ,317 3,543, % Table 1-2 presents the gross savings achieved from participating homes receiving electric utility service from OG&E. Table 1-2 Gross Realized Electricity Savings, OG&E # of homes Gross Peak Demand Savings (kw) Gross Annual Savings (kwh) Gross Lifetime Savings (kwh) Gross Realization Rate 1, ,057,508 46,242, % Table 1-3 presents the gas and electricity savings achieved for utilities other than AOG and OG&E, such as municipal utilities or propane customers, as well as the 226 participating homes receiving gas utility service from AOG where OG&E paid the full project cost. Table 1-3 Gross Realized Savings, Non-Program and AOG Homes Paid by OG&E Measure Gross Peak Demand Savings (Therms) Gross Annual Savings (Therms) Gross Lifetime Savings (Therms) Gross Peak Demand Savings (kw) Gross Annual Savings (kwh) Gross Lifetime Savings (kwh) CFLs - (2) (19) Ceiling Insulation , , ,133 4,542,657 Air Infiltration , , , ,061 Water Heater Jacket & Pipe , ,576 Total 1, ,789 1,117, ,735 5,232,294 Table 1-4 presents overall kwh and therms gross realization rates by measure. These gross realization rates are representative of all program savings, including all gas and electric savings presented in Table 1-1, Table 1-2 and Table 1-3. Executive Summary 1-5

145 2015 AOG/OG&E Weatherization Program EM&V Report Table 1-4 Overall Gross Realization Rates by Measure Measure kwh Gross Realization Rate Therms Gross Realization Rate CFLs 110% 193% Ceiling Insulation 100% 100% Air Infiltration 100% 100% Water Heater Jacket & Pipe 100% 100% A net-to-gross analysis for the program was performed during the 2012 evaluation year, resulting in a program level free-ridership rate of 2%. This free-ridership rate was applied to the program in both 2012 and In the 2014 evaluation report, the Evaluators found the gas and electric net-to-gross ratios to be very close to 1, and applied a net-to-gross ratio of 1 to the program for During discussions with the Independent Evaluation Monitor (IEM) in determining a default free-ridership rate for the Core Weatherization Program framework, the Evaluators and IEM concluded that in the absence of a NTG study the more conservative of the two past values should be adopted. In maintaining consistency with this approach, the Evaluators applied the free-ridership rate of 2% to the AOG-OG&E program for Although there is a stipulated free-ridership rate of 2% for 2015 and moving forward, there is no stipulated spillover savings value for either the 2015 AOG-OG&E Weatherization Program or other offerings under the upcoming Core Weatherization Program framework. Therefore, the Evaluators used the participant survey to conduct a spillover savings assessment for program participants in The Evaluators included a series of questions in the participant survey to inform the spillover savings assessment for These questions were designed to gather information regarding: Whether program participants have purchased and installed additional, nonincentivized energy saving measures since participating in the program; Which additional, non-incentivized energy saving measures program participants have purchased and installed since participating in the program; and The extent to which the AOG-OG&E Weatherization Program influenced the purchase of these additional non-incentivized energy saving measures. The spillover methodology represents a threshold approach, where additional energy efficiency measures implemented by program participants are either 100% attributable to program influence or 0% attributable. After applying free-ridership rates and participant spillover rates, the resulting programlevel net savings are divided by program-level gross savings in order to calculate the program-level net-to-gross ratio. Executive Summary 1-6

146 2015 AOG/OG&E Weatherization Program EM&V Report Table 1-5 presents net program savings for AOG, including the number of participating homes contributing to the savings totals. 4 The table does not include gas savings for the 226 AOG-serviced homes were OG&E paid the full project cost. 5 # of homes Table 1-5 Net Realized Gas Savings, AOG Net Peak Demand Savings (Therms) Net Annual Savings (Therms) Net Lifetime Savings (Therms) Net-to-Gross Ratio 1,003 4, ,192 3,478, % Table 1-6 presents net program savings for OG&E, including the number of participating homes contributing to the savings totals. 6 # of homes Table 1-6 Net Realized Electricity Savings, OG&E Net Peak Demand Savings (kw) Net Annual Savings (kwh) Net Lifetime Savings (kwh) Net-to-Gross Ratio 1, ,000,505 45,393, % Conclusions and Recommendations The AOG-OG&E Weatherization Program was evaluated for overall effectiveness, performance, and design, and the Evaluators developed conclusions with consideration of the seven comprehensiveness checklist factors developed by the Arkansas Public Service Commission. The key conclusions from the 2015 evaluation of the AOG/OG&E Weatherization Program are as follows: Continued Participant Satisfaction: The results of the 2015 participant survey suggest that nearly all participants are very satisfied with each element of their program experience. This is consistent with the results obtained during 2014 and prior years. Although some respondents had issues with the level of knowledge or level of service provided by contractors, instances of dissatisfaction were very infrequent and the majority of participants provided positive comments about their experiences. Consistent Participant Characteristics: As shown in Section of this report, there do not appear to have been any significant changes in the demographics or residential characteristics of participating customers since the 2014 program year, other than a slight increase in the average square footage of participant homes. 4 The net-to-gross ratio displayed is rounded to the nearest hundredth of a percent. 5 During the 2014 program year, OG&E paid the full project cost for 226 participating homes that were serviced by AOG. These 226 homes and their Therms savings are not included in the AOG savings total. 6 The net-to-gross ratio displayed is rounded to the nearest hundredth of a percent. Executive Summary 1-7

147 2015 AOG/OG&E Weatherization Program EM&V Report Continued Cross-Fuel Coordination: As with the 2014 program year, AOG fully expended its program budget by late August of 2015, and OG&E fully paid the cost of providing services to 226 participating homes that were customers of both AOG and OG&E. The Evaluators previously noted in 2014 that by focusing on homes that receive utility service from both AOG and OG&E, AOG would be able to fund a greater number of homes and possibly retain some funding for later in the program year. This occurred to some extent, as the percentage of participating homes receiving service from both AOG and OG&E increased slightly in 2015 compared to 2014 (58% and 54%, respectively), and more AOG customers participated in 2015 (1,229 as compared to 1,172 in 2014). Overall, this does not necessarily reflect a program performance issue as the program was able to provide services to nearly as many homes as were weatherized in previous years, and the utilities were able to coordinate their resources in order to prevent delays or interruptions in program delivery. Minimal Core Weatherization Transition Effects: The AOG-OG&E Weatherization Program is currently well suited to transitioning into the new Core Weatherization Program framework that has been developed by the Arkansas IOUs and other stakeholders. As many aspects of this framework referenced the design and operation of the AOG-OG&E program as a successful model, complying with the guidelines of the new framework will likely require few changes to this program. Likely modifications include incorporating aerators, showerheads, and smart power strips into the program measure mix, maintaining or increasing BPI certification of participating contractor staff, and slightly modifying program eligibility requirements such as residence age and square footage. Adequate Data Collection Procedures: As with the 2014 and 2013 program years, the measure implementation data reported by the installation contractors were found to be fairly accurate and few discrepancies were identified. The water heating type was not collected for some homes that did not receive water heating measures, and the Evaluators noted in 2014 that these data may be useful for tracking purposes and spillover calculations. However, the discrepancies were infrequent and minor, and do not appear to indicate any systematic issues with program delivery. Adequate Database Quality: The Evaluators found the ex ante savings values within the EnerTrek database to be accurate for nearly all measures. Additionally, Frontier Associates was very consistent in responding to data requests and correcting errors when necessary. The main persisting discrepancy between ex ante and ex post calculations was that the EnerTrek database continued to calculate annual CFL savings as an increment of lifetime savings, which incorporated future baseline changes that are not relevant to first-year savings. The Evaluators previous recommendation to modify this approach was not adopted, but this is a minor issue as the discrepancy only slightly affects CFL realization rates. Some Spillover Effects: The spillover savings assessment conducted for the 2015 program year found spillover savings equal to approximately 0.14% of total gross savings Executive Summary 1-8

148 2015 AOG/OG&E Weatherization Program EM&V Report for AOG and OG&E. Spillover mainly consisted of low-cost measures such as water heater insulation and lighting, but one customer installed ENERGY STAR windows and another installed an efficient furnace due to information and recommendations received during their participation in the program. As the program targets homes that are considered severely energy inefficient and participating customers may not have the means to improve their home s efficiency on their own, the level of spillover is likely to be low in any given year. The AOG/OG&E Weatherization Program was very successful in The Evaluators identified few specific, systematic or persistent issues with program operation and design. As the utilities plan to continue offering similar services and maintaining their current operational structure under the Core Weatherization Program, consideration of the following recommendations may be useful moving forward: Include Itemized Air Infiltration Measures: This recommendation is carried over from the prior evaluation, and the decision to adopt it may depend on programming costs. The initial home audit data collection form and the post-implementation measure verification form both include fields for detailed measure information and additional field notes. However, some of this information is not present in the tracking data exports. Frontier Associates has noted that this information is stored, but there is not an existing report format that will provide these details. The itemized measures and any field notes would be useful from an EM&V standpoint but if the programming costs to generate such a report are high, it would likely be more cost-effective to maintain the current approach of on-site visual inspection and blower door testing. Specify Propane Customers in Tracking Data: The tracking data included the electric utility provider for each participating home, and identified the gas utility provider as either AOG or None. The classification of None for a gas utility would intuitively refer to customers who are all-electric, but 30 of these customers are listed as having gas heat and had ex ante gas savings within the tracking data. Therefore these customers are most likely receiving propane service. In order to distinguish between a tracking data error and the presence of a propane customer, the Evaluators recommend that contractors identify propane customers on their audit forms and that the EnerTrek system generates a separate gas utility designation for these customers. Consider Working with Contractors to Ensure Sufficient Customer Service: The EM&V efforts and utilities QA/QC procedures have found that the participating contractors are sufficiently trained in the audit and measure installation process, but some customers have expressed that their contractor could have been more effective in explaining either the types of services that were being provided or explaining energy use and efficiency options in general. Based on the survey results this represents only a small percentage (fewer than 10%) of participants. However working with the contractors to ensure that they leave the home with the customer feeling aware of the work performed and knowledgeable in additional energy efficiency options would likely serve to further Executive Summary 1-9

149 2015 AOG/OG&E Weatherization Program EM&V Report increase customer satisfaction. Additionally, customers with a higher level of knowledge regarding their energy use, home weatherization, and energy efficiency opportunities are more likely to implement measures that may qualify for spillover savings. Table 1-7 presents the above items, outlining the relevant issue, potential consequences, and associated recommendations. Executive Summary 1-10

150 2015 AOG/OG&E Weatherization Program EM&V Report Table 1-7 Recommendations from 2015 Program Year Evaluation Issue Consequences Recommendation Tracking data do not include specific measure details for some items (i.e. whether door sweeps, window sealing, etc. were installed, whether lighting was installed indoors or outdoors) Difficult to completely inspect measure installation Limits level of detail possible for measure tracking Depending on programming costs, include an itemized list of all air infiltration reduction measures installed in each home. The Gas Utility field within the tracking data uses the classification of None to refer to both all-electric and propane customers. Some customers expressed that their contractor did not fully explain the services provided or could have been more knowledgeable Difficult to distinguish between a tracking data error and a propane customer. Negatively effects customer satisfaction, reduces spillover potential. Collect and report specific utility providers and identify propane customers within the program tracking database. Consider working with contractors to ensure that they provide sufficient customer service and are able to sufficiently answer questions about energy efficiency options and provide recommendations. Executive Summary 1-11

151 2015 AOG/OG&E Weatherization Program EM&V Report 1.4 Report Organization The report is organized as follows: Chapter 2 presents the impact findings and discusses the methods used for, and the results obtained from, estimating gross and net savings for the program; Chapter 3 presents the methods used for and the results obtained from the limited process evaluation of the program; Chapter 4 presents key conclusions and recommendations from the evaluation of the program; and Appendix A presents the survey instrument administered to a sample of 2015 program participants. Executive Summary 1-12

152 2. Impact Evaluation Findings This section presents the results of the gross savings verification and savings calculation review for the AOG/OG&E Weatherization Program in the 2015 program year. 2.1 Glossary of Terms As a first step to detailing the evaluation methodologies, the Evaluators provide a glossary of terms to follow: Ex Ante Savings Energy savings as determined and reported by program implementers/sponsoring utilities prior to evaluation by EM&V contractor Ex Post Gross Savings Energy savings as determined by the EM&V contractor through engineering analysis, statistical analysis, and/or onsite verification o Gross Realization Rate Ratio of Ex Post Gross Savings / Ex Ante Savings Ex Post Net Savings Ex Post Gross savings x Net-to-Gross Ratio o Net-to-Gross Ratio (NTGR) = (1 Free-Ridership % + Spillover %), also defined as Ex Post Net Savings / Ex Post Gross Savings o Free-Ridership Percentage of participants who would have implemented the same energy efficiency measures in a similar timeframe absent the program o Spillover Savings generated by a program that are not incentivized. Examples of this include a customer that is introduced to energy efficiency through the program and due to this undertakes other projects for which they do not receive an incentive. 2.2 Summary of Ex Ante Savings The AOG/OG&E Weatherization Program generated savings through the implementation of several energy efficient measure types, such as ceiling insulation, CFLs, air infiltration reduction, and water heater insulation. Table 2-1 and Table 2-2 present the overall ex ante savings for AOG and OG&E by measure, respectively. These values were obtained from the EnerTrek program tracking database exports that were provided to the Evaluators by Frontier Associates. These tables include all ex ante gas savings for participating homes serviced by AOG where AOG paid at least a portion of the project cost, and all ex ante electric savings for participating homes serviced by OG&E where OG&E paid at least a portion of the project cost. Impact Evaluation Findings 2-1

153 2015 AOG/OG&E Weatherization Program EM&V Report Table 2-1 Ex Ante Savings by Measure, AOG Measure Peak Demand Annual Savings Savings (Therms) (Therms) CFLs - (3) CeilingInsulation 1, ,901 AirInfiltration 2, ,346 WaterHeaterJacket&Pipe Total 4, ,211 Table 2-2 Ex Ante Savings by Measure, OG&E Measure Peak Demand Savings (kw) Annual Savings (kwh) CFLs ,925 CeilingInsulation ,639,000 AirInfiltration ,593 WaterHeaterJacket&Pipe ,805 Total 1, ,991,323 The following table presents the remaining ex ante gas and electric savings that were not included in the two tables above. This consists of gas and electric savings attributable to municipal utilities, co-op utilities, or other investor owned utilities, which are not sponsors of this program, as well as gas savings from the 226 homes serviced by AOG where OG&E paid the full project cost. Table 2-3 Ex Ante Savings by Measure Non-Program and AOG Homes Paid by OG&E Measure Peak Demand Savings (Therms) Annual Savings (Therms) Peak Demand Savings (kw) Annual Savings (kwh) CFLs - (1) - - Ceiling Insulation , ,120 Air Infiltration , ,101 Water Heater Jacket & Pipe Total 1, , , Ex Post Gross Savings Calculation Methodology For measures implemented through the 2015 program, savings verification was performed according to methodologies described in TRM V5.0. Table 2-4 identifies the sections in the TRM that were used for verification of measure-level savings under the AOG/OG&E Weatherization Program. Impact Evaluation Findings 2-2

154 2015 AOG/OG&E Weatherization Program EM&V Report Table 2-4 TRM Sections by Measure Type Measure Type TRM Section(s) Air Infiltration Ceiling Insulation CFLs Window AC Water Heater Measures 2.3.2, Three measures were responsible for nearly all of the ex post gross savings for the AOG/OG&E Weatherization Program: air infiltration reduction, ceiling insulation, and the replacement of incandescent lamps with compact fluorescent lamps (CFLs). The calculation methodologies for these measures are detailed in the following sections. In these examples, energy units are expressed in kwh Air Infiltration Reduction Savings Calculations The deemed savings algorithms in TRM V5.0 for air infiltration reduction were developed through simulation modeling in BEopt, a residential building simulation modeling platform that uses the DOE EnergyPlus simulation engine. Multiple equipment configurations were simulated in each of the four Arkansas weather zones in developing savings values denominated in deemed savings per CFM50 of air leakage rate reduction. The following table summarizes the deemed savings values for Weather Zone 7. Table 2-5 Deemed Savings Values for Air Infiltration Reduction, Zone 7 Equipment Type kwh Savings / CFM50 (ESF) kw Savings / CFM50 (DSF) Therm Savings / CFM50 (GSF) Peak Therms / CFM50 (GPSF) Electric AC with Gas Heat Gas Heat Only (no AC) Elec. AC with Resistance heat n/a n/a n/a Heat Pump n/a n/a The following example considers a residence in Weather Zone 7 with electric AC and gas heat. If the residence had a leakage rate of 16,100 CFM50 before air infiltration reduction and a leakage rate of 7,220 CFM50 after, then the residence would have an annual gross savings of 1,687 kwh. kwh Savings Air Infiltration Savings = (16,100 CFM CFM 50 pre 7,220 CFM 50 post ) 50 Impact Evaluation Findings 2-3

155 2015 AOG/OG&E Weatherization Program EM&V Report Air Infiltration Savings = 1,687 kwh TRM V5.0 also specifies Minimum Final Ventilation Rates (MVR) and Maximum Preinstallation Infiltration Rates in order to ensure that air infiltration work is performed in accordance with health and safety guidelines and that infiltration reduction is not attempted on homes with prohibitively severe leakage levels. TRM V5.0 specifes the the MVR must comply with current Arkansas building codes, which present three options for calculating MVR. However, as per Protocol E2 of TRM V5.0, the enforcement date for a code or standard update is the end of the current program year if the effective date of the code or standard update is before July 1. Therefore the Evaluators applied the MVR guidelines from TRM V4.0 for the 2015 evaluation. TRM V4.0 specifies MVR as follows: The MVR specifies the minimum post-installation air infiltration value that can be applied to the deemed savings calculation. If a home s final CFM50 value is below the MVR, the deemed savings calculation for air infiltration reduction on the home is calculated using the MVR rather than the actual post-installation leakage value. The MVR for a given home is calculated as follows: Where: Min CFM50 = [0.01 x Afloor x (BR + 1)] X N Min CFM50 = Minimum final ventilation rate (CFM50) AFloor = Floor area (ft 2 ) BR= Number of bedrooms (must be at least 1) N = N factor (deemed value based on type of wind shielding and number of stories in home) With regard to Maximum Pre-installation Infiltration Rate, TRM V4.0 specifies that in order to avoid incentivizing homes with severe building envelope issues that cannot be remedied with typical air infiltration procedures, the baseline pre-installation infiltration rate should be based on a maximum air change rate of 3.0. With this baseline in effect, the maximum allowable pre-installation CFM50 value is calculated as follows: Where: CFM 50,pre /ft 2 = ACH Nat,pre h N 60 CFM50,pre /ft 2 = Per square foot pre-installation infiltration rate (CFM50/ft 2 ) ACHNat,pre = Maximum pre-installation air change rate (ACHNat) = = Constant to convert from minutes to hours Impact Evaluation Findings 2-4

156 2015 AOG/OG&E Weatherization Program EM&V Report h = Ceiling height (ft) = 8.5 (default) N = N factor (deemed value based on type of wind shielding and number of stories in home) If a home s pre-installation infiltration rate exceeds the rate calculated above, the Maximum Pre-installation Infiltration Rate is used for deemed savings calculations. Additionally, TRM V4.0 specifies a maximum CFM50 per-square-foot value. For deemed savings calculations, pre-installation leakage rates cannot exceed these values Ceiling Insulation Savings Calculations The deemed savings algorithms in TRM V5.0 for ceiling insulation were developed through simulation modeling in BEopt, a residential building simulation modeling platform that uses the DOE EnergyPlus simulation engine. Multiple equipment configurations were simulated in each of the four Arkansas weather zones using both R-38 and R-49 insulation in developing savings values denominated in deemed savings per square footage of ceiling area. Table 2-6 summarizes the deemed savings values for R-38 Weather Zone 8. Ceiling Insulation Base R-value Table 2-6 Deemed Savings Values for R-38 Ceiling Insulation, Zone 8 AC/Gas Heat kwh Gas Heat (no AC) kwh Gas Heat (no AC) Therms AC/Electric Resistance kwh Heat Pump kwh AC Peak Savings (kw) Peak Gas Savings 7 (therms) (/ sq. ft.) (/ sq. ft.) (/ sq. ft.) (/ sq. ft.) (/ sq. ft.) (/ sq. ft.) (/ sq. ft.) 0 to to to to to The following example considers a residence in Weather Zone 8 with a heat pump, and a pre-retrofit R-value of ceiling insulation in the range of 9 to 14. If the residence has a ceiling area of 1,200 sq. ft., then the residence would have an annual gross savings of 1,100 kwh. Ceiling Insulation Savings = kwh ft 2 (1,200 ft2 ) = 1,100 kwh 7 Data in table are for Blytheville peak. Other Zone 8 peaks can be calculated by multiplying Blytheville peak by the appropriate factor, m. For Jonesboro, m=0.890 (0-1), m = (2 to 4), (5 to 8), (9 to 14), (15 to 22). For Fort Smith, m=0.859 (0-1), m = (2 to 4), (5 to 8), (9 to 14), (15 to 22). Impact Evaluation Findings 2-5

157 2015 AOG/OG&E Weatherization Program EM&V Report TRM V5.0 specifies an efficiency standard of R-38, meaning that in order to qualify for deemed savings the combined R-value of existing and added insulation should be at least R Compact Fluorescent Lamps (CFLs) Savings Calculations The deemed savings for compact fluorescent lamps can be calculated by using the following equation. kwh savings = ((Watts base Watts post )/1,000) x Hours x ISR x IEF E The inputs, which assume the following prerequisite knowledge, can be found in Section of TRM V5.0: The quantity and wattages of both pre and post fixtures; Whether or not the retrofits were time of sale or direct install (this defines the inservice rate); and The heating type of the residence. For example, if in March 2015 (5) 13W CFLs were directly installed to replace (5) 60W incandescent lamps in a residence with gas heating, the residence would have an annual gross savings of 198 kwh. kwh savings = (( )/1, = 198 kwh TRM V5.0 includes specifications for heating penalties from CFLs in natural gas heated homes, calculated as follows: Where: Therms penalty = ((W base W post )/1000) x ISR x IEF G IEFg = Interactive Effects Factor to account for gas heating penalties TRM V5.0 also accounts for future changes in lighting baselines as per EISA 2007 guidelines. Specifically, TRM V5.0 specifies that the 1 st Tier EISA 2007 baselines come into effect in January 2014, and that the 2 nd Tier EISA 2007 baselines come into effect in January These baseline changes affect lifetime savings calculations for CFLs. As per Protocol E2 of TRM V5.0, the enforcement date for a code or standard update is the end of the current program year if the effective date of the code or standard update is before July 1. Thus, the Evaluators calculated 2015 first-year savings using the 1 st Tier EISA baseline On-site Verification Procedure In addition to TRM verification, the Evaluators conducted on-site field verification of a sample of participant homes. This process involved reviewing tracking information and inspecting the completeness and accuracy of the implemented measures. The Impact Evaluation Findings 2-6

158 2015 AOG/OG&E Weatherization Program EM&V Report methodologies for sampling and conducting field visits during the 2015 program evaluation year are identical to those employed for the 2014 and earlier evaluations. A summary review of these methods is provided below Verification Sampling Methodology The Evaluators conducted a random sample of participants for the ex-post verification process. The sample size for verification surveys is calculated to meet 90% confidence and 10% precision (90/10). The main purpose of the verification activity was to determine whether measures were properly installed in the quantities reported in program tracking data. Thus, the coefficient of variation (CV) used for sampling was not based on participant savings but was assumed to be 0.5, which is a commonly assumed CV value for residential program evaluations. The resulting sample size is estimated at: Where, CV n 0 = ( ) RP = Z Score for 90% confidence interval in a normal distribution CV = Coefficient of Variation RP = Required Precision, 10% in this evaluation With 10% required precision (RP), this calls for a sample of 68 for programs with a sufficiently large population. In total, the Evaluators scheduled appointments with 65 participants. Due to cancellations and customer absences, Evaluator field staff members were able to conduct on-site visits for 58 program participants. This was supplemented by telephone verification with an additional 10 participants in order to meet the 68 participant target Verification Procedure The primary goal of field verification was to ensure that the reported measures were installed and operating correctly in participant homes. Participants were given VISA or Walmart gift cards for their time; these were in the amount of $25. During the on-site visits, the Evaluators field technicians accomplished the following: Verified the implementation status of the measures; verified that the measures were indeed installed, that they were installed correctly, and were functioning properly. Photographs were taken of most of the installed measures. Data collected at each site focused on obtaining more specific information regarding the characteristics of the home where the measures were implemented. Impact Evaluation Findings 2-7 2

159 2015 AOG/OG&E Weatherization Program EM&V Report A field visit form was completed for each visited site in order to document measure quantities, home characteristics, and any needed additional commentary regarding the visit. Specifically, the field form included the following fields: Home Characteristics: The field engineer documented the type of home (i.e. single story vs. multi-story), number of bedrooms, number of bathrooms, total conditioned area, and heating type. Measure Quantity Verification: The engineer documented reported vs. actual quantities of each measure type (i.e. CFLs, water heater measures) and any applicable notes regarding burnt out bulbs or non-operational equipment. Insulation Assessment: The field form asks for insulation square footage, the R- value or inches of insulation, and the type of insulation (i.e. blown cell). Infiltration Assessment: For homes receiving air infiltration measures, the field engineer conducted a blower door test and recorded ex-post leakage for comparison with reported leakage values. For the most part, field staff found the reported tracking information to be accurate, and confirmed that nearly all reported measures had been installed completely and correctly. No adjustments to measure in-service rates were made based on the results of the field verification activity. Further information detailing the overall results of the field verification visits can be found in Chapter Ex Post Gross Savings Results After reviewing the tracking data and inputs for savings calculations, the Evaluators provided ex post gross savings according to protocols from the TRM. Ex post gross savings were within 0.1% of ex ante estimates for air infiltration, ceiling insulation, and water heater measures. The savings calculated in this evaluation differed slightly from EnerTrek calculations for CFLs. As with the 2014 program year, EnerTrek calculated annual (first-year) kwh savings for CFLs in 2015 by first calculating lifetime savings and then dividing by the estimated useful life (EUL). Thus, the annual kwh values calculated by EnerTrek incorporate future CFL baseline changes that will occur as per EISA 2007 guidelines. As the Evaluators calculated annual kwh savings as first-year savings rather than as an increment of lifetime savings, ex post CFL savings are on average greater than ex ante CFL savings. Table 2-7 presents the gross savings achieved from participating homes receiving gas utility service from AOG. These savings do not include the 226 AOG homes whose project cost was fully paid by OG&E. Impact Evaluation Findings 2-8

160 2015 AOG/OG&E Weatherization Program EM&V Report Table 2-7 Gross Realized Gas Savings, AOG # of homes Gross Peak Demand Savings (Therms) Gross Annual Savings (Therms) Gross Lifetime Savings (Therms) Gross Realization Rate 1,003 4, ,317 3,543, % Table 2-8 presents the gross savings achieved from participating homes receiving electric utility service from OG&E. Table 2-8 Gross Realized Electricity Savings, OG&E # of homes Gross Peak Demand Savings (kw) Gross Annual Savings (kwh) Gross Lifetime Savings (kwh) Gross Realization Rate 1, ,057,508 46,242, % Table 2-9 presents the gas and electricity savings achieved for utilities other than AOG and OG&E, such as municipal utilities or propane customers, as well as the 226 participating homes receiving gas utility service from AOG where OG&E paid the full project cost. Table 2-9 Gross Realized Savings, Non-Program and AOG Homes Paid by OG&E Measure Gross Peak Demand Savings (Therms) Gross Annual Savings (Therms) Gross Lifetime Savings (Therms) Gross Peak Demand Savings (kw) Gross Annual Savings (kwh) Gross Lifetime Savings (kwh) CFLs - (2) (19) Ceiling Insulation , , ,133 4,542,657 Air Infiltration , , , ,061 Water Heater Jacket & Pipe , ,576 Total 1, ,789 1,117, ,735 5,232,294 Table 2-10 presents overall kwh and therms gross realization rates by measure. These gross realization rates are representative of all program savings, including all gas and electric savings presented in Table 2-7, Table 2-8, and Table 2-9. Table 2-10 Overall Gross Realization Rates by Measure Measure kwh Gross Realization Rate Therms Gross Realization Rate CFLs 110% 193% Ceiling Insulation 100% 100% Air Infiltration 100% 100% Water Heater Jacket & Pipe 100% 100% Impact Evaluation Findings 2-9

161 2015 AOG/OG&E Weatherization Program EM&V Report 2.5 Free-ridership Determination A net-to-gross analysis for the program was performed during the 2012 evaluation year, resulting in a program level free-ridership rate of 2%. This free-ridership rate was applied to the program in both 2012 and In the 2014 evaluation report, the Evaluators found the gas and electric net-to-gross ratios to be very close to 1, and applied a net-to-gross ratio of 1 to the program for During discussions with the Independent Evaluation Monitor (IEM) in determining a default free-ridership rate for the Core Weatherization Program framework, the Evaluators and IEM concluded that in the absence of a NTG study the more conservative of the two past values should be adopted. In maintaining consistency with this approach, the Evaluators applied the free-ridership rate of 2% to the AOG-OG&E program for Spillover Savings Although there is a stipulated free-ridership rate of 2% for 2015 and moving forward, there is no stipulated spillover savings value for either the 2015 AOG-OG&E Weatherization Program or other offerings under the upcoming Core Weatherization Program framework. Therefore, the Evaluators used the participant survey to conduct a spillover savings assessment for program participants in Spillover Savings Description While free-ridership represents the portion of gross savings that would still have been realized in the absence of the program, spillover refers to reductions in energy consumption or demand that are attributable to program influences beyond those directly associated with program participation. Participant spillover represents the energy savings that are achieved when a program participant as a result of the program s influence installs energy efficiency measures outside the efficiency program after having participated. Spillovers are not recorded in the program tracking system, but may be assessed during the evaluation effort through a variety of methods including participant surveying Spillover Assessment Methodology The Evaluators included a series of questions in the participant survey to inform the spillover savings assessment for These questions were designed to gather information regarding: Whether program participants have purchased and installed additional, nonincentivized energy saving measures since participating in the program; Which additional, non-incentivized energy saving measures program participants have purchased and installed since participating in the program; and Impact Evaluation Findings 2-10

162 2015 AOG/OG&E Weatherization Program EM&V Report The extent to which the AOG-OG&E Weatherization Program influenced the purchase of these additional non-incentivized energy saving measures. Survey respondents were first asked the following question: Q15. Since you participated in this program, have you installed any energy efficient equipment in your home that you have not received an incentive for? Respondents answering Yes to the above question were then provided with a list of common residential energy efficiency measures such as energy efficient LED or CFL lighting and Energy Star appliances, and are asked to indicate which of these items they have purchased, and how many they have purchased, since participating in the program. 8 Respondents who indicate that they have installed at least one additional energy efficient measure since participating in the program are then asked three questions to determine the level of influence that the AOG-OG&E Weatherization Program may have had on the decision to purchase and install the item(s). The first question serves to assess whether the participant had already planned to install the energy efficient items before they participated in the program, and is as follows: Q15-B. Did you have specific plans to purchase any of these additional items before you signed up for the weatherization program? Respondents answering No to the above question are considered not to have had preexisting plans to install these additional energy efficiency measures. Respondents answering Yes are flagged, and are subject to more stringent requirements for passing the following program influence variable screening. Two questions are used to construct the program influence variable, as follows: Q15-C. On a scale of 0 to 10, where 0 represents Not at all important and 10 represents Extremely important, how important was your experience with the AOG/OG&E Weatherization Program in your decision to purchase and install these additional items? Q15-D. On a scale of 0 to 10, where 0 represents Not at all likely and 10 represents Extremely likely, how likely would you have been to purchase these additional items if you had never participated in the AOG/OG&E Weatherization Program? The Program Influence Score (PI Score) is then calculated as the average of the responses to these two questions, where the numeric scale from Q15-D is reversed by subtracting the Q15-D score from 10 total possible points: PI Score = ((Q79 Score) + (10 Q80 Score))/2 8 A full list of the energy efficiency measures included in this question can be found in Appendix A. Impact Evaluation Findings 2-11

163 2015 AOG/OG&E Weatherization Program EM&V Report For example, a respondent providing a rating of 9 to Q15-C and a rating of 3 to Q15-D would receive a PI Score as follows: PI Score=(9+10 3)/2 PI Score = 8 Respondents who answered No to Q15-B and whose PI Scores are greater than 5 are considered to have made additional energy efficiency purchases that were significantly influenced by the program. Respondents who answered Yes to Q15-B have the same PI Score requirement, but must also have provided a rating of 0, 1, or 2 to Q15-D in order to be eligible for spillover savings. The spillover methodology described above represents a threshold approach, where additional energy efficiency measures implemented by program participants are either 100% attributable to program influence or 0% attributable. Savings for additional measures purchased and installed by these respondents are then calculated using deemed savings methods or average savings for that measure type in the participant population (if applicable). The total spillover savings are then distributed across the participant survey sample by fuel source (e.g. OG&E or AOG vs. nonparticipating IOUs or municipal utilities) in order to determine per-participant spillover rates. The per-participant spillover rate by utility provider is then applied to the population of program participants. After applying free-ridership rates and participant spillover rates, the resulting programlevel net savings are divided by program-level gross savings in order to calculate the program-level net-to-gross ratio Spillover Assessment Results The participant survey was administered to 300 customers who participated in the program during Of these 300 respondents, 24 indicated that they had installed additional, non-incentivized energy efficiency measures since participating in the program. When asked to specify what they had installed, three respondents identified measures that would result in very slight or non-quantifiable energy savings or were outside the scope of the AOG-OG&E Weatherization Program, such as installing a new door or adding a single weather strip. Of the remaining 21 respondents, five respondents met the attribution criteria specified in Section The energy efficiency measures identified by these five respondents consisted of the following items, shown in Table The customer who installed energy efficient windows was listed with a gas utility of None in program tracking data, but was also listed as having gas heat. This is likely a propane customer. Impact Evaluation Findings 2-12

164 2015 AOG/OG&E Weatherization Program EM&V Report Table 2-11 Measures Eligible for Spillover Savings Measure Type Quantity Electric Utility Provider Gas Utility Provider Water heater pipe insulation 2 OG&E AOG (paid by OG&E) Water heater jacket 1 OG&E AOG (paid by OG&E) LED light bulbs 6 OG&E AOG Low flow shower head 1 OG&E AOG Gas furnace 1 Non-OG&E AOG ENERGY STAR Windows 12 OG&E None The Evaluators calculated electric and/or gas savings for the above items based on the fuel type(s) of the individual respondents who identified the spillover measures. This resulted in sampled spillover savings by utility service provider. These savings were then extrapolated, by utility service provider, from the survey sample of 300 respondents to the program population of 1,612 participants. Total spillover effects were fairly minimal and amounted to less than 0.2% of gross program savings for both AOG and OG&E. 2.7 Ex Post Net Savings Results The following tables present the net savings results of the evaluation of the 2015 AOG/OG&E Weatherization Program. Table 2-12 includes net realized savings for AOG, incorporating the free-ridership rate of 2% and spillover savings. 10 This consists of all gas savings for participating homes serviced by AOG where AOG paid at least a portion of the project cost. # of Homes Table 2-12 Net Realized Gas Savings Attributable to AOG Net Peak Demand Savings (Therms) Net Annual Savings (Therms) Net Lifetime Savings (Therms) Net-to-Gross Ratio 1,003 4, , ,478, % Table 2-13 includes net realized savings for OG&E, incorporating the free-ridership rate of 2% and spillover savings. 11 This consists of all electric savings for participating homes serviced by OG&E where OG&E paid at least a portion of the project cost. Table 2-13 Net Realized Electricity Savings Attributable to OG&E # of Homes Net Peak Demand Savings (kw) Net Annual Savings (kwh) Net Lifetime Savings (kwh) Net-to-Gross Ratio 1, ,000, ,393, % 10 The net-to-gross ratio displayed is rounded to the nearest hundredth of a percent. 11 The net-to-gross ratio displayed is rounded to the nearest hundredth of a percent. Impact Evaluation Findings 2-13

165 2015 AOG/OG&E Weatherization Program EM&V Report In the AOG/OG&E Weatherization Program, the participating utilities are AOG and OG&E. Typically, the amount that either utility pays for a participating home depends on whether the utility is serviced by AOG, by OG&E, or by both utilities. Weatherization of a home receiving both gas service from AOG and electric service from OG&E would typically be paid for by both AOG and OG&E. However, in the 2015 program year, AOG expended its full program budget by mid-year and OG&E began to pay the full project cost for homes receiving utility service from both AOG and OG&E. This was done for 226 AOG-serviced homes. Table 2-14 and Table 2-15 present the remaining net ex post gas and electric savings that were not directly attributed to AOG or OG&E. This consists of gas and electric savings attributable to municipal utilities, co-op utilities, or other investor owned utilities which are not sponsors of this program, as well as gas savings from the 226 homes serviced by AOG where OG&E paid the full project cost. The classification of None for a gas utility was taken from program tracking data, and as the tracking data also listed 30 of these customers as having gas heat, the subset of customers shown in Table 2-14 most likely receive propane service. Table 2-14 Net Realized Gas Savings, Non-Program and AOG Homes Paid by OG&E Utility # of Homes Net Peak Demand Savings (Therms) Net Annual Savings (Therms) Net Lifetime Savings (Therms) Net-to-Gross Ratio Centerpoint , % None ,406 97, % AOG (OG&E Paid) , , % Source Gas , , % Total 609 1, ,621 1,099, % Utility Table 2-15 Net Realized Electricity Savings, Non-Program # of Homes Net Peak Demand Savings (kw) Net Annual Savings (kwh) Net Lifetime Savings (kwh) Net-to-Gross Ratio Arkansas Valley ,325 4,302, % Other 7 7 9, , % SWEPCO Arkansas , , % Total ,940 5,127, % Impact Evaluation Findings 2-14

166 3. Process Evaluation Findings This chapter presents the key findings from the limited process evaluation that the Evaluators conducted in 2015, including tracking recommendations from prior program evaluations and summarizing updates to program operation and delivery. Additionally, the chapter presents findings from participant surveys and in-depth interviews with program staff, and addresses the checklist factors for portfolio comprehensiveness. 3.1 Process Evaluation Considerations The Evaluators conducted a formal process evaluation of the AOG/OG&E Weatherization Program in 2012 and determined that the program was operating effectively and had been successful in meeting its goals. TRM V5.0 Protocol C addresses the criteria used to determine the timing and conditions needed for a process evaluation, and the following tables summarize the program in the context of these requirements. Table 3-1 Determining Process Evaluation Timing Component New and Innovative Components No Previous Process Evaluation New Vendor or Contractor Determination No. The program design has not been modified in the past year. No. A formal process evaluation was conducted in 2012, and limited process evaluations have been conducted in each year since No. The program continues to use three installation contractors and is otherwise operated by AOG and OG&E jointly. Table 3-2 Determining Process Evaluation Conditions Component Are program impacts lower or slower than expected? Are the educational or informational goals not meeting program goals? Are the participation rates lower or slower than expected? Are the program s operational or management structure slow to get up and running or not meeting program administrative needs? Is the program s cost-effectiveness less than expected? Do participants report problems with the programs or low rates of satisfaction? Is the program producing the intended market effects? Determination No. The program has consistently met its savings goals. No. Program awareness within the customer market has increased, and educational efforts have been successful. No. The program has consistently met its participation goals. No. The prior process evaluation found these structures to be operating efficiently with adequate resources. No. The program s cost-effectiveness has been maintained at expected levels. No. Participants have consistently reported high levels of satisfaction with their program experience. Yes. Non-program contractors are being informed of opportunities within the non-participant market. Surveyed participants also appear more aware of energy efficiency in general. Process Evaluation Findings 3-1

167 2015 AOG/OG&E Weatherization Program EM&V Report Based on these findings, the Evaluators determined that the 2015 evaluation of the AOG/OG&E Weatherization Program calls for a limited process evaluation. However, as 2015 marks the final year of operation for the program prior to the implementation of the Core Weatherization Program framework in 2016, the Evaluators determined that it would also be appropriate to conduct a participant telephone survey to assess current satisfaction levels and other feedback. Additionally, the participant survey served to inform the spillover savings analysis for Thus, the 2015 limited process evaluation consisted of the following research tasks: Tracking database and documentation review; On-site field verification; Participant surveying to assess current satisfaction levels and energy efficiency engagement; and Program utility staff interviews. Table 3-3 below summarizes the survey and interview data collection for the 2015 program evaluation, including data collection type and number of respondents. Table 3-3 Interview and Survey Data Collection Summary Target Component Activity N Details Program Staff Program Participants AOG Program Staff Interview 2 OG&E Program Staff Interview 2 Telephone Survey Survey 300 The program manager and operational staff are responsible for coordinating program data, managing program resources, directing installation contractors, and communicating with AOG or OG&E staff as needed during the program process. This consisted of a satisfaction questionnaire and a series of questions related to program and energy efficiency awareness and engagement. 3.2 Response to Program Recommendations Table 3-4 summarizes the status of issues and recommendations identified in the 2014 process evaluation and impact evaluation of the AOG/OG&E Weatherization Program. All prior recommendations were either resolved or at least partially resolved. Process Evaluation Findings 3-2

168 2015 AOG/OG&E Weatherization Program EM&V Report Tracking data do not include specific measure details for some items (i.e. whether door sweeps, window sealing, etc. were installed, whether lighting was installed indoors or outdoors) Table 3-4 Status of 2014 Evaluation Recommendations Issue Consequences Recommendation Program Response Status Difficult to completely inspect measures Water heater type and telephone number were not consistently recorded in some cases Minor tracking data errors such as missing savings for CFLs, incorrect wattage, and first-year savings calculation that incorporated lifetime baseline changes Program educational materials are fairly limited EnerTrek calculated some savings with TRM V4.0, such as CFLs in the first half of the program year Limits level of detail possible for measure tracking Increases difficulty of participant surveying and field verification May have minor limitations on spillover savings calculations May cause discrepancies between ex ante savings and ex post gross savings Upcoming weatherization framework will require educational component Participants may feel that they did not receive enough information about maintenance and EE May cause discrepancies between ex ante savings and ex post gross savings Include an itemized list of all air infiltration reduction measures installed in each home. Consistently record water heater type for all homes, and record full telephone number including area code, and a secondary telephone number if possible. Check tracking data for missing inputs and increase database quality control procedures if necessary. Calculate annual savings as first-year savings moving forward. Consider developing additional educational materials that installation contractors can provide to customers, or provide additional educational materials on the utility website and direct participants to the page. Update EnerTrek with any necessary TRM modifications as early in the program year as possible. Inform contractors of additional data collection needs as soon as they are in place. CFL location has been included. However itemized air infiltration measures can only be included in data through an EnerTrek programming change, which is associated with a cost Telephone numbers were included in nearly all cases, but water heater type is still absent for 439 homes. Frontier continues to calculate CFL savings as annualized lifetime savings. However, nearly all tracking data errors have been resolved. The utilities have incorporated additional educational elements such as presentations at trade shows, providing brochures to metering staff, and referring customers to the Arkansas Energy Office website. EnerTrek calculated savings consistently throughout the 2015 program year. Partially resolved Partially resolved Partially resolved Resolved Resolved Process Evaluation Findings 3-3

169 2015 AOG/OG&E Weatherization Program EM&V Report 3.3 Program Structure Overview The overall structure and delivery of the AOG/OG&E Weatherization Program has remained fairly constant throughout This section provides a summary of current program design features and procedures, noting any differences between 2015 and prior years. Program procedures and participation stages have remained unchanged since the 2014 program year. As with prior years, the 2015 program provided residential energy audits and energy efficiency installations to customers within the service territory of Arkansas Oklahoma Gas Corporation (AOG) and Oklahoma Gas and Electric (OG&E). Participating homes were evaluated in order to determine cost-effective energy efficiency measures that would improve overall building efficiency and reduce residential energy usage. The program provided funds for the installation of various measures, including insulation, lighting, air infiltration, and water heater jacket and pipe wrap. The AOG/OG&E Weatherization Program is designed to provide utility funds to customers in order to fully offset the costs of energy efficiency audits and resulting energy efficiency measures and installations. Weatherization of participating homes is funded by AOG and/or OG&E, depending on the home s utility service provider, in this co-funded program. In 2015, the utilities covered up to $3,500 of services in participant homes, an increase over the $3,000 limit for 2014 and prior years. Eligible OG&E customers include homeowners or leaseholders of a single family home, duplex condos, townhouses or mobile home constructed prior to Participants must meet three of the following eligibility criteria: 13 Attic insulation less than or equal to R-22; Wall insulation equal to or less than R-4; Floor insulation equal to R-0; Single pane windows with no storm windows attached; Heating system less than or equal to 78% AFUE; Cooling system with SEER of 10 or less; and Air infiltration problems identified through either a pre-blower door test or visual inspection procedures The home age requirement will be updated to a rolling 10 year age minimum beginning in Eligibility requirements are taken from AOG informational materials. Obtained from: 14 Measured air infiltration is measured at 50 Pascal (Pa) and must be greater than 2,200 CFM for households of five or fewer occupants, or greater than 2,700 CFM for households of more than five occupants. Process Evaluation Findings 3-4

170 2015 AOG/OG&E Weatherization Program EM&V Report These criteria are designed to target severely energy inefficient residences; this helps to ensure that each participating home has the potential to generate a substantial amount of energy savings through the program. In 2015 AOG and OG&E sought to provide services to larger homes in order to achieve a higher level of savings per home. During 2015 approximately 5% of participating homes were greater than 3,000 square feet, and the average square footage of participant homes was 1,753. The enrollment procedures for 2015 were consistent with prior years. Customers who are interested in participating in the program contact program staff members to sign up for the in-home audit. Additionally, prospective participants may learn about the program and apply for enrollment through a local community clearinghouse. This clearinghouse informs customers of eligibility requirements, provides informational marketing materials such as flyers, and assists customers with the program enrollment process. As with prior program years, the program uses three installation contractors who perform the weatherization and measure implementation services. During the preliminary in-home audit, contractors determine customer eligibility and identify potential energy efficiency measures for the residence. After the measures are installed, utility staff members perform post-inspections in order to verify that all measures have been properly implemented. 3.4 AOG/OG&E Weatherization Program 2015 Participation In 2015, the AOG/OG&E Weatherization Program serviced a total of 1,612 homes. This is a fairly consistent participation rate with prior years. Participants received in-home energy audits and one or more of the following measure types: Watt CFLs; Ceiling/attic insulation; Water heater pipe wrap; Water heater jacket; and Air infiltration reduction improvements. Unlike the prior year, window air conditioners were not installed in any homes during The utilities phased this measure out of the program in early Depending on the location of customers and the fuel sources used in their homes, services for each customer are funded by AOG, OG&E, or both AOG and OG&E. Table 3-5 cross-tabulates the number of participating homes by utility. As participants were only required to be customers of one of the two participating utilities, some residences in the program were serviced by utilities other than AOG and OG&E. These utilities included Process Evaluation Findings 3-5

171 Number of homes serviced APSC FILED Time: 5/2/2016 2:38:38 PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc AOG/OG&E Weatherization Program EM&V Report municipal utilities, co-ops, propane service providers, or non-participating investor owned utilities. 15 Table 3-5 Participation by Associated Utility Electric Utility AOG Gas Utility Other/None OG&E Other OG&E Total AOG Total Total Homes 1,325 1,229 1,612 Figure 3-1 displays the month of weatherization for homes serviced during the 2015 program year, based on the weatherization date listed in program tracking data. Program participation was fairly evenly distributed across the 2015 year, with the exception of November and December when zero homes were weatherized. The program met its goals and closed out by the end of October (N = 1,612) Month in 2015 Figure 3-1 Homes Serviced by Month, 2015 Table 3-6 displays the number of 2015 measure installations by measure type for each utility, arranged by the most commonly installed measures. 16 Ceiling insulation was the most common measure type, followed by air infiltration measures and CFLs. 15 The AOG total of 1,229 homes includes the 226 AOG-serviced homes for which OG&E paid the full project cost. 16 The values represent the number of homes receiving the measure, rather than the total number of measures installed at all homes. Thus, the values for CFLs do not present the total number of bulbs installed, but the total number of participants receiving at least one of that measure type. Process Evaluation Findings 3-6

172 2015 AOG/OG&E Weatherization Program EM&V Report Table 3-6 Total Implementations by Measure Measure Type Number of attributable installations generating savings AOG OGE Ceiling Insulation 1,089 1,177 Air Infiltration 1,060 1,156 CFL 895 1,266 Water Heater Jacket/Pipe Total 3,708 4,313 There were only minor variations in measure distribution and measure types implemented in 2015 compared to prior years. The notable differences between 2015 and 2014 measure counts include a substantial increase in the number of water heater jacket and pipe wrap measures installed in OG&E serviced homes (in 2014 there were only 279 such installations). Overall, the number of instances of measure installation increased by 679 for AOG and 202 for OG&E. 3.5 Utility Staff Member Interviews As part of the 2015 program evaluation, interviews were conducted with utility staff members responsible for managing the AOG/OG&E Weatherization Program. These interviews primarily served to assess the status of previous evaluation conclusions and recommendations, as well as to identify notable changes in program operation, delivery, and performance. Additionally, as 2016 marks the first year that all Arkansas IOUs will be implementing weatherization services as part of the Core Weatherization Program framework, the interviews addressed what operational changes are planned moving forward. These findings are based on utility staff in-depth interviews, as well as program documentation and periodic communications with program staff Data Collection and Reporting Successful Coordination and Error Checking: Staff noted that there were no major issues with data accuracy, program applicant eligibility, or quality of measure installation in As with prior years, staff have continued to check participation and enrollment data against their internal databases in order to ensure that the participant is one of their customers and that there are no discrepancies between the data sources. Staff also noted that there were no communication issues between AOG and OG&E, and that they are able to work together very efficiently at this point. No Issues with Data Collection Requirements: When asked about any increased data collection requirements as a result of TRM modifications, staff indicated that there were no significant modifications to the level of data needed for any of the program measures. Process Evaluation Findings 3-7

173 2015 AOG/OG&E Weatherization Program EM&V Report Additionally, staff noted that Frontier Associates has been able to provide data in an efficient and accurate manner. With regard to the prior recommendation for tracking data to include the individual air infiltration measures installed in each residence, staff noted that this had not yet occurred. However, Frontier Associates noted that these data are stored and the EnerTrek system could be programmed to generate a report with air infiltration measure details Program Marketing Additional Marketing Conducted in 2015: Utility staff explained that in order to generate additional awareness and interest in the program, additional marketing efforts were initiated during the 2015 program year including a radio campaign conducted by OG&E, and sending bill inserts to customers. AOG also mentioned that they had instructed their metering staff to inform customers of the program when on-site; metering staff were provided with informational brochures containing descriptions of all available residential programs. AOG staff also presented their programs at a property owners association in Fort Smith, and attended a home show to discuss all of AOG s energy efficiency services. Staff noted that although these marketing efforts likely improved program awareness, the program continues to generate a substantial amount of interest through word-of-mouth marketing among customers Program Implementation Focus on Larger Homes: Program staff noted that they focused on providing weatherization services to larger homes in 2015 after determining that more comprehensive work could be performed on residences of 3,000 square feet or greater. In order to provide sufficient services to these larger homes, the budget per home was increased to $3,500. This program change resulted in greater savings per home, as savings for both ceiling insulation and air infiltration reduction measures are based on the square footage of the residence. Residence Age Modification: Rather than using a specific year of construction as a program eligibility requirement, staff explained that they will now use a rolling 10 year age requirement to determine eligibility. Although staff previously discussed reducing the year of construction requirement from 1997 to 2000, this rolling age minimum will effectively increase the population of eligible residences annually rather than requiring an update every few years. Continued OG&E Funding of AOG-OG&E Homes: As with 2014, AOG provided funding for AOG-serviced homes weatherized in the first half of 2015, but had fully expended its program budget by mid August. In order to prevent any disruptions in program operation and to continue generating savings, OG&E fully funded all projects that were completed after August of This resulted in OG&E fully funding the weatherization of 226 homes where AOG was the gas provider. Process Evaluation Findings 3-8

174 2015 AOG/OG&E Weatherization Program EM&V Report AOG Transfer of Funds from AWP: As mentioned above, AOG fully expended its program budget by mid August of AOG staff noted that in anticipation of this, they had received commission approval to transfer funds from their Arkansas Weatherization Program (AWP) budget towards the AOG-OG&E program. This transfer of approximately $64,000 allowed AOG to fund additional homes through this program, but funds were still exhausted by August. This emphasizes the high level of participant interest in the AOG- OG&E Weatherization Program and suggests that additional funding would have resulted in even more homes completed during No Change in Installation Contractors: Program staff reported that there have been no changes to the existing group of contractors who implement measures under the AOG/OG&E Weatherization Program. AOG and OG&E continued to use the same three contractor firms for audits and measure installation during 2015 as were used in prior years. The utilities continued to provide training for new contractor staff who joined these firms for the 2015 program year. For example, as the program included larger homes during 2015, one area of contractor training focused on this program change and how weatherization services may be different for larger residences. Staff noted that the contractors have continued to work well together and have traded jobs based on geographical area. Each contractor would prefer to have a larger share of the work, but currently each contractor receives approximately one third of the projects on average. No Significant Contractor Quality Issues: Staff reported that the quality of work conducted by contractors has continued to be high, and that the contractors had been able to collect all necessary data to inform savings calculations Moving Forward: Core Weatherization Program Effects of New Framework: When asked about the Core Weatherization Program framework that was developed by the Arkansas IOUs and other stakeholders and will be implemented starting in 2016, program staff explained that the framework will only require a few modifications to the AOG-OG&E program. One component of the Core Weatherization Program is that IOUs must incorporate educational elements into their weatherization offerings, and program staff noted that they have already begun to incorporate educational elements by promoting the Department of Energy s Energy Savers Guide, attending homeowner shows and meetings and presenting program offerings, and providing customers with informational brochures. Another modification based on the Core Weatherization Program framework will be to add low flow showerheads, faucet aerators, and smart powerstrips to the program measure list. Contractors will be trained on these new measures, and AOG and OG&E are planning to implement the program with the same three contractor firms moving forward. As the AOG-OG&E Weatherization Program already meets the majority of guidelines contained in the Core Weatherization Program framework, the resources required to update the AOG-OG&E program for 2016 have been fairly minimal. Process Evaluation Findings 3-9

175 2015 AOG/OG&E Weatherization Program EM&V Report 3.6 Participant Survey Findings The following section presents key findings from surveys conducted with customers who participated in the 2015 AOG and OG&E Weatherization Program. The participant survey for 2015 was primarily designed to gauge current participant satisfaction levels and to identify any trends in customer characteristics or perspectives from prior program years. For 2015, data collected via participant surveying is used in evaluating: Sources of customer awareness of the program; Customer knowledge of energy efficiency; Customer satisfaction with the program; and Customer demographics and residence characteristics. The survey also included questions designed to assess participant spillover savings attributable to the program. The spillover methodology can be found in the Impact Evaluation Findings chapter of this report. A total of 300 participants responded to the telephone survey. This section summarizes the 2015 participant survey results and provides comparisons to the 2014 participant survey findings where appropriate. A copy of the 2015 participant survey instrument can be found in Appendix A Participant Removal of Measures In order to assess whether the measures reported in program tracking data were still installed at the time of the survey, respondents were asked whether they had removed or replaced any of the equipment or energy efficiency improvements that had been installed through the program. Twenty-one of the survey respondents (7%) stated that they had removed at least one measure, and when asked to provide additional information nearly all of these participants specified that they had removed one or more light bulbs. Additionally, two respondents indicated that they had removed minor air sealing measures such as door sweeps. As the lighting savings calculation in the TRM incorporates a deemed installation rate, and a low rate of removal of small air sealing measures likely results in a negligible effect on program savings, the Evaluators did not use the participant survey findings to adjust installation rates for any measures Participant Motivations and Awareness Participant respondents were asked about how they learned of the AOG-OG&E Weatherization Program; the distribution of responses is shown in Table 3-7. As with the 2014 participant survey, participants most commonly stated that they learned about the program through their friends, relatives, or other personal acquaintances. This is consistent with the program s marketing strategy, which has primarily relied on word-of- Process Evaluation Findings 3-10

176 2015 AOG/OG&E Weatherization Program EM&V Report mouth and other forms of indirect promotion. Additionally, ten percent of respondents specifically mentioned that they had seen a program sign in a neighbor s yard, which mirrors 2014 results. 17 How did you learn of the Weatherization Program sponsored by AOG and OG&E? Table 3-7 How Participants Learned of the Program Response Percentage of Respondents* (N = 300) Word of mouth from friends, relatives, or others 50% Information that came in the mail 12% Yard sign 10% Newspaper or magazine article/ad 7% Other 7% Radio ad 6% OG&E bill message 5% AOG bill message 4% Don't know 3% TV ad 2% Contractor 1% AOG or OG&E website 1% Retailer / in store 1% *Since respondents were able to select more than one response, the sum of the percentages in the table above can exceed 100%. Participants were then asked why they decided to participate in the AOG-OG&E Weatherization Program. Table 3-8 displays the distribution of responses, and respondents were able to provide more than one response. Although respondents were provided with a list of response options, 19% of respondents selected the option of Other and provided an open-ended response indicating that they participated in the program because their home needed specific improvements. These improvements included adding insulation sealing the windows, and increasing home comfort. Among the remaining response options, respondents most commonly indicated that they participated in the program in order to reduce their gas or electric utility bills or to save energy in general. This response pattern was also found during the 2014 and earlier evaluations. When asked which of these factors was the most important in their decision 17 During the 2014 evaluation, eight percent of respondents identified yard signs as their main source of program awareness. Process Evaluation Findings 3-11

177 2015 AOG/OG&E Weatherization Program EM&V Report to participate in the program, respondents most commonly reaffirmed that they wanted to reduce their monthly utility bills. Why did you decide to sign up for the program? Table 3-8 Motivations for Participating in the Program Response Percentage of Respondents* (N = 300) To reduce my monthly electric bill 29% To save energy 24% To reduce my monthly gas bill 19% My home needed improvements 19% AOG and OG&E paid for the improvements 8% Recommendation from a friend, relative, neighbor 8% It is the right thing to do 7% Help save the environment 2% Contractor recommendation 2% OG&E recommendation or information <1% AOG recommendation or information <1% Other 11% *Since respondents were able to select more than one response, the sum of the percentages in the table above can exceed 100% Pre-participation Energy Efficiency Perspectives Survey respondents were asked a series of questions related to their involvement with energy efficiency practices. As shown in Table 3-9, more than three-quarters of respondents reported that they were at least somewhat familiar with the benefits of energy efficiency improvements prior to their participation in the program. Only 7% of respondents indicated that they were very unfamiliar with these benefits. Compared to the 2014 participant survey, respondents for the current program year indicated a higher level of familiarity with energy efficiency improvements on average During the 2014 survey, approximately two-thirds of respondents indicated that they were at least somewhat familiar with energy efficiency improvements, and 18% indicated that they were very unfamiliar with these improvements. Process Evaluation Findings 3-12

178 2015 AOG/OG&E Weatherization Program EM&V Report Table 3-9 Prior Customer Awareness of Energy Efficiency Measures Prior to the audit, how familiar were you with the benefits of installing various energy efficiency improvements similar to those offered by the Weatherization Program? Response Percentage of Respondents (N = 300) Very familiar 27% Somewhat familiar 51% Somewhat unfamiliar 12% Very unfamiliar 7% Don't know 3% When specifically asked about their prior familiarity with energy saving activities such as washing with cold water, reducing light fixture use, and adjusting heating settings, only six percent of respondents indicated that they were somewhat unfamiliar or very unfamiliar with these practices. As Table 3-10 shows, the majority of respondents considered themselves to be very familiar with these practices, although it should be noted that this question may be influenced by a social desirability bias, where respondents select a response that they believe is the most socially acceptable. 19 These results are also fairly consistent with those obtained during the 2014 evaluation. 20 Table 3-10 Prior Customer Awareness of Energy Saving Behaviors Prior to the audit, how familiar were you with various household energy saving activities such as washing with cold water, reducing your use of light fixtures, and adjusting heating system settings? Response Percentage of Respondents (N = 300) Very familiar 57% Somewhat familiar 35% Somewhat unfamiliar 3% Very unfamiliar 3% Don't know 2% In addition to asking about participants familiarity with energy efficiency, the survey included several questions related to actual energy saving behaviors and purchases that may have taken place prior to the weatherization work being performed. As shown in 19 Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (1991). Measurement and control of response bias. Measures of social psychological attitudes, 1, During the 2014 evaluation, 49% of respondents indicated that they were very familiar with these energy saving practices. Process Evaluation Findings 3-13

179 2015 AOG/OG&E Weatherization Program EM&V Report Table 3-11, approximately of respondents indicated that they had previously performed energy saving activities in their home, which closely mirrors 2014 survey results. 21 When asked to elaborate on these activities, respondents most commonly stated that they wash with cold water, turn off lights when not in use, and have lowered their thermostat. Specific commentary regarding participants past energy saving behaviors includes: I keep my thermostat at 83 in the summer and 58 in the winter so that the cooler doesn t kick on or the heat. And I keep my water heater at the lowest heat settings and I conserve water. [I try] to do my laundry before 2pm and after 7pm, and turn off lights when not using them. Just small things like turn the hot water tank off during the summer. During the night I raise the thermostat to standard temperature just to limit energy being used. [We] wash with cold water, keep the lights off, and have the thermostat low. Table 3-11 Prior Customer Involvement with Energy Saving Behaviors Prior to the audit, did you perform any common household energy saving activities? If so, which activities? Response Percentage of Respondents (N = 300) Yes 65% No 30% Don't know 4% Post-participation Energy Efficiency Perspectives In order to gauge participants perceived benefits from enrolling in the program, respondents were asked whether the program increased their familiarity with energy efficiency. As shown in Table 3-12, 76% of respondents reported that they are now at least somewhat more knowledgeable about energy efficient options for their home than they were before participating in the program. Only 12% of respondents indicated that the 21 During the 2014 evaluation, 66% of respondents indicated that they had previously performed these activities. Process Evaluation Findings 3-14

180 2015 AOG/OG&E Weatherization Program EM&V Report program did not increase their knowledge of energy efficiency. These results are also similar to those found during the 2014 program evaluation. 22 As with the 2014 and earlier program years, these results continue to suggest that participants have gained useful information from program materials, on-site auditors and installation crews, and the overall experience of receiving home improvements through the program. Table 3-12 Post-participation Familiarity with EE Measures and Behaviors As a result of your experience with the AOG and OG&E Weatherization Program, how much more knowledgeable would you say you are about energy efficiency and energy efficient options for your home? Response Yes, much more knowledgeable than before participating Yes, somewhat more knowledgeable than before participating Yes, slightly more knowledgeable than before participating No, not more knowledgeable than before participating Percentage of Respondents (N = 300) 43% 33% 13% 12% When asked whether they currently take actions to save energy in their homes, such as washing with cold water or managing their lighting loads, 83% of respondents reported that they do conduct such practices. This is an increase from the 66% of respondents who indicated that they performed these activities prior to participating, suggesting that the program is increasing participants likelihood to consider additional efforts in managing their residential energy usage. The results for this question are similar to those obtained during the 2014 evaluation, although compared to 2014 a slightly lower percentage of participants indicated that they now take additional energy saving actions During the 2014 evaluation, 76% of respondents indicated that they were at least somewhat more knowledgeable about energy efficiency than before participating in the program, and 8% indicated that they were no more knowledgeable than before participating. 23 In the 2014 evaluation, 83% of respondents indicated that they now take additional energy saving actions. Process Evaluation Findings 3-15

181 2015 AOG/OG&E Weatherization Program EM&V Report Table 3-13 Post-participation Energy Efficiency Behaviors As a result of your experience with the program, do you now take additional action to save energy in your home, such as wash with cold water, reduce your use of light fixtures, and adjust heating system settings? Response Percentage of Respondents (N = 300) Yes 70% No 26% Don't know 4% Specific examples of energy saving actions taken by participants after receiving measures through the AOG-OG&E Weatherization Program include: I do all of those things; washing with cold water, adjusting thermostat and limiting use of light fixtures. [We] adjust the heating, we manage to keep the lights off unless we need them, and we wash with cold water for certain things. [I] wash with cold water, I heat with wood in the winter because it doesn t cost me anything, adjust the heating system and use the least amount of energy as possible. In addition to asking about energy saving behaviors, the participant survey included a series of questions designed to assess whether participants have made additional, nonincentivized energy efficiency purchases since participating in the program. Responses to these questions were used to inform the spillover savings assessment that was conducted for the 2015 program year. A description of the questions used in the spillover assessment, as well as the results of this assessment, can be found in the Impact Evaluation Findings chapter of this report Participant Satisfaction As with prior program years, survey respondents were asked about their levels of satisfaction with selected elements of their experience with the 2015 AOG-OG&E Weatherization Program. Results were provided on a scale of 1 to 5, with 1 representing very dissatisfied and 5 representing very satisfied. As displayed in Table 3-14, respondents generally reported high satisfaction levels with the majority of program elements. Other than high satisfaction with their overall program experience, respondents provided the highest satisfaction ratings for the service provided by AOG and OG&E staff, the Process Evaluation Findings 3-16

182 2015 AOG/OG&E Weatherization Program EM&V Report quality of work and information provided by the installation contractor, and the performance of installed equipment. Program elements receiving relatively lower average satisfaction scores include the savings on monthly utility bills, improvement in home comfort, and information provided by the utilities regarding how to reduce utility bills. Although a small percentage of respondents expressed dissatisfaction with one or more program elements, the overall satisfaction ratings were very positive. It should also be noted that lower levels of somewhat satisfied and very satisfied ratings for some program elements are mainly due to more participants selecting neutral or don t know for that program element. Compared to the satisfaction ratings obtained for these program elements in 2014 and prior years, participants from the 2015 program year appear to have a similar level of satisfaction. Total instances of dissatisfaction have increased slightly since the 2014 program year, but the program elements that received the highest rate of dissatisfaction during the 2014 program year received fewer instances of dissatisfied ratings for the 2015 program year. Overall, given that a maximum of four percent of respondents reported being dissatisfied with any one program element in either 2014 or 2015, there do not appear to have been any significant shifts in participant satisfaction levels. Process Evaluation Findings 3-17

183 2015 AOG/OG&E Weatherization Program EM&V Report Table 3-14 Participant Satisfaction with Selected Program Elements Element of Program Experience Overall program experience The service provided by AOG and OG&E staff The quality of installation work by the contractor Information provided by the contractor The performance of the equipment installed The effort required for the application process The wait-time to receive the services Usefulness of the energy audit Information provided by AOG and OG&E on how to reduce your utility bills Improvement in home comfort The savings on your monthly utility bills Very satisfied Somewhat satisfied Satisfaction Rating (N = 300) Neutral Somewhat dissatisfied Very dissatisfied Don't know 78% 16% 4% <1% <1% 1% 77% 12% 2% 1% 2% 6% 77% 14% 4% 2% 1% 2% 77% 16% 4% 1% 1% 1% 77% 12% 4% 1% 1% 5% 75% 13% 6% 1% 1% 5% 67% 18% 10% 2% 1% 2% 67% 19% 8% <1% <1% 5% 66% 19% 8% 1% 1% 5% 61% 19% 11% 1% 0% 8% 39% 26% 17% 1% 1% 16% Following the satisfaction instrument, respondents were given the option to provide openended commentary regarding their experiences in the program, their perceptions of AOG or OG&E, or about any other topics related to energy efficiency. The majority of respondents stated that they had no additional comments, and the remaining respondents primarily used this opportunity to provide positive commentary about the program. Specific commentary of this nature includes: I think it is excellent, they covered everything and did a good job. I wouldn t know of any changes to make. I think they did everything they said they were going to do and they left the place like they found it. They covered everything they went over a lot of things that I have never heard of before. I was very pleased with it. They did a very thorough job. They added and replaced what needed to be done. Process Evaluation Findings 3-18

184 2015 AOG/OG&E Weatherization Program EM&V Report Approximately six percent of respondents also identified issues with the measures that were installed or with the work conducted by the contractors. Examples of this type of commentary include: One place I feel like they didn t do a good job in is the caulking and the sealings around the doors. I feel the contractors could have been more thorough and more knowledgeable about the types of installations covered by the program. Their clean up wasn t as good as I thought it should be, stripping and screws, stuff like that were left behind. I would like to see [that] when they come out, they do everything I had windows and doors that were not weather-stripped. Finally, some respondents provided suggestions for improving or adding measures to the program. Examples of this type of commentary include: Make the program available to households that have a bigger square footage. I think they need to work a little bit on educating people, I know a lot of people that don t know much about the thermostat. Improve the visibility of the program to others. It would have been nice to get a rebate on my windows. As with the 2014 participant survey, the results from the satisfaction instrument and openended portions suggest that nearly all participants are very satisfied with each element of their program experience. However, the results suggest that participants experienced more issues with the level of knowledge and quality of work conducted by contractors in 2015 as compared to prior years. Although these comments were only expressed by approximately six percent of all respondents, there may be opportunities for increased training or contractor education if AOG and OG&E continue to use the same contractors for the upcoming Core Weatherization Program Participant Characteristics This section presents the results of a series of survey questions related to participant and residence characteristics. Residence characteristics include the age, square footage, heating type, and water heating type of participating homes. Additionally, respondents were asked about the number of bedrooms, bathrooms, showers, and total residents in Process Evaluation Findings 3-19

185 2015 AOG/OG&E Weatherization Program EM&V Report their homes. There were only minor differences between these results and those obtained for these questions during the 2014 participant survey effort. There was a slight increase in the average square footage of residences reported by survey respondents, which likely reflects the program s efforts to provide services to larger homes in Table 3-15 Reported Age of Participant Homes Response Percentage of Respondents (N = 300) Before 1970's 32% 1970's 25% When was your home built? 1980's 17% % % % 2006 or newer 0% Don't Know 9% Refused 2% Table 3-16 Reported Square Footage of Participant Homes What is the approximate square footage of your home? Response (in square feet) Percentage of Respondents (N = 300) Less than 1,000 1% 1,001-1,500 26% 1,501-2,000 26% 2,001-2,500 19% Greater than 2,500 13% Don't know 13% Refused 2% Process Evaluation Findings 3-20

186 2015 AOG/OG&E Weatherization Program EM&V Report What type of heating system do you have in your home? 3.33% 7.67% 1.67% Natural gas heating 22.67% 64.67% Electric heating Combination of types Other Don t know (N = 300) Figure 3-2 Reported Participant Residence Heating Type What type of water heater do you have in your home? 2.33% 4.00% Natural gas water heater 28.00% 65.67% Electric water heater Other Don t know (N = 300) Figure 3-3 Reported Participant Residence Water Heating Type Table 3-17 Other Reported Participant Residence Characteristics Residence characteristic type Average number reported Bedrooms Bathrooms Showers Total residents N Process Evaluation Findings 3-21

187 2015 AOG/OG&E Weatherization Program EM&V Report 3.7 Onsite Verification Results As described in Section of this report, the Evaluators conducted onsite verification visits to 58 participant homes, supplemented by 10 telephone verifications. These site visits were conducted in order to verify complete and proper measure installation, to conduct post-implementation measurements, and to collect data regarding participant residence characteristics. This section summarizes the key findings from these field visits, highlighting results from the measure verification tasks and supplemental questionnaire Measure Verification Findings The onsite field verification showed that the weatherization measures had for the most part been installed in the quantities reported within program tracking data. Specific notes illustrating the accuracy of program tracking data include: Contact information: All residences were located at the addresses provided within the tracking data. Although the majority of telephone numbers were found to be accurate during the appointment scheduling and field visit activities, the Evaluators identified a few telephone numbers during the appointment scheduling process that were disconnected or did not belong to a program participant. However this was the case for only nine percent of attempted calls, which is an improvement over the approximately 30% of disconnected or incorrect numbers during the 2014 program evaluation. Air infiltration: For the 23 homes receiving blower door testing for air infiltration during verification site visits, the reported CFM leakage value was within 10% of the measured leakage value in approximately 60% of cases. The reported CFM leakage value was within 25% of the measured leakage value in approximately 80% of cases. There were three instances where measured leakage was more than 50% greater than reported leakage. Measured leakage was greater than reported leakage in approximately 65% of cases. Ceiling insulation: All reported instances of ceiling insulation were verified. Any identified discrepancies between reported insulation levels and measured insulation levels were very minor and infrequent. Additionally, there were no instances where the reported insulation square footage differed significantly from the observed insulation square footage. CFLs: All reported instances of CFL installation were verified. There were no significant differences between the reported quantity and verified quantity of CFLs installed, after accounting for the 97% in-service-rate specified in the TRM. Water heater measures: All reported instances of water heater jacket and pipe insulation were verified. Process Evaluation Findings 3-22

188 2015 AOG/OG&E Weatherization Program EM&V Report Field Visit Summary As with the 2014 and 2013 program years, the measure implementation data reported by the installation contractors were found to be fairly accurate and few discrepancies were identified. However, as the measured CFM leakage value was greater than the reported CFM leakage value in a majority of cases, there may be some differences between the Evaluators and the contractors blower door testing methodology. Alternatively, some customers may have removed door sweeps or other air sealing items after they were installed, and reliable duplication of blower door testing results is difficult due to the number of variables involved. As stated in prior reports, including an itemized list of air infiltration measures would assist in verifying that the work was performed as reported. 3.8 Tracking Database Review The EnerTrek database system managed by Frontier Associates includes a full list of all AOG-OG&E Weatherization Program participants, the measures that were installed in their homes, and the kwh and Therms savings associated with each measure. During the 2015 program year, the Evaluators received periodic tracking data updates as well as final tracking exports. The EnerTrek system was updated to include necessary inputs as per TRM V4.0 and TRM V5.0. Other than these updates, there were no major updates to the structure or content of program tracking data. The Evaluators previously reviewed program tracking data in 2014 in order to assess its compliance with Protocol A of the TRM, which specifies that tracking data should be checked for: Participating Customer Information; Measure Specific Information; Vendor Specific Information; Program Tracking Information; Program Costs; and Marketing & Outreach Activities. The Evaluators conducted a review of each of the above factors within the 2015 program tracking data with the exception of marketing and outreach activities as these are outside the scope of EnerTrek reporting Customer, Premise, Cost, and Vendor Information Each of these factors was assessed individually based on the guidelines stated in TRM V5.0. Overall, the Evaluators conclude the following regarding tracking data completeness: Process Evaluation Findings 3-23

189 2015 AOG/OG&E Weatherization Program EM&V Report Participating customer information was complete for nearly all participants. This included Job IDs, telephone numbers, addresses, full names, and utility account numbers for AOG and OG&E. The only exception to this was that five participants had telephone numbers that did not include an area code. Overall, this is a slight improvement from the 2014 program year. All participant records included the name of the installation contractor who performed the implementation as well as the invoice date and weatherization date. This was also the case during the 2014 program year. Tracking data included the measure and project costs for each home. As with the prior program year, premise characteristics such as home heating type, cooling type, and ceiling square footage were present for all participants where appropriate and needed. However, 439 participants were listed as having a water heating type of N/A. This also occurred in the 2014 program year, although the water heater type was included for all participants who received either water heater jackets or water heater pipe insulation Measure Specific Information The content of tracking data was found to include sufficient information for all measures in The Evaluators identified two minor issues with measure inputs: one with CFLs where the wattage was listed as 135 rather than 13, and one where the number of bedrooms was listed as 2,019 (the home s square footage). Other than these minor items, there were no issues with measure specific information in the 2015 program tracking data Tracking Data Recommendations Although program tracking data were found to be fairly complete and accurate in 2014, the tracking data review for the 2014 program year resulted in several recommendations regarding tracking data accuracy and comprehensiveness. The Evaluators reviewed the 2015 tracking data to assess whether these recommendations had been met, and whether there had been any other changes to tracking data completeness or accuracy. The Evaluators previously recommended that additional field notes be added into the tracking database, such as the presence of window air conditioner units, inprogress construction work, or whether the home configuration required any atypical methods to be performed during the contractor blower door test. These notes have not been added to the tracking data as of The Evaluators previously recommended that the tracking data include the specific measures installed as part of the air infiltration reduction process (e.g. door sweeps, window sealing). This information has not been added to the tracking data as of Process Evaluation Findings 3-24

190 2015 AOG/OG&E Weatherization Program EM&V Report The Evaluators previously recommended that the tracking data include any information collected by installation contractors related to program marketing and customer awareness during their home visits. This information has not been added to the tracking data as of The Evaluators previously recommended that the water heater type be included for all homes, even those that do not receive water heater measures. In the 2015 tracking data, the water heater type was listed as N/A for 439 homes. Although none of these homes had received water heater jackets or pipe wrap, 364 other homes that had not received these measures did have a listed water heater type. It appears that there is some inconsistency among installation contractors regarding whether to collect water heater type data for homes that do not receive water heater measures. The Evaluators recommend that water heater type be collected for all homes, as this information may be useful in the event of savings spillover calculations for water-saving measures. 3.9 Comprehensiveness Checklist Factors The Arkansas Public Service Commission has in place a set of criteria in order to determine whether a DSM portfolio qualifies as Comprehensive. These criteria are: Factor 1: Whether the programs and/or portfolio provide, either directly or through identification and coordination, the education, training, marketing, or outreach needed to address market barriers to the adoption of cost-effective energy efficiency measures; Factor 2: Whether the programs and/or portfolio, have adequate budgetary, management, and program delivery resources to plan, design, implement, oversee and evaluate energy efficiency programs; Factor 3: Whether the programs and/or portfolio, reasonably address all major end-uses of electricity or natural gas, or electricity and natural gas, as appropriate; Factor 4: Whether the programs and/or portfolio, to the maximum extent reasonable, comprehensively address the needs of customers at one time, in order to avoid cream-skimming and lost opportunities Factor 5: Whether such programs take advantage of opportunities to address the comprehensive needs of targeted customer sectors (for example, schools, large retail stores, agricultural users, or restaurants) or to leverage non-utility program resources (for example, state or federal tax incentive, rebate, or lending programs) Factor 6: Whether the programs and/or portfolio enables the delivery of all achievable, cost-effective energy efficiency within a reasonable period of time and maximizes net benefits to customers and to the utility system; Factor 7: Whether the programs and/or portfolio, have evaluation, measurement, and verification "EM&V") procedures adequate to support program management Process Evaluation Findings 3-25

191 2015 AOG/OG&E Weatherization Program EM&V Report and improvement, calculation of energy, demand and revenue impacts, and resource planning decisions. This section provides updates to the review of the AOG/OG&E Weatherization Program that was conducted by the Evaluators in prior years in relation to each factor. It should be noted that this does not provide a portfolio-wide perspective for AOG and OG&E. As these criteria are intended to evaluate a portfolio of programs as a whole, assessment of the comprehensiveness checklist factors is best suited to portfolio-level evaluations and reports. As such, a review of how the AOG/OG&E Weatherization Program fits into the overall utility profile can be found in the Evaluation of 2015 DSM Portfolio Report for AOG. The portfolio report includes the AOG/OG&E Weatherization Program in its tests for portfolio comprehensiveness, assessing the comprehensiveness checklist factors in a crossprogram context. Additionally, as there were few changes to program design and operation during the 2015 program year, this review uses the prior comprehensiveness findings as a baseline and provides updates where appropriate. Factor 1: Education, Training, Marketing, and Outreach o Assessment of Education The AOG/OG&E Weatherization Program sufficiently implements educational efforts towards its prospective participants and other customers. This includes: Providing educational materials (flyers, brochures) both to participating customers and prospective participants (e.g. metering staff providing informational brochures on-site) Providing outreach through multiple channels (in-person, utility websites, radio campaigns, bill messages, community events, homeowner association meetings) Providing education targeted to specific market barriers (focusing on connection between comfort and energy efficiency, demonstrating potential savings from program measures, providing documentation that discusses health and safety, providing documentation in Spanish to reach non-english speaking customers) Providing coordinated education from multiple entities (staff members from both utilities and each of the three program contractors) The program has enhanced its educational components by increasing the number of outreach sources and types of information provided to Process Evaluation Findings 3-26

192 2015 AOG/OG&E Weatherization Program EM&V Report customers. These educational materials also take advantage of crossprogram promotion opportunities. o Assessment of Training As mentioned in utility staff interviews and the utilities annual energy efficiency reports, the program has continued to provide updated and relevant training to its contractors. Additionally, the contractors have increased their levels of internal training and have pursued BPI and RESNET certifications for their staff members. This will serve to meet guidelines of the Core Weatherization Program, which recommends that contractors have BPI certified staff. o Marketing and Outreach Program marketing was enhanced during 2015 to generate additional interest in the program. The marketing methods of the program meet the following criteria: Address specific barriers (informing customers that the program is available at no additional cost, demonstrating potential savings and increase in comfort level, brochures are offered in both English and Spanish) Promoted by trade allies (program and non-program contractors inform prospective participants of program services and opportunities) Performed through several channels (in-person, websites, word-ofmouth, radio) Interest in the program has remained high throughout the program cycle and bridge years. The current set of marketing methods appear to be effective and the utilities have already demonstrated that marketing can be scaled up or scaled back depending on program participation rates. Factor 2: Budgetary, Management, and Program Delivery Resources The AOG/OG&E Weatherization Program currently has adequate utility and contractor staffing resources. As can be seen within the utilities annual reports, program budgets are sufficient to support the savings goals, and the overall program infrastructure is able to meet program demands. Although AOG fully expended its budget by August of the 2015 program year, this does not necessarily reflect a budgetary issue, as the program was able to provide services to nearly as many homes as were weatherized in previous years, and the utilities are able to coordinate their resources in order to prevent delays or interruptions in program delivery. Factor 3: Addressing Major End-Uses Process Evaluation Findings 3-27

193 2015 AOG/OG&E Weatherization Program EM&V Report The AOG/OG&E Weatherization Program offers a wide range of measures, which are chosen based on cost-effectiveness testing. The large list of eligible program measures covers all major end-uses for targeted customer homes, including: o HVAC systems; o Hot water measures; o Appliances (refrigerators); o Lighting; and o Building envelope measures. 24 Currently, the program does not implement all of the measures from the eligible measure list (such as refrigerators), and customers have expressed interest in receiving additional services. However, staff noted that several measures will be added to the program moving forward, such as aerators, shower heads, and smart power strips, and this measure mix will meet the guidelines of the Core Weatherization Program moving forward. Factor 4: Comprehensively Addressing Customer Needs The program comprehensively addresses the major needs of its targeted customer market by providing several benefits to participants. The program provides services to customers who likely would not otherwise make major efficiency improvements to their homes, and may not have the opportunity to participate in other utility-sponsored energy efficiency programs. Specifically, the program provides the following benefits: o Technical assistance through in-home audits; o Energy and monthly bill savings through measure installation; and o Increased comfort and/or safety for participants. Factor 5: Targeting Market Sectors & Leveraging Opportunities Consistent with prior years, the AOG/OG&E Weatherization Program focuses on a specific market of utility residential customers whose homes are sufficiently energy inefficient. This is an important program in the residential sector of portfolio offerings. The AOG/OG&E Weatherization Program is also an example of utility leveraging of available partnerships: AOG and OG&E have successfully engaged in cross-fuel coordination in order to provide combined benefits to customers of one or both utilities. Factor 6: Cost-Effectiveness of Energy Efficiency 24 A complete list of eligible measures for the AOG-OG&E Weatherization Program can be found in utility documentation such as filing documents and annual reports (for example: Process Evaluation Findings 3-28

194 2015 AOG/OG&E Weatherization Program EM&V Report The program is designed to cost-effectively generate net savings and meet the stated annual program goals. It has been successful in these efforts thus far, meeting specific criteria such as: o Meeting net savings goals (overall program net-to-gross ratio is approximately 0.98, program has met goals through 2015); o Meeting industry norms for net-to-gross (expected net-to-gross of close to 1); and o Meeting cost-effectiveness goals (the program is designed to meet costeffectiveness on the measure level and as a whole, and has been successful in doing so). 25 Factor 7: Adequacy of EM&V Procedures The AOG/OG&E Weatherization Program was reviewed for EM&V procedures in the following areas: o QA/QC and EM&V procedures conducted by utility staff; o QA/QC and EM&V procedures conducted by installation contractor staff; and o QA/QC and EM&V procedures conducted by the Evaluators. As stated previously, the quality assurance and verification procedures currently conducted by utility staff and installation contractors are adequate for monitoring implementation quality and ensuring the accuracy of ex ante installation records. The Evaluators field data were fairly consistent with reported tracking data values, indicating that overall measure implementation is recorded accurately and consistently. 26 Program staff has historically been responsive in resolving data issues, and have been responsive to data collection needs. There are some minor data-related recommendations that have not yet been fully resolved but the existing QA/QC and data collection processes are sufficient overall. These results indicate that the AOG/OG&E Weatherization Program continues to contribute to utility portfolio comprehensiveness as a residential services offering. As mentioned above, a full review of the utilities portfolio comprehensiveness checklist factors can be found in the utilities portfolio-wide evaluation reports. 25 Further information regarding program cost-effectiveness can be found in utility-specific cost-benefit spreadsheets on the Arkansas Public Service Commission (APSC) website: 26 See Section 3.7 of this report for detailed information regarding program implementation verification. Process Evaluation Findings 3-29

195 4. Conclusions and Recommendations After reviewing the AOG/OG&E Weatherization Program for 2015, the Evaluators provide the following conclusions: Continued Participant Satisfaction: The results of the 2015 participant survey suggest that nearly all participants are very satisfied with each element of their program experience. This is consistent with the results obtained during 2014 and prior years. Although some respondents had issues with the level of knowledge or level of service provided by contractors, instances of dissatisfaction were very infrequent and the majority of participants provided positive comments about their experiences. Consistent Participant Characteristics: As shown in Section of this report, there do not appear to have been any significant changes in the demographics or residential characteristics of participating customers since the 2014 program year, other than a slight increase in the average square footage of participant homes. Continued Cross-Fuel Coordination: As with the 2014 program year, AOG fully expended its program budget by late August of 2015, and OG&E fully paid the cost of providing services to 226 participating homes that were customers of both AOG and OG&E. The Evaluators previously noted in 2014 that by focusing on homes that receive utility service from both AOG and OG&E, AOG would be able to fund a greater number of homes and possibly retain some funding for later in the program year. This occurred to some extent, as the percentage of participating homes receiving service from both AOG and OG&E increased slightly in 2015 compared to 2014 (58% and 54%, respectively), and more AOG customers participated in 2015 (1,229 as compared to 1,172 in 2014). Overall, this does not necessarily reflect a program performance issue as the program was able to provide services to nearly as many homes as were weatherized in previous years, and the utilities were able to coordinate their resources in order to prevent delays or interruptions in program delivery. Minimal Core Weatherization Transition Effects: The AOG-OG&E Weatherization Program is currently well suited to transitioning into the new Core Weatherization Program framework that has been developed by the Arkansas IOUs and other stakeholders. As many aspects of this framework referenced the design and operation of the AOG-OG&E program as a successful model, complying with the guidelines of the new framework will likely require few changes to this program. Likely modifications include incorporating aerators, showerheads, and smart power strips into the program measure mix, maintaining or increasing BPI certification of participating contractor staff, and slightly modifying program eligibility requirements such as residence age and square footage. Adequate Data Collection Procedures: As with the 2014 and 2013 program years, the measure implementation data reported by the installation contractors were found to be fairly accurate and few discrepancies were identified. The water heating type was not Conclusions and Recommendations 4-1

196 2015 AOG/OG&E Weatherization Program EM&V Report collected for some homes that did not receive water heating measures, and the Evaluators noted in 2014 that these data may be useful for tracking purposes and spillover calculations. However, the discrepancies were infrequent and minor, and do not appear to indicate any systematic issues with program delivery. Adequate Database Quality: The Evaluators found the ex ante savings values within the EnerTrek database to be accurate for nearly all measures. Additionally, Frontier Associates was very consistent in responding to data requests and correcting errors when necessary. The main persisting discrepancy between ex ante and ex post calculations was that the EnerTrek database continued to calculate annual CFL savings as an increment of lifetime savings, which incorporated future baseline changes that are not relevant to first-year savings. The Evaluators previous recommendation to modify this approach was not adopted, but this is a minor issue as the discrepancy only slightly affects CFL realization rates. Some Spillover Effects: The spillover savings assessment conducted for the 2015 program year found spillover savings equal to approximately 0.14% of total gross savings for AOG and OG&E. Spillover mainly consisted of low-cost measures such as water heater insulation and lighting, but one customer installed ENERGY STAR windows and another installed an efficient furnace due to information and recommendations received during their participation in the program. As the program targets homes that are considered severely energy inefficient and participating customers may not have the means to improve their home s efficiency on their own, the level of spillover is likely to be low in any given year. The AOG/OG&E Weatherization Program was very successful in The Evaluators identified few specific, systematic or persistent issues with program operation and design. As the utilities plan to continue offering similar services and maintaining their current operational structure under the Core Weatherization Program, consideration of the following recommendations may be useful moving forward: Include Itemized Air Infiltration Measures: This recommendation is carried over from the prior evaluation, and the decision to adopt it may depend on programming costs. The initial home audit data collection form and the post-implementation measure verification form both include fields for detailed measure information and additional field notes. However, some of this information is not present in the tracking data exports. Frontier Associates has noted that this information is stored, but there is not an existing report format that will provide these details. The itemized measures and any field notes would be useful from an EM&V standpoint but if the programming costs to generate such a report are high, it would likely be more cost-effective to maintain the current approach of on-site visual inspection and blower door testing. Specify Propane Customers in Tracking Data: The tracking data included the electric utility provider for each participating home, and identified the gas utility provider as either AOG or None. The classification of None for a gas utility would intuitively refer to Conclusions and Recommendations 4-2

197 2015 AOG/OG&E Weatherization Program EM&V Report customers who are all-electric, but 30 of these customers are listed as having gas heat and had ex ante gas savings within the tracking data. Therefore these customers are most likely receiving propane service. In order to distinguish between a tracking data error and the presence of a propane customer, the Evaluators recommend that contractors identify propane customers on their audit forms and that the EnerTrek system generates a separate gas utility designation for these customers. Consider Working with Contractors to Ensure Sufficient Customer Service: The EM&V efforts and utilities QA/QC procedures have found that the participating contractors are sufficiently trained in the audit and measure installation process, but some customers have expressed that their contractor could have been more effective in explaining either the types of services that were being provided or explaining energy use and efficiency options in general. Based on the survey results this represents only a small percentage (fewer than 10%) of participants. However working with the contractors to ensure that they leave the home with the customer feeling aware of the work performed and knowledgeable in additional energy efficiency options would likely serve to further increase customer satisfaction. Additionally, customers with a higher level of knowledge regarding their energy use, home weatherization, and energy efficiency opportunities are more likely to implement measures that may qualify for spillover savings. Table 4-1 presents the above items, outlining the relevant issue, potential consequences, and associated recommendations. Conclusions and Recommendations 4-3

198 2015 AOG/OG&E Weatherization Program EM&V Report Table 4-1 Recommendations from 2015 Program Year Evaluation Issue Consequences Recommendation Tracking data do not include specific measure details for some items (i.e. whether door sweeps, window sealing, etc. were installed, whether lighting was installed indoors or outdoors) Difficult to completely inspect measure installation Limits level of detail possible for measure tracking Depending on programming costs, include an itemized list of all air infiltration reduction measures installed in each home. The Gas Utility field within the tracking data uses the classification of None to refer to both all-electric and propane customers. Some customers expressed that their contractor did not fully explain the services provided or could have been more knowledgeable Difficult to distinguish between a tracking data error and a propane customer. Negatively effects customer satisfaction, reduces spillover potential. Collect and report specific utility providers and identify propane customers within the program tracking database. Consider working with contractors to ensure that they provide sufficient customer service and are able to sufficiently answer questions about energy efficiency options and provide recommendations. Conclusions and Recommendations 4-4

199 Appendix A: Participant Survey Instrument AOG/OG&E Weatherization Program Participant Telephone Survey ID No. Customer Name: Date of interview: Date data entered... Hello. May I please speak with [CONTACT NAME]: )? Hello. My name is and I m calling from [Surveying Company Name] on behalf of Oklahoma Gas & Electric and Arkansas Oklahoma Gas [if necessary, refer to AOG and OG&E, the customer s utility companies] about the weatherization program your household participated in this year. Are you the person who is most familiar with your household s participation in this program? (IF NOT RIGHT PERSON) May I please speak to the person who would know the most about your household s participation in this program? REPEAT INTRODUCTION AND CONTINUE (IF RIGHT PERSON) We are conducting a study to evaluate AOG and OG&E s Weatherization Program, which is a partnership program between AOG and OG&E to offer energy efficiency and weatherization improvements to customers who have either AOG, OG&E, or both AOG and OG&E as their utility provider. AOG and OG&E will use the results of this evaluation to determine the effectiveness of the program and to make improvements. We would like to include feedback about your experience with the program in our evaluation. Q-1 Our records indicate that you participated in AOG and OG&E s Weatherization Program this year by completing an energy audit and receiving several energy efficient measures installed in your home. Do you recall participating in this program? Yes [SKIP TO Q-4] No [THANK RESPONDENT AND TERMINATE INTERVIEW] Don t know [ASK TO SPEAK WITH SOMEONE IN THE HOME WHO MAY KNOW] Q-2 Is there anyone else in your household who may be familiar with your household s participation in the program? Yes [SKIP TO Q-3] Appendix A: Participant Survey Instrument A-1

200 2015 AOG/OG&E Weatherization Program EM&V Report No [THANK RESPONDENT AND TERMINATE INTERVIEW] Don t know [THANK RESPONDENT AND TERMINATE INTERVIEW] Q-3 May I speak with that person? Yes [RETURN TO Q-1 AND BEGIN QUESTIONS WITH NEW RESPONDENT] No [THANK RESPONDENT AND TERMINATE INTERVIEW] Don t know [THANK RESPONDENT AND TERMINATE INTERVIEW] RESPONDENT BACKGROUND At this time, I d like to let you know that your responses to this survey will be kept completely confidential. I ll begin with a few questions about your decision to participate in the program. Q-4 How did you learn of the Weatherization Program sponsored by AOG and OG&E? [SELECT ALL THAT APPLY] Information that came in the mail Newspaper or magazine article/ad Contractor Word of mouth from friends, relatives, or others TV ad Radio ad AOG bill message OG&E bill message AOG website OG&E website Retailer / in store Other (Specify) Don t know [DO NOT READ] Q-5 Why did you decide to sign up for the program? [SELECT ALL THAT APPLY] To reduce my monthly gas bill To reduce my monthly electric bill AOG and OG&E paid for some or all of the improvements Contractor recommendation AOG recommendation or information OG&E recommendation or information Recommendation from a friend, relative, neighbor It is the right thing to do Help save the environment Save energy Other (Specify) Appendix A: Participant Survey Instrument A-2

201 2015 AOG/OG&E Weatherization Program EM&V Report Q-5A Of the things you mentioned, which was the most important? To reduce my monthly gas bill To reduce my monthly electric bill AOG and OG&E paid for some or all of the improvements Contractor recommendation AOG recommendation or information OG&E recommendation or information Recommendation from a friend, relative, neighbor It is the right thing to do Help save the environment Save energy Other (Specify) MEASURE INSTALLATION Next, I have some questions about the work that was performed in your home through the Weatherization Program. Q-6 Since the work was performed, have you removed or replaced any of the equipment or energy efficiency improvements implemented in your home through the program? Yes (Please specify which items have been removed or replaced): No Don t know OVERALL ENERGY EFFICIENCY DECISION MAKING Q- 14 Before you participated in the AOG and OG&E Weatherization Program, had you purchased any energy efficient items or equipment on your own? If so, which ones? Yes (Please explain): [GO TO Q-15] No Q-15 Since you participated in this program, have you installed any energy efficient equipment in your home that you have not received an incentive for? Yes [ASK Q-15A] No [SKIP TO Q-16] Q-15A What energy efficient items have you installed on your own since participating in this program? [PROMPT WITH ITEMS IN TABLE IF NECESSARY. FOR ITEMS NOT LISTED, ASK RESPONDENT TO DESCRIBE THE ITEM IN AS MUCH DETAIL AS POSSIBLE] (In the following table, please Appendix A: Participant Survey Instrument A-3

202 2015 AOG/OG&E Weatherization Program EM&V Report indicate the quantity of each item type purchased, or specify another item type and quantity) CFLs Measure Type Water Heater Pipe Insulation Water Heater Jacket/Blanket/Insulation LED Light Bulbs Low Flow Bathroom Aerators Low Flow Kitchen Aerator Low Flow Showerhead LED Nightlights Other insulation (specify location) Energy Star Appliances (specify type) Other: Quantity Purchased Q-15B Did you have specific plans to purchase any of these additional items before you signed up for the Weatherization Program? 1. Yes (please specify which items were already planned): 2. No 3. Don t know Q-15C On a scale of 0 to 10 where 0 represents Not at all important and 10 represents Extremely important, how important was your experience with the AOG/OG&E Weatherization Program in your decision to purchase and install these additional items? 1. [RECORD NUMBER 0 10] 2. Don t know Q-15D On a scale of 0 to 10 where 0 represents Not at all likely and 10 represents Extremely likely, how likely would you have been to purchase and install these additional items if you had never participated in the AOG/OG&E Weatherization Program? 1. [RECORD NUMBER 0 10] 2. Don t know Appendix A: Participant Survey Instrument A-4

203 2015 AOG/OG&E Weatherization Program EM&V Report Q-15E Which, if any, of these energy efficiency improvements were recommended during the Weatherization Program energy audit? [VERBATIM]: Q- 16 Prior to the audit, how familiar were you with the benefits of installing various energy efficiency improvements similar to those offered by the Weatherization Program? Very familiar Somewhat familiar Somewhat unfamiliar Very unfamiliar Don t know Q-16A Prior to the audit, how familiar were you with various household energy saving activities such as washing with cold water, reducing your use of light fixtures, and adjusting heating system settings? Very familiar Somewhat familiar Somewhat unfamiliar Very unfamiliar Don t know Q-16B Prior to the audit, did you perform any common household energy saving activities? If so, which activities? Yes (please explain): No Don t know Q-17 As a result of your experience with the Weatherization Program, would you buy energy efficient items in the future, even if they cost more than standard items? Yes No Q-18 As a result of your experience with the AOG and OG&E Weatherization Program, would you say you are any more knowledgeable about energy efficiency and energy efficient options for your home? Yes, much more knowledgeable than before participating Yes. somewhat more knowledgeable than before participating Yes, slightly more knowledgeable than before participating No, not more knowledgeable than before participating Q-18A As a result of your experience with the program, do you now take additional action to save energy in your home, such as wash with cold Appendix A: Participant Survey Instrument A-5

204 2015 AOG/OG&E Weatherization Program EM&V Report water, reduce your use of light fixtures, and adjust heating system settings? Yes (please explain): No Don t know PROGRAM SATISFACTION Now I d like to ask you about your satisfaction with several aspects of this program. Q-19 On a scale of 1 to 5, where 5 is very satisfied and 1 is very dissatisfied, and a 3 is neutral, how would you rate your satisfaction with the following? Element of Program Experience Information provided by the contractor The quality of installation work by the contractor The performance of the equipment installed The savings on your monthly utility bills The effort required for the application process The wait-time to receive the services The service provided by AOG and OG&E staff Information provided by AOG and OG&E on how to reduce your utility bills Improvement in home comfort Usefulness of the energy audit Overall program experience 5- Very Satisfied 4- Somewhat Satisfied 3- Neither Satisfied or Dissatisfied 2- Somewhat Dissatisfied 1- Very Dissatisfied Don't Know Q-20 (If any item in Q-19 rated 2 or 1) Why were you dissatisfied with [Program Element]? [VERBATIM]: Q-21 Are there any changes or improvements you would like to see for the Weatherization Program? [VERBATIM]: Appendix A: Participant Survey Instrument A-6

205 2015 AOG/OG&E Weatherization Program EM&V Report DEMOGRAPHICS Finally, I have a few questions about your household. As a reminder, your responses will remain confidential. Q-22 When was your home built? [IF RESPONDENT DOES NOT GIVE VERBATIM ANSWER, READ OFF YEAR RANGES UNTIL RESPONDENT INDICATES ONE] Verbatim Before 1970 s 1970 s 1980 s or newer Don t know [DON T READ] Refused Q-23 What is the approximate square footage of your home? [IF RESPONDENT DOES NOT GIVE VERBATIM ANSWER, READ OFF SIZE RANGES UNTIL RESPONDENT INDICATES ONE] Verbatim Less than 1,000 1,001-1,500 1,501-2,000 2,001-2,500 Greater than 2,500 Don t know [DON T READ] Refused Q-24 How many bedrooms are there in your home? Quantity: Don t know [DON T READ] Refused Q-25 What type of heating system do you have in your home? Natural gas heating Electric heating Combination of types (Specify): Other (Specify): Don t know [DON T READ] Q-26 What type of water heater do you have in your home? Appendix A: Participant Survey Instrument A-7

206 2015 AOG/OG&E Weatherization Program EM&V Report Natural gas water heater Electric water heater Other (Specify): Don t know [DON T READ] Q-27 How many bathrooms are there in your home? Quantity: Don t know [DON T READ] Refused Q-28 How many showers are there in your home? Quantity: Don t know [DON T READ] Refused Q-29 Including yourself, how many people currently live in your home year-round? Quantity: Don t know [DON T READ] Refused Q-30 Do you have any other comments that you would like to relay to AOG or OG&E about energy efficiency in residences or about their programs? [VERBATIM] This completes the survey. Your input is greatly appreciated and will be used to help improve AOG and OG&E s energy efficiency programs in the future. Thank you very much for your time! Appendix A: Participant Survey Instrument A-8

207 Attachment D: AEG s Evaluation of Remaining Programs

208 Oklahoma Gas & Electric (OG&E) Arkansas Energy Efficiency Portfolio (EE) Evaluation Report PY2015 Final Report Applied Energy Group, Inc. 500 Ygnacio Valley Road, Suite 250 Walnut Creek, CA Prepared for: Oklahoma Gas & Electric February 29, 2016

209

210 This report was prepared by Applied Energy Group, Inc. 500 Ygnacio Valley Blvd., Suite 250 Walnut Creek, CA Project Director: C. Williamson Project Manager: G. Cook D. Burdjalov P. Ignelzi K. Parmenter J. Prijyanonda B. Ryan

211

212 EXECUTIVE SUMMARY As per regulatory requirements, in 2014 OG&E Arkansas implemented the approved modified EE Portfolio of programs to the approved EE plan for Applied Energy Group ( AEG ) evaluated four programs implemented by OG&E in Arkansas: Multi-Family Direct Install, Student Energy Education LivingWise, Commercial Lighting, and Commercial and Industrial Standard Offer, This report covers evaluated savings for program year 2015 (PY2015), and li fetime savings by measure and program, provides findings and recommendations for improvements, addresses OG&E s response to PY 2014 recommendations as well as the Comprehensive Factors. Approach AEG s evaluation of the PY2015 programs included checking compliance with the Technical Reference Manual (TRM 4.0 or TRM 5.0 as applicable), replication of savings from prescriptive measures, engineering reviews of program results, applying net-to-gross values to AEG-adjusted savings, and determining net lifetime impacts by program and measure. Process evaluation activities included in-depth interviews with program staff and implementers, comparing database tracking to recommendations from Protocol A (Program Tracking and Database Development), and in-depth interviews with property manager participants in the Multi-Family Direct Install program. Evaluation Results OG&E s programs were very successful in 2015, achieving 108% of planned net energy savings (kwh) and 90% of planned net demand reduction (kw). As in previous years, the Commercial Lighting program provided most of the savings (over 84 million kwh and 885 kw), but the C&I Standard Offer program and the Multi-Family Direct Install program also made significant contributions. In addition to verifying the savings reported by OG&E, AEG calculated lifetime impacts 2 for the program and measures. As part of this process, in the body of the report we refer to the impacts (energy savings or peak demand reduction) accrued during the program year being evaluated (PY2015) as first year impacts. 3 Table ES-1 shows the OG&E goals, reported gross impacts, AEG evaluated first year gross energy savings (18,297,269 kwh) and gross demand reductions (2,245 kw), gross realization rates (95% for kwh, 98% for kw), net impacts (17,542,535 kwh, 2,167 kw), net-to-gross (NTG) ratios, and lifetime impacts. 1 Order NO. 55, Docket TF 2 Lifetime impacts are the sum of energy savings over the course of the measure s effective useful life (EUL) and the weighted average demand reduction across the lifetime of the measure divided by the EUL (in years). 3 The first year impacts for PY2015 are the impacts for one full year of measures installed during 2015 as part of the program that was implemented in PY2015. Applied Energy Group, Inc. i

213 Table ES-1 Impact Participation Energy Savings (kwh) Annual Demand Reduction (kw) Summary of OG&E Arkansas PY2015 Evaluation Results Metric MF Direct Install SEE LivingWise Com Lighting C&I Standard Offer 4 Goals 2,832 1, ,099 Reported 1,795 1, ,878 Goals (Net) 2,851, ,792 6,599,411 6,541,238 16,281,175 Reported (Gross) 4,407, ,100 6,821,191 7,772,630 19,273,892 Evaluated (Gross) 4,408, ,603 6,595,356 6,978,237 18,297,269 Realization Rate 100% 116% 97% 90% 95% Evaluated (Net) 3,969, ,745 6,529,402 6,717,507 17,542,535 NTG Ratio 90% 103% 99% 96% 96% % of Goal (Net) 139% 113% 99% 103% 108% Lifetime (Net) 60,362,655 2,768,789 84,559,682 67,881, ,572,224 Goals (Net) , ,402 Reported (Gross) , ,298.1 Evaluated (Gross) , ,245.1 Realization Rate 99% 126% 107% 85% 98% Evaluated (Net) , ,166.6 NTG Ratio 90% 101% 99% 96% 97% % of Goal (Net) 119% 114% 110% 63% 90% Lifetime (Net) , Total 4 C&I Standard Offer includes Direct Install measures in the Executive Summary Table, but they are separated elsewhere within the report. 5 Weighted average lifetime demand reduction per program will not sum to the total, since the total is the annual demand reduction expected over the course of 20 years. Applied Energy Group, Inc. ii

214 Contents EXECUTIVE SUMMARY... I Approach... i Evaluation Results... i 1 Introduction... 1 Background for OG&E Arkansas EE Programs... 1 Residential Program Descriptions... 1 Multi-Family Direct Install Program... 1 Student Energy Education LivingWise Program... 3 Commercial & Industrial (C&I) Program Descriptions... 4 Commercial Lighting... 4 Commercial and Industrial Standard Offer Program (C&I SOP)... 4 Structure of the Report Evaluation Methods... 6 Process Evaluation Methods... 6 Impact Evaluation Methods... 6 Impact Evaluation Tasks... 7 Estimate Gross Impacts and Gross Lifetime Impacts... 9 Estimate Net Impacts and Net Lifetime Impacts Residential Measure Algorithms and EULs Algorithms Effective Useful Life of Measures Commercial & Industrial Measure Algorithms and EULs Algorithms Effective Useful Life of Measures Residential Program Findings...24 Multi-Family Direct Install Evaluation Process Evaluation Impact Evaluation Net Impacts SEE LivingWise Program Process and Impact Evaluation Net Impacts Residential Programs Adherence to Protocol A Commercial & Industrial Program Findings...43 Process Evaluation for C&I Programs In-Depth Interviews C&I Programs Adherence to Protocol A Commercial Lighting Impact Evaluation Gross Impacts Net Impacts Applied Energy Group, Inc. iii

215 C&I Direct Install & Standard Offer Impact Evaluation Gross Impacts Net Impacts Lifetime Impacts Net Lifetime Impacts by Program Net Lifetime Impacts by Measure Recommendations Multi-Family Program SEE LivingWise Program Commercial Lighting Program C&I Standard Offer Program C&I Direct Install C&I Standard Offer Response to PY2014 Recommendations SEE LivingWise Program Multi-Family Direct Install Program Commercial Lighting Program Standard Offer Program Comprehensive Factors A Program Manager and Implementer Interview Guide... A-1 B Multi-Family Property Manager Interview Guide... B-1 C Documentation Review Results... C-1 Commercial Lighting Program... C-1 C&I Standard Offer Program... C-6 D Lifetimes Impacts by Program, Measure, and Year... D-1 Lifetime Gross Energy (kwh) Savings... D-1 Lifetime Net Energy (kwh) Savings... D-1 Lifetime Gross Weighted Average Demand (kw) Reduction... D-2 Lifetime Net Weighted Average Demand (kw) Reduction... D-2 E Glossary... E-1 Applied Energy Group, Inc. iv

216 List of Figures Figure 1-1 PY2014/PY2015 SEE LivingWise Kit... 3 Figure 4-1 Example Print Ad for OG&E Commercial Programs Figure 4-2 Summary Counts of Main Issues in Commercial Lighting Documentation Review Figure 5-1 Lifetime Energy (kwh) Savings by Measure Figure 5-2 Lifetime Energy (kwh) Savings by Program Applied Energy Group, Inc. v

217 List of Tables Table ES-1 Summary of OG&E Arkansas PY2015 Evaluation Results..ii Table 1-1 Customer Inducement Per Apartment Unit... 2 Table 1-2 Incentives for C&I Direct Install Measures... 5 Table 2-1 Summary of Impact Evaluation Activities... 6 Table 2-2 C&I Standard Offer Documentation Review Sample Design Matrix... 8 Table 2-3 Commercial Lighting Documentation Review Sample Design Matrix: Direct Options Projects Only... 9 Table 2-4 Energy and Demand Intensities for Water in OG&E Territory Table 2-5 TRM 5.0 Interactive Effects Factors for Energy and Cooling Demand Impacts Table 2-6 EULs for Measures in the Multi-Family Direct Install Program Table 2-7 EULs for Measures in the SEE LivingWise Program Table 2-8 EULs for Measures in the Commercial Lighting Program Table 2-9 EULs for Measures in the C&I Direct Install Program Table 2-10 EULs for Measures in the C&I Standard Offer Program Table 3-1 Multi-Family Direct Install Program: Summary of First Year Gross Impacts Table 3-2 Multi-Family CFLs: Summary of Gross Impacts Table 3-3 Multi-Family Faucet Aerators: Summary of Gross Impacts Table 3-4 Multi-Family Low-Flow Showerheads: Summary of Gross Impacts Table 3-5 Multi-Family Air Infiltration Reduction: Summary of Gross Impacts Table 3-6 Multi-Family Duct Sealing: Summary of Gross Impacts Table 3-7 Multi-Family Power Strips: Summary of Gross Impacts Table 3-8 Multi-Family Direct Install Program: PY2015 First Year Impacts Table 3-9 Distribution of SEE LivingWise Kits in PY Table 3-10 TRM-Calculated Impacts by Measure, Per Unit Installed Table 3-11 Derivation of Weighted Interaction Factors for CFL Installation Impacts Table 3-12 SEE LivingWise Evaluated Gross Program Impacts Summary Table 3-13 Calculation of Embedded Energy and Demand Impacts Table 3-14 Natural Gas and Electric 2015 Btu Savings Comparison (Aerators & Showerheads).. 35 Table 3-15 Faucet Aerator Evaluated Gross Impacts Table 3-16 Low-Flow Showerhead Evaluated Gross Impacts Table 3-17 CFL Evaluated Gross Impacts Table 3-18 Energizing Indiana Education Program Free Ridership, Spillover, and NTG Summary. 40 Table 3-19 SEE LivingWise Evaluated Net Program Impacts by Measure Table 3-20 SEE LivingWise Program Goals, and Reported and Evaluated Impacts Table 3-21 SEE LivingWise In-Service Rate Comparison between PY2015 and PY Table 3-22 Adherence to Protocol A: Residential Programs Table 4-1 Adherence to Protocol A Commercial Programs Table 4-2 Commercial Lighting: Summary of Evaluated Gross Impacts Table 4-3 Commercial Lighting: Summary of Savings Replication Results Table 4-4 Energy Savings Replication Results for Custom Lighting Projects Applied Energy Group, Inc. vi

218 Table 4-5 First Year Demand Reduction Replication Results for Custom Lighting Projects Table 4-6 C&I Lighting Documentation Review Results: Sampled Projects Table 4-7 Table 4-8 Table 4-9 Expansion of Sample Data to Population Using Ratio Estimation: Commercial Lighting Projects Implemented by Direct Options Achieved Precision of the Impact Estimates: Commercial Lighting Projects Implemented by Direct Options Evaluated Lifetime Energy and Demand Impacts: Commercial Lighting Projects Implemented by Direct Options Table 4-10 Overall First Year Impacts for the PY2015 Commercial Lighting Program Table 4-11 C&I Direct Install & Standard Offer Programs: Summary of Evaluated Gross Impacts. 55 Table 4-12 C&I Direct Install Measures: Summary of Evaluated Gross Impacts Table 4-13 Mapping of EISA 2007 Wattages to Baseline Wattages Table 4-14 C&I Standard Offer Documentation Review Results: Sampled Projects Table 4-15 Expansion of Sample Data to Population Using Ratio Estimation: C&I Standard Offer Projects Table 4-16 Achieved Precision of the Impact Estimates: C&I Standard Offer Projects Table 4-17 Evaluated Lifetime Energy and Demand Impacts: C&I Standard Offer Projects Table 4-18 First Year Impacts for PY2015 C&I Direct Install and Standard Offer Programs Table 5-2 Table 5-1 Lifetime Energy (kwh) Savings and Weighted Average Lifetime Demand (kw) Reduction by Program Lifetime Energy (kwh) Savings and Weighted Average Lifetime Demand (kw) Reduction by Measure Table 8-1 End Uses Addressed by OG&E Arkansas Portfolio of Programs Applied Energy Group, Inc. vii

219

220 SECTION 1 Introduction Background for OG&E Arkansas EE Programs In January 2006, the Arkansas Public Service Commission (APSC) began the rulemaking for developing and implementing energy efficiency programs for Arkansas s four electric utilities. By May of 2007, these rules were finalized, adopting protocols and proced ures for testing the costeffectiveness of energy efficiency (EE) programs and conducting evaluation, measurement, and verification (EM&V) of reported impacts. In October 2007, OG&E introduced a Quick Start Program in the Arkansas jurisdiction. Two of the Quick Start measures, Weatherization and Education, were collaborative efforts by all Arkansas utilities. In June 2011, the APSC approved OG&E s portfolio of energy efficiency programs for that program year ( Energy Efficiency and Load Management Plan). In Sept 2011, OG&E filed a revised proposal to achieve the energy savings goals required by the Order for the 2012 and 2013 program years, Oklahoma Gas & Electric s Arkansas Energy Efficiency Program Analysis and Plan prepared by Frontier Associates, Sept 2011 ( the Plan ). OG&E Electric Services offers retail electric service in Oklahoma and Arkansas, servicing approximately 65,000 customers in Arkansas. OG&E s Arkansas service area encompasses the City of Fort Smith and several nearby municipalities. In 2010, OG&E s Arkansas retail customer classes used 2,700,703 MWh which is 10.8% of all OG&E energy. Applied Energy Group (AEG) evaluated the results for PY 2015 for two Commercial and Industrial (C&I) programs Commercial Lighting, Standard Offer, and two Residential Programs Multi- Family, Student Energy Education LivingWise. Residential Program Descriptions This section briefly describes the two residential programs. Multi-Family Direct Install Program In 2014 OG&E implemented the Multi-Family Direct Install program targeting the underserved market of residential customers in multi-family dwellings 6. OG&E identified over 13,000 multi-family units in its service area, representing almost 25% of their residential customers. OG&E engaged a third-party implementer (CLEAResult) to promote the program and reach out to property management companies, property owners, and tenants. The program provides energy saving fixtures and installation at no cost to the customer; replacement fixtures include Compact Fluorescent Lights ( CFL s ), water heating pipe insulation, low flow shower heads, faucet aerators, advanced power strips and duct sealing and air sealing on electrically heated homes. The incentive structure includes incentive payments to the contractor covering the entire cost of measures and installation, and an incentive for the participating property management groups and owners. The number of bathrooms in each unit determines the number of measures installed and the contractor incentive amount. Table 1-1 shows the costs for the program measures. 6 Source: In the matter of the request for approval of its quick start energy efficiency programs and the tariff related to the program by Oklahoma Gas And Electric Company, Docket no TF, Direct Testimony and Exhibits of Billy Dean Pollock on behalf of Oklahoma Gas and Electric Company, February 14, Applied Energy Group, Inc. 1

221 APSC FILED Time: 5/2/2016 2:38:38 Introduction PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Table 1-1 Customer Inducement Per Apartment Unit 7 Direct Install Measure Contractor Incentive Per Measure Customer Incentive Per Unit Faucet Aerator $ 5.00 Shower Head $ CFLs $ 5.00 Water Heater Pipe Wrap (max. 4 ft.) $5.00/ft. $ per unit APS 8 $ Air Sealing 6 $87.50 Duct Sealing 6 $87.50 This program complements OG&E s existing programs servicing multi-family properties. The implementer screens and enrolls contractors to perform work, recruits customers (property owners/management groups) to participate, coordinates project installations, processes project completion forms for payment and inspects completed work. Measures may be purchased through CLEAResult s preferred vendor or directly by participating contractors as long as the measures meet program requirements. OG&E staff and contractors actively promote the program directly to customers and a CLEAResult Account Lead recruits, educates and maintains contact with qualifying property management groups and property owners. All work is performed by independent contractors, which promotes local businesses and allows contractors to develop relationships with property management groups and facilitate additional energy efficiency projects. CLEAResult leverages its relationships with local contractors that perform these services to recruit new contractors. Contractors complete a participation agreement and are required to maintain liability and worker s compensation insurance and maintain high customer satisfaction ratings with OG&E s customers. The implementer provides contractors with marketing materials and forms for project completion such as: Customer Flyer; Customer Enrollment Form/Agreement; Contractor Direct Install Participation Agreement; Contractor Direct Install Form; and Tenant Fact Sheet/Flyer. The implementer s Measurement and Verification (M&V) process for measures with deemed savings includes pre- and post-installation inspections on a sample of projects with pictures and information to verify installed measures and kwh and kw impacts per project. CLEAResult does pre-inspections during the initial qualification of the property to verify units eligible for replacement, document existing wattages and flow rates of eligible measures. In addition, during the pre-inspections the CLEAResult inspectors coordinate with the installing contractor, arrive on a site at random times, and sample a percentage of each type of unit. 7 Source(s): In the matter of the request for approval of its quick start energy efficiency programs and the tariff related to the program by Oklahoma Gas And Electric Company, Docket no TF, Direct Testimony and Exhibits of Billy Dean Pollock on behalf of Oklahoma Gas and Electric Company, February 14, CLEAResult, OG&E Multi-Family Direct Install Statement of Work, January 15, Measures piloted in preparation for Duct sealing and air sealing were incentivized as a package at $175/unit. Applied Energy Group, Inc. 2

222 APSC FILED Time: 5/2/2016 2:38:38 PM: Introduction Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Student Energy Education LivingWise Program The Student Energy Education LivingWise (SEE LivingWise ) program is a turnkey program, with all activities managed directly by the provider, Resource Action Programs (RAP). Nothing in the structure or delivery of the program in PY2015 changed from the previous years and the items in the kit remained the same as PY2014. So, our description of the program and its compliance with Arkansas guidelines are the same. For completeness, we largely repeat that here from our previous evaluation reports, with updates as appropriate. Additional details about the program implementation are included in the SEE Process and Impact Evaluation discussion later in this report. The purpose of the SEE LivingWise program is to shape household behaviors about resource (energy and water) use and encourage reduced energy use through a combination of information about resource efficiency and access to efficient products. The program has been in operation since Under the program, 6 th grade students in participating schools are each provided with a take-home kit containing energy and water efficiency devices and are exposed to information about energy efficiency, both in the classroom and through materials in the kit. Under contract to OG&E, RAP implements its LivingWise program by enrolling schools and furnishing the materials and training to teachers who then conduct the in-classroom lessons and provide the students with take-home kits (shown in Figure 1-1) that contain several energy and water savings devices, along with additional information about how to install the devices and save resources. The PY2014/PY2015 LivingWise kits included: a low-flow showerhead, 3 CFLs (compared with 1 in earlier years), a kitchen faucet aerator, a bathroom faucet aerator, an LED nightlight, other items designed to help families check for energy inefficiencies in their homes, and a curriculum for teachers. Both the kits and the RAP website contain explicit instructions on how to install each of the items. Figure 1-1 PY2014/PY2015 SEE LivingWise Kit A participant is defined as a student. Under the program, each participant is issued a kit with the above noted items. The impacts the program expects to realize and that OG&E is reporting, derive from the installation of three item types (six individual items) in the kit: the low-flow showerhead, the 3 CFLs, and the kitchen and bathroom faucet aerators. OG&E claims no impacts for the other items in the kit. Applied Energy Group, Inc. 3

223 APSC FILED Time: 5/2/2016 2:38:38 Introduction PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 To meet the program objectives and impact goals, OG&E provides RAP with a list of potential schools eligible to participate. Each year, RAP researches the number of eligible students/schools in the area. Teachers may enroll in any of several ways via telephone, , or website. RAP also mails letters to and calls the schools each year. Teachers can also contact RAP or OG&E to request inclusion of their classes in the program. RAP confirmed that they have no trouble enrolling teachers into the program to meet the goal for number of kits distributed. The number of kits available is limited by the program budget. To optimize savings, RAP prioritizes teacher invitations based, in part, on their demonstrated past performance, as evidenced by returns in the student surveys. Once the quota is reached each year, RAP stops recruitment. RAP confirmed that once recruited, no teacher is turned away. RAP has created a set of instructional materials and measures for students to install at home. They also provide an educational curriculum for the teachers to use. In a quick review of the student materials, we found that the installation instructions seem complete and easy to follow. The kits come with specialized tools to install and measure the low-flow showerheads; the faucet aerators and CFLs require common or no tools to install. The RAP manager told us that the company keeps in contact with the enrolled teachers. Teachers have a phone number to contact RAP. As part of enrollment, RAP asks teachers to have students complete surveys about their understanding of energy and their installation of the measures at home. The student survey responses provide key program year-specific information, most importantly about which and how many of the measures are installed. As inducement, teachers return surveys from at least 80% of their students are offered a gift card for purchase of educational materials or supplies for their classrooms. At the outset each program year, RAP calculates an average per-kit savings based on the then most current TRM and some assumptions about installation and net-to-gross rates. Later in the year, RAP updates the estimates to make use of guidelines in the new TRM issued during the program year. RAP sends electronic reports to OG&E throughout the year on the number of kits delivered to classrooms and the associated impacts. RAP provides OG&E with a final report after the program year is complete that shows the number of kits delivered, as well as their final estimates of annual kwh and kw impacts for the program year. OG&E maintains a tracking system that shows the number of participants in the program each year and recorded impacts. The data are provided by RAP and transferred into the Saratoga tracking system by OG&E. OG&E uses the participation information and impact estimates provided by RAP as the reported amounts for the program year. Commercial & Industrial (C&I) Program Descriptions This section describes the two C&I programs that were evaluated. Commercial Lighting The Commercial Lighting program provides incentives to Arkansas C&I customers who purchase and install energy efficient indoor and outdoor lighting, lighting controls, light emitting diode (LED) exit lights in both retrofit and new construction applications. Incentive levels for the lighting measures are based on $0.12 per kwh of energy saved. The incentives are based on the kwh savings calculated from a lighting survey that takes into account the type and quantity of lighting fixtures replaced, the new fixtures installed, the building type, and any control technologies in place. Commercial and Industrial Standard Offer Program (C&I SOP) This program has two components which we refer to as C&I Standard Offer and C&I Direct Install. Applied Energy Group, Inc. 4

224 APSC FILED Time: 5/2/2016 2:38:38 PM: Introduction Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 C&I Standard Offer This program component offers financial incentives of $0.12/kWh for the installation of a wide range of measures that reduce customer energy costs, reduce peak demand, and/or save energy in non-residential facilities such as public authority buildings, schools, hospitals, and other industrial customers. Large individual customers, energy service companies, and qualified contractors are all eligible to participate in the C&I SOP. The C&I SOP provides incentives for many energy efficiency measures that are not covered under other OG&E programs. C&I Direct Install OG&E also engaged CLEAResult to install pre-rinse spray valves, CFLs, and faucet aerators at no cost to the customer as a direct install component of this program. Unit costs for the measures installed through are shown in Table 1-2 below. Table 1-2 Direct Install Measure Incentives for C&I Direct Install Measures Contractor Incentive per Unit Pre-rinse spray valves $140 CFLs $4 Faucet aerators $13 Structure of the Report This report is structured as shown below: Section 1, Introduction Section 2, Evaluation Methods Section 3, Residential Program Findings Section 5, Commercial & Industrial Program Findings Section 6, Lifetime Impacts Section 7, Recommendations Section 8, Response to PY2014 Recommendations Section 9, Comprehensive Factors Appendices Applied Energy Group, Inc. 5

225 SECTION 2 Evaluation Methods Process Evaluation Methods Process evaluations focus on determining the overall effectiveness of program delivery, identifying opportunities for program improvements and assessing key program metrics, including participation rates, market barriers, and overall program operations. The process evaluation included the following research tasks: Interviews with program managers (Residential and C&I) and implementation contractors. Interviews with property managers participating in the Multi-Family program. Review compliance with Protocol A. Response to PY2014 recommendations. Recommendations for improvements to the program(s). For the SEE program, we reviewed the design of the program, the measures, and the way it is deployed. This is a turnkey program, meaning that all aspects of the implementation are handled by the implementation contractor. We obtained detailed explanation about the design of the program and how it was deployed in PY2015 compared with past years from discussions with the OG&E program staff and the implementation contractor. Impact Evaluation Methods Table 2-1 summarizes the evaluation activities AEG carried out during the impact analysis for the residential and C&I programs. The subsections below discuss each activity in greater detail. Table 2-1 Evaluation Activity Summary of Impact Evaluation Activities SEE LivingWise Multi-Family Direct Install Commercial Lighting C&I Standard Offer C&I Direct Install Algorithm Review Tracking System Review Replication of Savings Calculations Complete Documentation Review (for sample) (for sample) Estimation of Gross Impacts Estimation of Net Impacts Applied Energy Group, Inc. 6

226 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Impact Evaluation Tasks Tracking System Review AEG reviewed tracking systems and summary reports maintained by OG&E and each program implementer to verify consistency between the tracking systems and to assess how well the tracked fields comply with Protocol A requirements. For the impact evaluation, we paid particular attention to whether or not the implementers tracking systems contained all the variables required to determine energy and demand impacts. Our review also included a basic verification to ensure that the sum of the project-level savings from implementer reports equaled the total reported impacts in OG&E s reports. Replication of Savings Calculations AEG used information in the tracking systems to replicate energy savings and demand reduction calculations for as many projects as possible to verify that the project-level reported impacts were calculated correctly. The steps were as follows: Requested and received spreadsheets with detailed tracking data from implementers. Developed automated calculations and lookup tables to replicate savings estimates based on current deemed savings algorithms in TRM 5.0. Compared the replicated estimates with reported impacts and identified discrepancies. Requested additional clarification from implementers for projects with discrepancies. Finalized adjusted impact estimates for 2015 and for subsequent years across the lifetime of each project. Developed project-level (and program-level) realization rates for 2015 based on the adjusted estimates. We replicated savings and determined realization rates for the following program datasets: Multi-Family Direct Install (all projects). Commercial Lighting (all projects implemented by Direct Options and OG&E; partial year of projects implemented by CLEAResult 9 ). C&I Direct Install (all projects). Due to the more custom nature of some of the C&I Standard Offer projects and since the tracking reports did not contain all the parameters required to calculate impacts, AEG reviewed detailed documentation for a sample of individual C&I Standard Offer projects implemented by Direct Options, CLEAResult, and OG&E to evaluate impacts. AEG also reviewed detailed documentation for a sample of Commercial Lighting projects implemented by Direct Options to verify the accuracy of the information recorded in the tracking databases. For SEE LivingWise we did not attempt replication of the savings calculations. It is our understanding that the implementer used TRM 5.0 and additional assumptions to develop per-kit savings which served as the PY2015 reported savings. AEG ultimately calculated our own permeasure and per-kit estimates as described below. Algorithm Review AEG verified that the Arkansas TRM algorithms were applied to PY2015 impact estimates. TRM Version 5.0 is the version currently in effect. AEG reviewed all databases and calculators to check for TRM compliance and identified instances where the TRM values were not applied properly. 9 AEG replicated savings calculations for the CLEAResult projects that were reported in the tracking database through early September The resulting realization rates were then applied to full year results to estimate impacts for all of CLEAResult s lighting projects. Applied Energy Group, Inc. 7

227 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 For SEE LivingWise we reviewed and applied the algorithms in TRM 5.0. Wherever possible, we used program-specific data instead of the TRM default values in the calculations. We present the algorithms and effective useful life (EUL) used for each measure later in this section. Complete Documentation Review For the Commercial Lighting and C&I Standard Offer programs, AEG did a complete documentation review of a sample of projects to assess reported impacts and come up with adjusted impact estimates for 2015 and for subsequent years across the lifetime of each program. The documentation review was completed for 20 C&I Standard Offer sample points and 30 Commercial Lighting sample points. Sample Design for C&I Standard Offer AEG designed a stratified random sample to select projects for the C&I Standard Offer documentation review. While a simple random sample involves selecting sample points at random from the entire population, a stratified random sample involves selecting sample points at random from the population within mutually exclusive groups called strata. For this program, we used the reported kwh savings as the stratification variable for the projects. As long as the stratification variable is correlated with the variable of interest, in this case evaluated savings, using a stratified design increases the precision of the estimates holding sample size constant, or decreases sample size holding precision constant. Our first step was to specify the sample frame, which consisted of the 81 unique Rebate IDs in PY2015. The next step was to determine the stratification variable and number of strata. We used the magnitude of kwh savings for stratification and planned for four strata: 1) low savings; 2) medium savings; 3) high savings; and 4) highest savings (a census stratum comprising the seven Rebate IDs with highest kwh savings). We next applied the Dalenius-Hodges method to determine stratum boundaries and estimated the precision we would achieve with 20 sample points. Finding the precision acceptable, we allocated the 20 sample points to the strata using a Neyman Allocation. This method assigns sample points to each stratum based on a combination of the weights and standard deviation (amount of variation) for each stratum. Finally, we randomly selected sample points for Strata 1, 2 and 3 (Stratum 4 was a census stratum). Table 2-2 shows the resulting sample design. Table 2-2 C&I Standard Offer Documentation Review Sample Design Matrix Stratum Reported Savings (kwh) # in Population (Rebate IDs) # in Sample 1. Low savings 19, Medium savings > 19,000 and 69, High savings > 69,000 and 220, Highest savings (census) > 220, Total Sample Design for Commercial Lighting For the Commercial Lighting program, AEG carried out a documentation review for a sample of projects implemented by Direct Options. The focus was on that subset of projects because the tracking system review revealed more inconsistencies in tracked data for those projects than for projects implemented by CLEAResult. Applied Energy Group, Inc. 8

228 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 AEG designed a segmented and stratified random sample to select projects implemented by Direct Options for the documentation review. We first segmented the Direct Options lighting projects into four groups: 1) retrofit; 2) sensor; 3) delamp; and 4) new construction. For the retrofit group, which contained 181 unique Fixture IDs, we designed a stratified random sample to select 19 sample points from the population within four mutually exclusive strata. For this analysis, we used the replicated kwh savings as the stratification variable for the retrofit projects. We then used the Dalenius-Hodges method to determine stratum boundaries and a Neyman Allocation to assign the sample points to the strata. For the sensor, delamp, and new construction groups, our sample design included a census of projects because each of these segments contained only a few projects. In all, the sample size was 30 Fixture IDs. Table 2-3 shows the resulting sample design for the review of Direct Options lighting projects. Table 2-3 Commercial Lighting Documentation Review Sample Design Matrix: Direct Options Projects Only Stratum Replicated Savings (kwh) # in Population (Fixture IDs) # in Sample Retrofit Projects (stratified random sample) Low savings 20, Medium savings > 20,000 and 100, High savings > 100,000 and 400, Highest savings (census) > 400, Sensor Projects (census) 6 6 Delamp Projects (census) 4 4 New Construction Projects (census) 1 1 Total Estimate Gross Impacts and Gross Lifetime Impacts For the Multi-Family Direct Install program, the C&I Direct Install program, and custom lighting projects implemented by OG&E through the Commercial Lighting program, AEG summed the project-level adjusted savings from the replication step to estimate the program-level gross impacts for 2015 and for subsequent years across the lifetimes of the measures. For Commercial Lighting projects implemented by CLEAResult, AEG multiplied the realization rates determined during the replication of partial year results by the full year 2015 reported impacts to estimate the first year program-level gross impacts. For program-level lifetime impacts, AEG multiplied the lifetime impacts determined for the partial year of projects by the ratio of full-year to partial-year 2015 reported impacts. For C&I Standard Offer and Commercial Lighting, AEG expanded the adjusted impacts from the sample to estimate the impacts for the population of projects using a combined ratio estimate according to the steps below. We used this same estimation approach for the energy and demand impacts. Here are the steps for C&I Standard Offer: 1. Calculated the average reported impacts and average adjusted impacts by stratum based on the sample results. Did this for each year in the lifetime ( ). Applied Energy Group, Inc. 9

229 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc For each year, calculated weighted average reported impacts and weighted average adjusted impacts using weights that reflected the proportion of customers in each stratum of the population. 3. For each year, calculated the ratio of the sample adjusted weighted average to the sample reported weighted average to come up with a combined ratio. 4. For each year, multiplied the combined ratio by the program s total PY2015 reported population impacts to estimate the evaluated gross impacts for 2015 through The steps for Commercial Lighting are similar, but they tie back to the lifetime replicated impacts instead of the first year reported impacts: 1. Calculated the average replicated impacts and average adjusted impacts by stratum and segment based on the sample results. AEG did this for each year ( ). 2. For each year, calculated weighted average replicated impacts and weighted average adjusted impacts using weights that reflect the proportion of customers in each stratum and each segment in the population. 3. For each year, calculated the ratio of the sample adjusted weighted average to the sample replicated weighted average to come up with a combined ratio. 4. For each year, multiplied the combined ratio by the total replicated impacts for the given year for the population of 2015 participants. This step yielded evaluated gross impacts for 2015 through We analyzed the impacts from the SEE LivingWise program using the following approach: 1. Applied TRM 5.0 algorithms/assumptions to data to calculate per-unit measure savings. 2. Calculated and applied in-service rates and any other installation data from participant surveys. 3. Calculated gross annual energy and demand impacts by measure and for the program as a whole. 4. Calculated lifetime gross energy savings for each measure using the EULs in TRM 5.0 and baseline wattages for CFLs using a combination of replaced lamp wattage for first year and EISA-compliant values for future year savings. 5. Calculated the lifetime demand reduction as a weighted average annual kw demand reduction value calculated by dividing the sum of annual kw reductions by the EUL of the measure. The SEE LivingWise tables, figures, and related text have all been updated to reflect results for the PY2015 program. The key differences between this report and PY2014 are that the impacts have been calculated based on PY2015 participant data. They incorporate measure installation rates and water and space heating fuel shares from the survey responses of this year s participants. Estimate Net Impacts and Net Lifetime Impacts There were no significant changes to the Arkansas DSM programs in PY2015. Therefore, AEG used the same net-to-gross (NTG) values from PY2014 to estimate net impacts for each program. Applied Energy Group, Inc. 10

230 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Residential Programs AEG used evaluations of comparable programs in Illinois 10 and in Maine 11 to determine the NTG ratio for the Multi-Family Direct Install measures used in OG&E s program (the NTG ratio was 0.90 for both programs). This approach is consistent with the TRM protocols (Figure 5: Decision Tree for Application of Programmatic NTG). 12 As in past years, and in agreement with the IEM, AEG did not conduct an independent assessment of NTG ratios to calculate net impacts for SEE LivingWise. In PY2014, AEG used the NTG estimates from a comparable student education program in Indiana to determine net impacts for the SEE LivingWise program; these same NTG estimates were used for PY2015. Commercial & Industrial Programs AEG applied the 2014 NTG ratios used by other programs in Arkansas 13 (0.96 for C&I SOP custom, 1.0 for Direct Install, 0.99 for Commercial Lighting) for the PY2015 evaluation. Net Lifetime Impacts AEG applied the appropriate NTG ratios to the gross lifetime impacts to determine net lifetime impacts for each program and measure. Residential Measure Algorithms and EULs Impacts and EULs for PY2015 measures in the SEE LivingWise and Multi-Family Direct Install programs were determined in accordance with the Arkansas TRM Version 5.0. Algorithms Faucet Aerators Energy savings and peak demand reductions were calculated using the following algorithm: kwh or Therms = ρ C p ΔV (T mixed T supply ) ISR (RE Conversion Factor) (1) kw = kwh DSF (2) Where: ρ Cp ΔV Tmixed = Water density, 8.33 lbs/gal (TRM default) = Specific heat of water, 1 Btu/lb F (TRM default) = Gallons of water saved per faucet, calculated with default TRM values: gal/year for a faucet rated at 1.5 GPM gal/year for a faucet rated at 1.0 GPM 14 = Mixed water temperature at faucet (TRM default of F for Fort Smith) Tsupply = Average water main temperature (TRM default of 66.1 F for Fort Smith) 10 Com Ed Programs NTG Approach for Programs values were applied for EPY8. ( Note that the Com Ed MF Direct Install program did not include air infiltration, duct sealing, and power strips so the entire program NTG of 0.9 was used for these measures. 11 Source: Opinion Dynamics, Efficiency Maine, Multifamily Efficiency Program Evaluation, Final, March 17, Source: Arkansas TRM 4.0 Volume 1 Protocols, August 29, Comparable Arkansas programs included SWEPCO Commercial Energy Efficiency and Small Business Direct Install programs and Entergy Arkansas C&I Custom and C&I Prescriptive Programs. 14 The rounded TRM default values are 381 gal/year and 636 gal/year for 1.5 and 1.0 GPM faucets; AEG calculated the non-rounded value using the TRM water consumption formulas for baseline (2.2 GPM) and installed faucets. Applied Energy Group, Inc. 11

231 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 ISR = In-service rate, dependent on survey response (only for SEE LivingWise ) RE = Recovery efficiency of water heater, excluding standby losses (TRM default): 0.98 for electric resistance water heaters 2.2 for heat pump water heaters for gas water heaters Conversion Factor = 3,412 Btu/kWh for electric or 100,000 Btu/Therm or gas DSF = Demand savings factor, ratio of kw peak demand to kwh annual energy use: kw/kwh (TRM default) Embedded energy savings for water use reduction were also calculated based on an earlier study done by the evaluation team 16 to reflect electricity savings due to avoided water supply and wastewater treatment. The water savings from each project (in gallons) were multiplied by the total energy and demand intensity values (after conversion to gallons from mega-gallons) that are seen in Table 2-4. Fort Smith values were used for the faucet aerator and low-flow showerhead measures in the SEE LivingWise program. The weighted averages for the territory were used for the faucet aerator and low-flow showerhead measures in the Multi-Family Direct Install program to accommodate the program s incorporation of prior guidance from the evaluation team. Table 2-4 Location Energy and Demand Intensities for Water in OG&E Territory Drinking Water Energy Intensity (kwh/mg) Wastewater Total Drinking Water Demand Intensity (kw/mg) Wastewater Oklahoma City, OK 2,996 1,806 4, Ardmore, OK 1,470 3,287 4, Muskogee, OK 1,389 2,274 3, Fort Smith, AR 480 1,917 2, Weighted Average 2,401 1,914 4, Total The embedded energy savings and embedded demand reduction were then added to the direct energy savings (Equation 1) and direct demand reduction (Equation 2) calculated through TRM algorithms to result in final savings values. For PY2016, AEG recommends that OG&E use the Fort Smith energy and demand intensities from Table 2-4 to calculate embedded impacts for all programs. That approach will lead to more accurate impact estimates since most Arkansas participants are located in Fort Smith. It will also allow for greater consistency across the programs. Low-Flow Showerheads Energy savings and peak demand reductions were calculated using the following algorithm: 15 While the AR TRM states that the median efficiency factor for heat pumps in the AHRI database is 2.2 Btu/Btu, AEG found that the median efficiency for heat pump water heaters is 2.3 Btu/Btu from a June 2014 AHRI database extract. 16 Parmenter, K., Ehrhard, R., Cook, G. and Williamson, C. Embedded Energy Savings from Water Saving Measures: Electricity Savings Due to Avoided Water Supply and Wastewater Treatment, Jan Applied Energy Group, Inc. 12

232 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 kwh or Therms = ρ C p ΔV (T mixed T supply ) ISR (RE Conversion Factor) (3) kw = kwh DSF (4) Where: ρ Cp ΔV Tmixed = Water density, 8.33 lbs/gal (TRM default) = Specific heat of water, 1 Btu/lb F (TRM default) = Gallons of water saved per showerhead, calculated with default TRM values: 1, gal/year for a showerhead rated at 2.0 GPM 2, gal/year for a showerhead rated at 1.75 GPM 3, gal/year for a showerhead rated at 1.5 GPM 17 = Mixed water temperature at shower (TRM default of F for Fort Smith) Tsupply = Average water main temperature (TRM default of 66.1 F for Fort Smith) ISR = In-service rate, dependent on survey response (only for SEE LivingWise ) RE = Recovery efficiency of water heater, excluding standby losses (TRM default): 0.98 for electric resistance water heaters 2.2 for heat pump water heaters 0.79 for gas water heaters Conversion Factor = 3,412 Btu/kWh for electric or 100,000 Btu/Therm for gas DSF = Demand savings factor, ratio of kw peak demand to kwh annual energy use: kw/kwh (TRM default) Embedded energy impacts for water use reduction were also calculated with the same factors as for faucet aerators. Fort Smith values were used for the SEE LivingWise program and the weighted territory averages were used for the Multi-Family Direct Install program. As noted above, AEG recommends that OG&E use the Fort Smith energy and demand intensities from Table 2-4 to calculate embedded impacts for all programs for PY2016 to improve the accuracy of impact estimates and to allow for greater consistency across the programs. Compact Fluorescent Lights Energy savings and peak demand reductions were calculated using the following algorithm: kwh = ( W base W post ) AOH ISR IEF 1000 E (5) kw = ( W base W post ) ISR IEF 1000 D CF (6) Where: Wbase Wpost AOH = Baseline wattage for the existing lamp Use EISA Tier 1 baseline for years Use EISA Tier 2 baseline for years 2023 and later = Wattage of installed CFL = Annual operating hours, hours/year (TRM default) 17 The rounded TRM default values are 1,457 gal/year and 3,246 gal/year for 2.0 and 1.5 GPM showerheads; AEG calculated the nonrounded value using the TRM water consumption formulas for baseline (2.5 GPM) and installed showerheads. Applied Energy Group, Inc. 13

233 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 ISR = In-service rate, 0.98 for retail and direct install programs (TRM default) 18 IEFE = Energy interactive effects factor (see Table 2-5) IEFD = Interactive effects factor for cooling demand reduction (see Table 2-5) CF = Coincidence factor (TRM default): 0.10 for indoor fixtures 0 for outdoor fixtures Table 2-5 TRM 5.0 Interactive Effects Factors for Energy and Cooling Demand Impacts HVAC System Type IEFE IEFD Gas Heat with AC Gas Heat with No AC Electric Resistance with AC Electric Resistance with no AC Heat Pump Heating/Cooling Type Unknown Air Infiltration Reduction Energy savings and peak demand reductions were calculated using the following algorithm: kwh = CFM 50 ESF (7) kw = CFM 50 DSF (8) Where: ΔCFM50 = reduction in infiltration in ft 3 /min at 50 pascals, as measured by the difference between pre- and post-installation blower door air leakage tests ESF DSF = Energy Savings Factor (TRM default for Zone 8 Fort Smith): kwh/cfm50 for gas heat and AC system kwh/cfm50 for gas heat and no AC kwh/cfm50 for electric resistance heat and AC system kwh/cfm50 for heat pump system = Demand Savings Factor (TRM default for Zone 8 Fort Smith): kw/cfm50 for homes with either AC or heat pump systems 0 kw/cfm50 for homes without AC or heat pump systems 18 The TRM default is not used for the SEE LivingWise program; the results from the participant survey are used to calculate the ISR for this program. Applied Energy Group, Inc. 14

234 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Duct Sealing Energy savings and peak demand reductions for duct sealing measures in homes with central electric heating and cooling systems were calculated per apartment using the following algorithms: kwh Cooling = (DL pre DL post ) EFLH C (h out ρ out h in ρ in ) 60 (1, 000 SEER) (9) kwh Heating = (DL pre DL post ) 60 HDD (1, 000 Eff heat ) (10) kw = kwh Cooling EFLH c CF (11) Where: DLpre DLpost = Pre-improvement duct leakage at 25 Pascals, in ft 3 /min (CFM25) This value is capped at 40 percent of total fan flow, as per the TRM 19 = Post-improvement duct leakage at 25 Pascals, in ft 3 /min (CFM25) EFLHc = Equivalent full load cooling hours (TRM default of 1432 hours/yr for Fort Smith) hout hin = Outdoor design specific enthalpy (39 Btu/lb TRM default for Fort Smith) = Indoor design specific enthalpy (29 Btu/lb TRM default for Fort Smith) ρout = Outdoor air density at 95 F (TRM default of lb/ft 3 ) ρin = Conditioned air density at 75 F (TRM default of lb/ft 3 ) SEER = Seasonal Energy Efficiency Ratio of existing system (recorded) HDD = Heating degree days (TRM default of 3,919 HDD for Fort Smith) = Volumetric heat capacity of air (Btu/ft 3 - F) CF = Coincidence factor of 0.87 (TRM default) 60 = Factor to convert from minutes to hours 1,000 = Factor to convert from W to kw 24 = Factor to convert from days to hours Effheat = Efficiency of heating system. For electric resistance heat, use Btu/Wh For heat pump, use the prescribed Heating Seasonal Performance Factor (HSPF) 21 Advanced Power Strips Energy savings and peak demand reductions per installed advanced power strip were calculated using TRM 5.0 average savings values of kwh/unit and 0.03 kw/unit for the home entertainment system application. 19 Total fan flow = cooling capacity (tons) * A base temperature of 65 F is used for HDD in this evaluation. 21 TRM defaults prescribe minimum efficiency of 6.8 HSPF for pre-2006 heat pump units and a minimum efficiency of 7.7 HSPF for units manufactured after January 2006, per the U.S. DOE. Applied Energy Group, Inc. 15

235 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Effective Useful Life of Measures Multi-Family Direct Install For the Multi-Family Direct Install program, the first-year savings are expected to persist for the entirety of a measure s EUL, except for CFLs, for which the first year savings are based on the existing baseline. The next 7 years for CFL savings are based on a First Tier (2007) EISA standard baseline. For the last (9 th ) year of the measure life the savings are determined with a Second Tier (2020) EISA standard baseline. Table 2-6 lists the EULs applicable to the Multi- Family Direct Install program. Table 2-6 EULs for Measures in the Multi-Family Direct Install Program Measure EUL CFL 9 Faucet Aerator 10 Low-Flow Showerhead 10 Air Infiltration Reduction 11 Duct Sealing 18 Advanced Power Strip 10 SEE LivingWise For the SEE LivingWise program, savings for faucet aerators and low-flow showerheads are assumed to persist for the whole measure lifetime. For CFLs, the first-year savings are based on the existing baseline; the next 7 years, the savings are based on a First Tier (2007) EISA standard baseline; the last (9 th ) year of the measure life, the savings are determined with a Second Tier (2020) EISA standard baseline. Table 2-7 lists the EULs applicable to the SEE LivingWise program. Table 2-7 EULs for Measures in the SEE LivingWise Program Measure EUL CFL 9 Faucet Aerator 10 Low-Flow Showerhead 10 Commercial & Industrial Measure Algorithms and EULs The following depicts the TRM 5.0 algorithms and EULs used to determine gross impacts for C&I measures in PY2015 as well as lifetime impacts. Algorithms Lighting Retrofits Energy savings and peak demand reductions were calculated per measure using the following algorithms: kwh = ( N basew base N post W post ) AOH IEF 1000 E (12) kw = ( N pre(i)w pre(i) N post(i) W post(i) ) CF IEF 1000 D (13) Applied Energy Group, Inc. 16

236 Where: Npre(i) APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 = Pre-retrofit number of fixtures of type i Npost(i) = Post-retrofit number of fixtures of type i Wpre(i) = Rated wattage of pre-retrofit fixtures of type i (Appendix E of TRM 5.0) Wpost(i) = Rated wattage of post-retrofit fixtures of type i (Appendix E of TRM 5.0) 22 AOH = Annual operating hours for specified building type (Table 362 of TRM 5.0) CF = Coincidence factor, based on building type (Table 362 of TRM 5.0) IEFE = Energy interactive effects factor (Table 363 of TRM 5.0) IEFD = Interactive effects factor for cooling demand reduction (Table 363 of TRM 5.0) Per TRM 5.0, Equations 12 and 13 do not have an in-service rate (ISR) factor. However, there may be circumstances for which an ISR of less than 1.0 would be warranted. For example, other program jurisdictions have used an ISR of for CFLs installed through a direct install program delivery approach. 23 Lighting in New Construction Energy savings and peak demand reductions for efficient commercial lighting in new construction projects were calculated using the following TRM 5.0 algorithms: kwh = ((SF LPD 1000 ) (N post(i) W post(i) 1000 )) AOH IEF E (14) kw = ((SF LPD 1000 ) (N post(i) W post(i) 1000 )) CF IEF D (15) Where: SF = Total affected square footage of the new construction facility LPD = Maximum power density by building type (TRM 5.0 default, Table F1) Lighting Controls Energy savings and peak demand reductions for lighting control measures were calculated using the following TRM 5.0 algorithms, adjusted for the format of data recorded in the program: kwh = (N Sensors kw = (N Sensors Where: Fixt Sensor W fixt 1000 ) (1 PAF) AOH IEF E (16) Fixt Sensor W fixt 1000 ) CF IEF D (17) NSensors = Number of installed sensors Fixt/Sensor = Fixtures controlled by each sensor Wfixt = Rated wattage per controlled fixture (Appendix E of TRM 5.0) PAF = Power adjustment factor (TRM 5.0 default, Table 357): 0.70 for typical occupancy sensor controls without daylighting control or dimming 22 Original Equipment Manufacturer (OEM) Specifications and wattage ratings in CEE, DesignLights Consortium, or ENERGY STAR Qualified Products Lists were also used to determine the rated wattages of post-retrofit fixtures. 23 In-Service Rate for Direct Install delivery of CFLs in the non-residential sector from: Pacific Gas & Electric Company, Compact Fluorescent, Downstream & Direct Install. Work Paper PGE3PLTG173, Revision 0. Aug 30, Applied Energy Group, Inc. 17

237 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 CF = Peak demand coincidence factor = Pre-Rinse Spray Valves Energy savings and peak demand reductions per pre-rinse spray valve installation were calculated using the following TRM 5.0 algorithms: kwh = ρ C p U (F B F P ) (T H T supply ) 1 Days (3, 412 Btu/kWh) (18) E t kw = ρ C p U (F B F P ) (T H T supply ) 1 P (3, 412 Btu/kWh) (19) E t Where: ρ Cp U FB FP TH = Water density, 8.33 lbs/gal (TRM default) = Specific heat of water, 1 Btu/lb F (TRM default) = Water usage duration based on application (min/day/unit): TRM 5.0 defaults found in Table 453 = Baseline flow rate, in gal/min: Yr TRM default average is 2.25 GPM = Post-installation flow rate, in gal/min: TRM default average is 1.28 GPM = Average mixed hot water temperature: TRM default of 120 F Tsupply = Average supply water temperature: Et TRM default of 66.1 F for Zone 8/Fort Smith = Thermal efficiency of water heater: TRM default of 0.98 for electric Days/yr = Annual facility operating days for application: TRM 5.0 defaults are found in Table 454 P = Peak factor (hourly peak demand as fraction of daily hot water consumption) : TRM 5.0 defaults found in Table 453 Embedded energy savings and demand reductions were also calculated for pre-rinse spray valves based on the annual gallons of water saved per measure and regional average energy and demand intensities from Table 2-4. Faucet Aerators Energy savings and peak demand reductions per faucet aerator installation were calculated using the following TRM 5.0 algorithms: kwh = ρ C p U (F B F P ) (T H T supply ) 1 Days (3, 412 Btu/kWh) (20) E t kw = ρ C p U (F B F P ) (T H T supply ) 1 P (3, 412 Btu/kWh) (21) E t Where: Yr 24 TRM 5.0 states that this term is a combination of the savings factor and peak coincidence factor; the Power Adjustment Factor was included in the TRM 4.0 algorithm. Applied Energy Group, Inc. 18

238 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 FB FP = Baseline flow rate, in gal/min: TRM default average is 2.2 GPM = Post-installation flow rate, in gal/min Default values for other variables are found in TRM 5.0, Table Embedded energy savings and demand reductions were also calculated for C&I faucet aerators based on the annual gallons of water saved per measure and regional average energy and demand intensities from Table 2-4. Low-Flow Showerheads Energy savings and peak demand reductions per C&I low-flow showerhead installation were calculated using the following TRM 5.0 algorithms: kwh = ρ C p ΔV (T H T supply ) 1 Days (3, 412 Btu/kWh) (22) E t Yr kw = ρ C p ΔV (T H T supply ) 1 P (3, 412 Btu/kWh) (23) E t Where: ΔV = Gallons of water saved per day, per showerhead: These were calculated with equation 249 in TRM 5.0 Default values for other variables are found in TRM 5.0, Table 339 Embedded energy savings and demand reductions were also calculated for C&I low-flow showerheads based on the annual gallons of water saved per measure and regional average energy and demand intensities from Table 2-4. Vending Misers Energy savings and peak demand reductions per vending miser, or vending machine occupancy control, were calculated using TRM 5.0 savings values of 1,612 kwh/unit and 0.03 kw/unit. Motors Energy savings and peak demand reductions for premium efficiency motor replacements were calculated with the following TRM 5.0 algorithms: kwh = HP rated kw/hp LF ( kw = HP rated kw/hp LF ( Where: 1 η base 1 η base HPrated = Nameplate horsepower rating of the motor LF ηbase = Estimated load factor for the motor: TRM 5.0 provides deemed values in Table η installed ) Hrs (24) 1 η installed ) CF (25) = Baseline or existing energy efficiency rating of the motor: TRM 5.0 default values in Table 341 for Replace-on-Burnout measures TRM 5.0 default values in Table 342 for Early Retirement measures 25 While most values are quite similar between pre-rinse spray valves and faucet aerators, building segment types are different. Applied Energy Group, Inc. 19

239 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 (In case of rewound motors, the efficiency may be reduced by a percentage found in Table 343) ηinstalled = Nameplate energy efficiency rating of the newly installed motor CF = Coincident factor (default = 0.74) Hrs = Estimated annual operating hours: TRM 5.0 provides deemed values in Table 344 Chillers Energy savings and peak demand reductions for chiller replacements were calculated with the following TRM 5.0 algorithms: kwh = Capacity EFLH C (η base η installed ) (26) kw = Capacity (η base η installed ) CF (27) Where: Capacity = Rated equipment cooling capacity of the new chiller unit (tons) ηbase ηinstalled = Baseline or existing energy efficiency rating of the chiller (kw/ton): TRM 5.0 default values in Table 255 for Replace-on-Burnout measures TRM 5.0 default values in Tables for Early Retirement measures = Nameplate energy efficiency rating of the new chiller (kw/ton) CF = Coincident factor from Table 456 EFLHc = Equivalent full-load hours for cooling from Table 458 HVAC Energy savings and peak demand reductions for unitary and split system AC equipment replacements were calculated with the following TRM 5.0 algorithms: 1 kw kwh = Capacity ( 1000 W kw = Capacity ( 1000 W Where: 1 kw 1 η base 1 η base 1 η installed ) EFLH C (28) 1 η installed ) CF (29) Capacity = Rated equipment cooling capacity of the new unit, in Btu/hr ηbase ηinstalled = Baseline or existing energy efficiency rating of the equipment: TRM 5.0 default values in Table 249 for Replace-on-Burnout measures TRM 5.0 default values in Tables for Early Retirement measures = Nameplate energy efficiency rating of the installed equipment EFLHc = Equivalent full load cooling hours (TRM 5.0 default in Table 458) CF = Coincidence factor (TRM 5.0 default in Table 456) Strip Curtains Energy savings and peak demand reductions for strip curtain additions on walk-in coolers and freezers were calculated with the following TRM 5.0 algorithms: kwh = ( kwh ) Area (30) sq.ft. Applied Energy Group, Inc. 20

240 kw = ( kwh kwh sq.ft. APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 sq.ft. ) 8760 Area (31) = 365 t T open (η new η old ) 20 C d A i Tr g H (ρ T i h i ρ r h r ) i 3,412 Btu kwh COP adj A Where the defaults for the following variables are given in Tables of TRM 5.0: topen ηnew ηold Cd A = Minutes that the walk-in door is open per day = Efficacy of the new strip curtain = Efficacy of the old strip curtain = Discharge Coefficient empirically determined scaling factor = Doorway Area (in sq. ft.) Ti = Dry-bulb temperature of infiltrating air, in Rankine ( F ) Tr = Dry-bulb temperature of refrigerated air, in Rankine ( F ) ρi, ρr = Density of infiltrating and refrigerated air, respectively, in lb/ft 3 hi, hr COPadj = Enthalpy of infiltrating and refrigerated air, respectively, in Btu/lb (32) = Time and weather-dependent coefficient of performance of the refrigeration system, based on nominal COP of 1.5 for freezers and 2.5 for coolers 3,412 = Conversion factor from Btu to kwh Area = Actual area of doorway (in sq.ft.) Door Gaskets Energy savings and peak demand reductions for door gasket installations on walk-in coolers and freezers were calculated with the following TRM 5.0 algorithms: kwh = ( kwh ) Length (33) ft kw = ( kw ) Length (34) ft Where: kwh/ft = Deemed energy savings value per linear foot of gasket 3 kwh/ft for cooler (per TRM 5.0 Table 385) 23 kwh/ft for freezer kw/ft = Deemed peak demand reduction value per linear foot of gasket kw/ft for cooler (per TRM 5.0 Table 385) kw/ft for freezer Length = Total gasket length as measured, in feet Computer Power Management Energy savings and peak demand reductions for computer power management (CPM) control additions were calculated with the following TRM 5.0 algorithms: kwh = W active/idle (hours active/idle PRE hours active/idle POST )+W sleep (hours sleep PRE hours sleep POST ) 1,000 kw = (W active/idle W sleep ) CF 1,000 (35) (36) Applied Energy Group, Inc. 21

241 Where: APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Wactive/idle = Equipment wattage in active/idle mode (Table 371 of TRM 5.0) Hoursactive/idle = Annual number of hours that equipment is in active/idle mode 6,293 hours/yr before management software installation 1,173 hours/yr after management software installation Wsleep = Equipment wattage in sleep mode (Table 371 of TRM 5.0) Hourssleep = Annual number of hours that equipment is in sleep mode 0 hours/yr before management software installation 5,120 hours/yr after management software installation CF = Peak demand coincidence factor = ,000 = Conversion factor Effective Useful Life of Measures Commercial Lighting For the Commercial Lighting program, LEDs that replaced screw-in lamps (i.e. halogens, incandescents, or CFLs) were assumed to have an integrated ballast and a lifetime of 9 years; LEDs that replaced HID or linear fluorescent fixtures (as well as pin-based and exit light fixtures) were assumed to have remote drivers and a lifetime of 15 years, per TRM 5.0. For measures that replaced lamps non-exempted by the EISA, first-year savings were based on the reported baseline; savings for all following years of the measure life were based on the corresponding Tier 1 EISA 2007 baseline wattage from Table 358 of TRM 5.0. The Commercial Lighting Efficiency section of the TRM does not provide Tier 2 EISA wattages for screw-in lamps as they are for Residential Lighting Efficiency; only Tier 1 EISA equivalent wattages are provided. For consistency with residential lighting, savings in 2023 (9 th and last year of measure life for screw-in LEDs with integrated ballasts) are evaluated with Tier 2 EISA baselines from Table 174 of TRM 5.0. For T12 fixtures replaced with T8 or T5 fixtures, TRM 5.0 specifies an EUL of 8.5 years based on the estimated RUL of 8.5 years for T12 fixtures (in 2012). 26 Thus, the T12 baseline is applicable for the full 8.5 years for these measures. The lifetime of LED measures that replace T12 fixtures, however, is 15 or 9 years (depending on presence of integrated ballast). Thus, the applicable baseline after 8.5 years is a T8 fixture wattage (corresponding FxxLL fixture code in the TRM Standard Wattage Table). Savings for T12 to LED measures are evaluated with this reduced baseline after 8.5 years. AEG also devised a consistent approach to estimate lifetime savings for delamping measures. The first-year savings were based on the stated baseline wattage, and the most likely substitute fixture (what the fixture would have been replaced with if it was not delamped) was determined. 27 The delamping savings from a baseline of the substitute fixture s wattage were then applied for the remaining years of the original fixture s lifetime. 26 Frontier Associates. Petition to Revise Existing Measurement & Verification Guidelines for Lighting Measures for Energy Efficiency Programs: Docket No Public Utility Commission of Texas. Approved June 6, T8 equivalents for existing T12 fixtures were found in the AR TRM s Standard Wattage Table. A delamped 400 W metal halide was assumed to be replaced by a 156 W LED equivalent based on the schotopic/photopic multiplier method from Howard Lighting Products, HID to LED Wattage Cross Reference Table. A delamped 1000 W metal halide was assumed to be replaced by a W (375W average) LED based on typical equivalent replacement information from CREE and RAB Lighting. Applied Energy Group, Inc. 22

242 APSC FILED Time: 5/2/2016 2:38:38 Evaluation PM: Recvd Methods 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Table 2-9 lists the EULs applicable to the Commercial Lighting program. Table 2-8 EULs for Measures in the Commercial Lighting Program Measure EUL LEDs Replace Screw-In 9 LEDs Replace HID or Linear Fluorescent 15 T-8 or T-5 Replace T C&I Direct Install The measures in the C&I Direct Install program have well-defined EULs per TRM 5.0. CFLs have an EUL of 2.5 years. First year CFL savings were evaluated based on the existing equipment baseline, and the savings for the rest of the lifetime were evaluated based on the EISA 2007 standard baseline. Lifetime savings for pre-rinse spray valves, faucet aerators, low-flow showerheads, and vending misers are assumed to persist for the whole measure lifetime. Table 2-9 lists the EULs applicable to C&I Direct Install measures. Table 2-9 EULs for Measures in the C&I Direct Install Program Measure EUL CFL 2.5 Pre-Rinse Spray Valve 5 Faucet Aerator 10 Low-Flow Showerhead 10 Vending Miser 5 C&I Standard Offer For C&I Standard Offer, lifetime savings for motors, chillers, and HVAC equipment were calculated with Early Retirement designations based on existing equipment Remaining Useful Life (RUL) in TRM 5.0. Lifetime savings for strip curtains, gaskets, and computer power management are assumed to persist for the whole measure lifetime. Lifetime savings for fully custom measures were calculated on a case-by-case basis as described in Appendix C. Table 2-10 lists the EULs applicable to the C&I Standard Offer measures. Table 2-10 EULs for Measures in the C&I Standard Offer Program Measure EUL Motors 15 Chillers 20 Various HVAC equipment 15 HVAC controls 10 Strip Curtain 4 Gasket 4 Computer Power Management 4 Applied Energy Group, Inc. 23

243 SECTION 3 Residential Program Findings This section describes the evaluation of the two residential programs in PY2015: Multi-Family Direct Install and the SEE LivingWise Program. Multi-Family Direct Install Evaluation This section describes the process evaluation and impact evaluation conducted for the PY2015 Multi-Family Direct Install program. Process Evaluation In-Depth Interviews In fall 2015 AEG conducted a joint in-depth interview with the residential program manager and the implementation contractor for the Multi-Family program (see Appendix A for the interview guide). We also conducted interviews with 5 property managers who participated in the program in PY2015 (Appendix B shows the interview guide). The following are some key points from the interviews with the program manager and the implementation contractor. Program Goals The kwh goal is the primary goal for the program. At the time of the interview the implementation contractor expected to reach their goal by the end of November. Once the goal was met they planned to stop the program until PY2016. Marketing and Outreach QA/QC Landlords and property managers are the main target market for the program. They sign a contract with OG&E allowing access to the tenants units. CLEAResult has established contacts with the multi-family segment and does most of the program's outreach through cold calling and setting up face to face meetings. CLEAResult has worked with the OG&E marketing department to develop handouts for the program. Once they are aware of the program, the main reason that property managers do not participate is because they have already made the upgrades on their own. CLEAResult maintains the program tracking database. Pre-testing and qualification is also done by CLEAResult. If the project qualifies CLEAResult sets up a meeting with the contractor and the property manager. Property managers are provided with a tenant notification letter that informs the tenants of the process with at least 48 hours notice. CLEAResult staff goes to the site with the contractor to do QA/QC to make sure the measures are installed correctly and the project is completed in a timely manner. Applied Energy Group, Inc. 24

244 APSC FILED Time: 5/2/2016 Residential 2:38:38 PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 Program Satisfaction and Effectiveness According to both the program manager and the implementer, feedback from participants is very positive. Customers are highly satisfied. CLEAResult gets referrals from participating property managers throughout Arkansas. The tenants understand why they are installing the measures and are very receptive to the upgrades. Program Improvements In the future, attic insulation may be an additional measure that could add value to the program. Property Manager Interviews Approach AEG conducted in-depth interviews with five participating property managers in December The interviews discussed reasons for participating, satisfaction, and program effectiveness. The interviews ranged from 15 to 25 minutes in length. Reasons for Participating For three of the five property managers this was their first experience with an OG&E program. One of the other property managers had participated in a residential program and another had experience with OG&E commercially. We worked previously on an energy conservation house that was used for education purposes. OG&E was involved with that. We have also been involved in commercial programs for vending machines and things like that. Most of the property managers heard about the program when they were contacted by CLEAResult. One property manager heard about the OG&E programs in Oklahoma and asked if there was anything similar available in Arkansas. The primary motivation for participating in the Multi-Family program was to achieve energy savings for their tenants. A couple of property managers were also interested in water savings. Our water costs have increased. So we were hoping to see some water savings and get savings for tenants at the same time. Almost all the property managers said their savings expectations have been realized; one property manager said it was too soon to tell. Satisfaction We ve had an increase in sewer costs that have masked the water savings. But looking at the water used, we can tell that the savings have been substantial. Overall satisfaction with the program is very high. All of the property managers involved in the program said it was a positive experience. It s a winning season. We didn t bat 1,000 but in just one month we could tell the savings were worth the effort. Satisfaction with the contractors performing the service was also high for most participants, but one property manager had difficulty with a contractor and had to complain to CLEAResult to get the issue resolved. The contractor did not set up power strips correctly. But it was a lot of work in a very short period of time. In all they worked very well with us notifying tenants and coordinating everything. Applied Energy Group, Inc. 25

245 APSC FILED Time: 5/2/2016 Residential 2:38:38 Program PM: Recvd Findings 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Property managers reported that the tenants were very happy with the measures installed. All the property managers said they would definitely participate in the program again. One property manager volunteered to be a case study for OG&E and CLEAResult. He has kept hard data that shows how he has benefited from the program. Program Effectiveness Three of the five property managers had not previously installed any of the measures. The other two had limited experience with some, but not all of the measures installed mainly high efficiency light bulbs. Four of the five property managers said they would not have installed the equipment without OG&E s program. One property manager said he would probably have installed CFL light bulbs and had started thinking about aerators and showerheads because of the increase in the water bill. But he would have installed fewer measures without the program and would have used a phased-in approach instead of retrofitting all the units at one time. Four of the five property managers said all measures remained installed; one property manager said tenants took the measures with them when they moved out. These measures were replaced with standard efficiency equipment. Five showerheads and several lightbulbs have been taken by tenants who moved away. Tenants like the stuff so much they steal the CFLs and showerheads when they leave! All the property managers said the program increased their interest in energy efficiency. Even property managers who felt they were fairly knowledgeable about energy efficiency learned additional ways to improve. The entire experience has changed the way I work. I m using the rebate check to purchase additional energy efficient equipment. Impact Evaluation Documentation for PY2015 from OG&E shows that the implementer completed 6,893 projects with 16,759 measures. Using TRM 5.0 algorithms and data provided by OG&E and the implementer in the form of a tracking database, AEG estimated total program impacts for each of the measures. Table 3-1 summarizes the first year evaluated gross impacts for PY2015 participants. Table 3-1 Multi-Family Direct Install Program: Summary of First Year Gross Impacts Measure Entry Count Measure Count Evaluated First Year kwh Savings kwh Realization Rate Evaluated First Year kw Reduction kw Realization Rate CFLs 1,712 9, , % % Aerators 2,196 3, , % % Showerhead 1,126 1, , % % Air Infiltration , % % Duct Sealing ,860, % % Advanced Power Strip 892 1, , % % Total 6,893 16,759 4,408, % % Applied Energy Group, Inc. 26

246 APSC FILED Time: 5/2/2016 Residential 2:38:38 PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 Compact Fluorescent Lighting AEG calculated the energy savings and peak demand reductions for the 9,808 CFLs installed in PY2015 using the TRM 5.0 algorithms; inputs from the program tracking database included the apartment space heating type (electric resistance or heat pump) as well as baseline and installed wattages. A residential direct install in-service rate (ISR) of 98% was used to be consistent with the AR TRM 5.0. The CFL impacts calculated by AEG accounted for 100% of the total reported energy savings and demand reductions. The final evaluated first year impacts for CFLs were found to be 194,810 kwh/yr with a peak demand reduction of 37.2 kw. These first year impacts were based off the reported 43 watt existing baseline, compliant with the Tier 1 requirements of the Energy Independence and Security Act (EISA) of After 2022, the Tier 2 EISA code baseline to 20 W should be used. 29 Thus, the measure will save 45,456 kwh/yr and reduce peak demand by 8.7 kw for the last year of the CFL s useful life (i.e., PY2023). The first year and total lifetime gross energy savings as well as the first year and weighted lifetime demand reductions are depicted in Table 3-2, where the lifetime demand reduction is a weighted average annual kw demand reduction value calculated by dividing the sum of annual kw reductions by the EUL of the measure. Table 3-2 Multi-Family CFLs: Summary of Gross Impacts Type of Impact Energy Savings (kwh) Annual Peak Demand Reduction (kw) Reported Annual Impacts 194, First Year Evaluated Impacts 194, Evaluated Impacts for EISA Tier 2 Baseline 45, Evaluated Lifetime Impacts 1,603, Faucet Aerators AEG calculated the energy savings and peak demand reduction for the 3,149 installed aerators using the TRM 5.0 algorithm and with the assumption that all water heaters in the program were electric resistance water heaters. The implementer confirmed this assumption to be accurate, since program staff verified the presence of electric water heaters at the site. Calculated energy savings for all installed aerators were 100% consistent with reported values after the addition of embedded energy savings. Unlike energy savings, calculated demand reductions for faucet aerators were not as consistent with the reported values. AEG found that the demand reductions (kwh savings multiplied by a factor of kw/kwh) for all installations were calculated using energy savings that already included the embedded energy savings. This double-counting was removed, resulting in an overall demand reduction realization rate of 96%. An impact summary for the faucet aerator measure (which has a 10-year EUL per TRM 5.0) can be seen in Table EISA code requirements are stipulated in the Arkansas TRM 5.0, Vol. 2, Table This approach is consistent with measure life and baseline considerations for CFL lamps with a rated measure life of 10,000 hours stipulated in the Arkansas TRM 5.0 Vol. 2, Table 173. Applied Energy Group, Inc. 27

247 APSC FILED Time: 5/2/2016 Residential 2:38:38 Program PM: Recvd Findings 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Table 3-3 Multi-Family Faucet Aerators: Summary of Gross Impacts Type of Impact Energy Savings (kwh) Annual Peak Demand Reduction (kw) Reported Annual Impacts 153, First Year Evaluated Impacts 153, Evaluated Lifetime Impacts 1,532, Low-Flow Showerheads The impacts for 1,462 installed showerheads were calculated with TRM 5.0 algorithms and the assumption that all water heaters in the program were electric resistance water heaters. Energy savings for the installations were 100% consistent with reported values after adding embedded energy savings. Demand inconsistencies were still prevalent due to issues with double -counting the embedded energy intensity as described for aerators. In the PY2014 measure calculator file, AEG found another inconsistency with the demand reduction calculations; this error was fixed in the PY2015 calculator file but the embedded energy intensity was still counted twice for embedded demand reduction calculations. The overall demand reduction realization rate for low - flow showerheads was determined to be 96%. An impact summary for the low-flow showerhead measure (which has a 10-year EUL per TRM 5.0) can be seen in Table 3-4. Table 3-4 Multi-Family Low-Flow Showerheads: Summary of Gross Impacts Type of Impact Energy Savings (kwh) Annual Peak Demand Reduction (kw) Reported Annual Impacts 467, First Year Evaluated Impacts 467, Evaluated Lifetime Impacts 4,673, Air Infiltration Reduction AEG calculated the energy savings and peak demand reductions for the 369 infiltration reduction projects in PY2015 using the TRM 5.0 algorithms; inputs from the program tracking database included pre- and post-installation CFM50 measurements as well as the heating type (electric resistance or heat pump) for the apartment. The energy savings and demand reductions calculated by AEG were 100% consistent with the reported values. An impact summary for the air infiltration measure (which has an 11-year EUL per TRM 5.0) can be seen in Table 3-5. Table 3-5 Multi-Family Air Infiltration Reduction: Summary of Gross Impacts Type of Impact Energy Savings (kwh) Annual Peak Demand Reduction (kw) Reported Annual Impacts 386, First Year Evaluated Impacts 386, Evaluated Lifetime Impacts 4,246, Duct Sealing The energy savings and peak demand reductions were calculated for the 598 duct sealing projects using TRM 5.0 algorithms; inputs from the program tracking database included pre - and Applied Energy Group, Inc. 28

248 APSC FILED Time: 5/2/2016 Residential 2:38:38 PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 post-installation CFM25 measurements as well as the heating type (electric resistance or heat pump) for the apartment. A cap on the pre-installation leakage rate was applied as a maximum 40% of total fan flow, per TRM One of the reasons for this restriction is that the TRM algorithm may only be applicable within a certain range of duct leakage reductions. Outside of the range, the algorithm could fail and incorrectly estimate the savings. Furthermore, the Arkansas TRM cites that data from installations in Texas shows that more than 70 percent of all pre-retrofit leakage rates fall below 38 percent. During the savings replication effort for duct sealing measures, AEG found numerous reported value errors in the tracking database. These were resolved based on the Direct Install calculator submitted by the implementer for complex COM In this complex, 60 entries were found to have 1.5 ton AC systems, though the capacity values in the tracking database were reported as 2 tons; this issue had resulted in a replication error for 55 entries with pre -installation leakage rate limits in this complex, since the total fan flow is based on the cooling system capacity. Given that the reported capacity was found to be the main source of error in savings replication, AEG adjusted the cooling system capacity for 229 total duct sealing entries to 1.5 tons; three more entries were adjusted to 2.5 tons to match reported savings values. Furthermore, 20 entries with heat pump systems were reporting the heating seasonal performance factor (HSPF) as 7 Btu/Wh in the tracking database; the default value in TRM 5.0 is 7.3 Btu/Wh. Adjusting the HSPF value to 7.3 Btu/Wh resulted in correct savings replication for these projects. Once capacity and heat pump efficiency errors were fixed, the final evaluated impacts for duct sealing were found to be 2,860,446 kwh/yr with a peak demand reduction of kw. This corresponds to energy savings and demand reduction realization rates of 100%. An impact summary for the duct sealing measure (which has an 18-year EUL per TRM 5.0) can be seen in Table 3-6. Table 3-6 Multi-Family Duct Sealing: Summary of Gross Impacts Type of Impact Energy Savings (kwh) Annual Peak Demand Reduction (kw) Reported Annual Impacts 2,860, Replicated Impacts with 40% Cap 3,353, First Year Evaluated Impacts 2,860, Evaluated Lifetime Impacts 51,488, Advanced Power Strips There were 1,373 advanced power strip (APS) units installed in the PY2015 program. In PY2015 OG&E confirmed that program staff installed power strips for interested customers on home entertainment systems. Per the TRM 5.0, this resulted in energy savings of kwh and a demand reduction of 0.03 kw per installed unit. AEG confirmed with the impleme nter that all APS units were installed on home entertainment systems, even if a particular project had more than one APS unit installed. Evaluated impacts were consistent with reported values. An impact summary for advanced power strip installation (which has a 10-year EUL per TRM 5.0) can be seen in Table The total fan flow is calculated as 400 CFM/ton multiplied by the cooling system capacity in tons in the AR TRM. Applied Energy Group, Inc. 29

249 APSC FILED Time: 5/2/2016 Residential 2:38:38 Program PM: Recvd Findings 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Table 3-7 Multi-Family Power Strips: Summary of Gross Impacts Type of Impact Energy Savings (kwh) Annual Peak Demand Reduction (kw) Reported Annual Impacts 346, First Year Evaluated Impacts 346, Evaluated Lifetime Impacts 3,462, Net Impacts There was no program in Arkansas comparable to OG&E s Multi-Family Direct Installation program. As a result, we used comparable programs in Illinois and Maine (see Section 2) to determine the NTG ratio for the direct install measures used in OG&E s program, as follows: CFLs: NTG ratio = 0.81 Aerators: NTG ratio = 0.94 Showerhead: NTG ratio = 0.93 Since the Illinois program did not include air infiltration, duct sealing, and power strips, the average program NTG of 0.9 was used for these measures. This value was the same for the Efficiency Maine Multi-Family Program. Table 3-8 includes the first year impact results for PY2015 participants of the Multi-Family Direct Install program. AEG applied the NTG ratios to the gross impacts resulting in first year net impacts of 3,969,881 kwh per year and kw. Section 5 and Appendix D present the net lifetime impacts for this and other programs by measure. Applied Energy Group, Inc. 30

250 APSC FILED Time: 5/2/2016 Residential 2:38:38 PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 Table 3-8 Multi-Family Direct Install Program: PY2015 First Year Impacts Impact Measure Reported Gross Impact Evaluated Realization Rate NTG Net Impact CFLs 194, , % ,796 Aerators 153, , % ,033 Energy Savings (kwh/yr) Showerhead 467, , % ,594 Air Infiltration 386, , % ,412 Duct Sealing 2,860,446 2,860, % ,574,401 Advanced Power Strip 346, , % ,644 Total 4,407,971 4,408, % ,969,881 CFLs % Aerators % Peak Demand Reduction (kw) Showerhead % Air Infiltration % Duct Sealing % Advanced Power Strip % Total % SEE LivingWise Program Process and Impact Evaluation Program Process Review In reviewing the program implementation process, we conducted interviews and follow-up discussions with the OG&E program team and the implementation contractor (RAP) staff. The design and delivery of the PY2015 program was not changed in any significant way from PY2014. It is clear, however, that RAP continues to review its practices in an ongoing effort to further improve the effectiveness of the program. In particular, RAP reported taking the following steps to help increase savings: Review the kit components with OG&E and modify as appropriate (none in PY2015). Use results from ongoing focus groups that RAP conducts nationwide to come up with new educational contractor each year, including 5-day teaching plans for new participant teachers. Do more targeted recruitment and contact with low-response teachers with more followups with teachers to help them stay on schedule. Recruitment targets the teachers most likely to show high student participation rates: Tier 1 = teachers who participated previously, returned surveys and showed high installation; Tier 2 = lower of returned survey and installation and good candidate new teachers; Tier 3 (budget allowing) = new teachers. Applied Energy Group, Inc. 31

251 APSC FILED Time: 5/2/2016 Residential 2:38:38 Program PM: Recvd Findings 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 In the prior program year (PY2014), AEG examined a program kit. RAP confirmed that the PY2015 kits were identical to those used in PY2014. All measures in the kits were verified to have efficiency levels at or above the minimum required by TRM 5.0 which was used to assess the program s savings. OG&E s participation goal for SEE LivingWise PY2015 was 1,840 participants. OG&E did, however, allocate funding to allow RAP to distribute 1,919 kits in PY2015. RAP indicated that the population of 6 th graders in OG&E s Arkansas territory is typically 2,100 to 2,300 students. As Table 3-9 shows, RAP delivered a total of 1,919 kits; of these, 1,885 were for distribution to students in 23 Arkansas schools, the rest were for the participating teachers. (Only kits for students are used in calculating the program savings.) In PY2015, 1,036 of the 1,885 students who were provided kits returned completed surveys, a response rate of 55%, just about the same response rate as in PY2014 and PY2013. Also similar to previous years, there was very high response rate from the majority of classes and zero responses from some classes. Table 3-9 Distribution of SEE LivingWise Kits in PY2015 Number of Kits Distributed Spring 2015 Fall 2015 Total RAP Delivered to Classrooms 340 1,579 1,919 For Teachers For Students 333 1,552 1,885 Compliance with TRM 5.0 AEG replicated the example results in the most recently published TRM, version 5.0, for each of the measures, ensuring we could properly apply them. We then used the algorithms to estimate the impacts in OG&E s Arkansas service territory, using default input values for Fort Smith and program-specific data from participants. The TRM 5.0 algorithms and resulting estimates in Table 3-10 represent energy and demand impacts per measure installed. That is, they do not adjust for the installation rate of each measure, the relative share of participants homes with electric versus natural gas water heating, nor any embedded energy savings (discussed below). Table 3-10 TRM-Calculated Impacts by Measure, Per Unit Installed Measure Annual kwh Savings Annual kw Reduction Annual Therms Kitchen Faucet Aerator Bathroom Faucet Aerator Low-Flow Showerhead W CFL # n/a 13 W CFL # n/a 13 W CFL # n/a The electrical impact values above adjust for the presence of heat pumps in water heating and HVAC end uses, as specified in TRM 4.0 and 5.0. The student survey, which provided most of the fuel/technology saturations for our calculations, does not identify heat pumps, so the heat pump shares needed to be estimated separately. For aerator and showerhead measures, a 2.9% Applied Energy Group, Inc. 32

252 APSC FILED Time: 5/2/2016 Residential 2:38:38 PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 overall heat pump water heater (HPWH) saturation (across all fuel types) in single family homes 31 was used to estimate the saturation of HPWH within the electric water heater population. To do so, 2.9% of the 983 PY2015 survey responses regarding water heater fuel were assumed to be HPWH; the rounded value (29 water heaters) was then taken out of the number of total electric water heater responses (538) resulting in a 2.9% HPWH saturation across all water heaters and a 5% saturation across electric water heaters. The electrical impacts for aerators and showerheads in Table 3-10 are a weighted average of TRM-derived results for electric resistance (95%) and heat pump (5%) water heaters. For the CFL measures, data from the 2009 Residential Energy Consumption Survey (RECS) 32 was analyzed to determine the relative saturation of heat pump HVAC systems among homes with electric resistance heating and AC. 33 This 11.5% saturation was then used to break out heat pump HVAC systems from the total LivingWise survey respondents that identified their HVAC system as electric resistance heating with AC (central or otherwise). The resultant relative weights for each type of HVAC system in TRM 5.0 were then used to determine weighted interactive effects factors (IEF) for electrical energy and demand impacts for systems with electric and direct-fuel space heating, as seen in Table The impacts for CFL installations in Table 3-10 also take into account the relative fuel share of electric versus direct-fuel (i.e., nonelectric) space heating. Table 3-11 Derivation of Weighted Interaction Factors for CFL Installation Impacts Fuel Type HVAC System Type from TRM 5.0 IEFEnergy TRM 5.0 IEFDemand TRM 5.0 Survey Weights IEFEnergy IEFDemand Gas, Oil, Wood, Propane, Other Electric Gas Heat with AC % Gas Heat with No AC % Electric Resistance with AC % Electric Resistance with no AC % Heat Pump % We show how the algorithms were applied in the results for each measure in the following subsection. To estimate the overall program impacts and those realized per participant, following guidelines from the IEM about using as much reliable program-specific data as possible to inform the impact estimates. We used the following data from the participant surveys to estimate the per-participant and total program impacts by measure reported below: Wattage of the existing lamps replaced by each of the CFLs in the kit (included in the per-unit impacts in Table 3-10). Share of heating and cooling system types, with electric resistance/ AC system types broken out to include heat pump systems (for CFL interactive effects, included in the per - unit impacts). 31 Arkansas Energy Efficiency Potential Study. Navigant Consulting, Inc. Appendix B: Residential Survey Results. June 1, Residential Energy Consumption Survey. U.S. Energy Information Administration. RECS Public Use Microdata File (Data for AR, LA, OK States) Accessed at: 33 HVAC equipment saturations in tables B.1 and B.2 of the 2015 Arkansas Energy Efficiency Potential Study were provided for three segments (single family, multi-family and manufactured homes) and provided saturation values for both heating and cooling systems. While the values did not contradict the RECS 2009 data, they did not provide analogous equipment breakdowns to allow for using the potential study values in the derivation of IEFs. Applied Energy Group, Inc. 33

253 APSC FILED Time: 5/2/2016 Residential 2:38:38 Program PM: Recvd Findings 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Share of electric versus natural gas space heating (for CFL interactive effects, included in the per-unit impacts). Share of electric versus natural gas water heating (for aerators and showerhead, not included in the per-unit impacts). Installation rate of each measure (for aerators, showerhead, and CFLs, not included in the per-unit impacts). Program Year 2015 Evaluated Gross Impacts Using data provided by OG&E and RAP, the TRM-based per-unit measure energy savings estimates, and information from the participant surveys, we estimated total first year and lifetime program impacts for each measure in the PY2015 program. By first year we mean impacts achieved by the PY2015 participants in the first year of their installations; elsewhere we simply refer to these as 2015 impacts. The lifetime impacts reflect gross savings expected to accumulate over the EUL of the measures installed. (See Section 2 for a summary of EULs by measure.) Combining the impacts from all measures for all participants, the average first year impacts per participant are kwh, kw, and almost 3 annual therms. Table 3-12 depicts the evaluated program gross energy savings (first year and lifetime) and gross demand reductions (first year and weighted average over lifetime) by measure, total, and per participant. Table 3-12 SEE LivingWise Evaluated Gross Program Impacts Summary Measure First Year kwh Savings Lifetime kwh Savings First Year kw Reduction Weighted Average Annual kw Reduction First Year Therms Savings Lifetime Therms Savings Aerators 42, , ,082 10,818 Showerhead 152,325 1,523, ,843 38,429 CFLs 120, , Total 315,603 2,635, ,925 49,246 Per Participant , The measures vary considerably in their contribution to total savings low-flow showerheads alone account for almost half of the annual energy and demand impacts. While results were similar in PY2014, showerheads and CFL contributed almost equally to the total kwh savings in PY2013. Several factors account for this, including changes in TRM allowances in the different versions and in installation rates across the years; but more than anything the driving factor was the higher efficiency of the showerhead included in the PY2015 and PY2014 kits. In addition to the at-site (end-user) impacts, the values in the table above include water supply and wastewater treatment impacts accrued to local water agencies due to reduced water usage from faucet aerators and low-flow showerheads, just as they did in our PY2014 evaluation. Each gallon of water saved reduces the drinking water or wastewater utility s energy and demand requirements for functions such as collecting, treating, storing, and transporting 34 The total weighted average annual kw reduction is the sum of the kw reductions per measure per year divided by the maximum measure lifetime in the program (10 years). Thus, it is indicative of the average annual kw reduction expected over the course of 10 years. Applied Energy Group, Inc. 34

254 APSC FILED Time: 5/2/2016 Residential 2:38:38 PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 water/wastewater. We refer to these as embedded impacts and apply them based on a report prepared for OG&E by AEG in The study involved interviewing local water and wastewater agencies to determine typical energy and demand intensity values for representative cities in Oklahoma and Arkansas and a literature review of energy intensity values as a function of water and wastewater plant characteristics to validate interview findings. Based on the research, the project team determined energy savings and average demand reductions per unit of avoided water as a function of location across the electric utility s service territory. The research revealed that water treatment in the part of Arkansas that OG&E serves has lower energy intensity than other parts of the service territory. We used the lower Fort Smith, Arkansas per-gallon kwh and kw intensities rather than the regional averages, resulting in a more conservative estimate of the embedded impacts. Using the same TRM water use reduction savings as for the end-user impacts, we calculated the following embedded energy and demand impacts (see Table 3-13), which are reflected in the evaluated impacts for faucet aerators and low-flow showerheads. Table 3-13 Calculation of Embedded Energy and Demand Impacts Embedded Energy and Demand Impacts Annual water savings per unit installed Kitchen faucet aerator = gal/yr Bathroom faucet aerator = gal/yr Showerhead = 3,245.6 gal/yr Water & wastewater energy and demand impacts 2.4 Watt-hr/gal Watt/gal Total embedded energy and demand impacts Units installed x gallons saved x savings/gal Faucet aerators = 2,054 kwh, 0.23 kw Showerheads = 6,969 kwh, 0.78 kw While OG&E does not provide natural gas to customers in Arkansas and has no goals for natural gas savings, 33% of the participants said they have gas water heat; they realized significant natural gas savings from the installation of aerator and showerhead measures. In Btu equivalents, the annual natural gas kbtu savings are 75% of total annual electric kbtu savings for showerheads and aerators. This comparison is shown in Table Table 3-14 Natural Gas and Electric 2015 Btu Savings Comparison (Aerators & Showerheads) Measure Gas kbtu/yr Electric kbtu/yr Aerator 108, ,619 (42,972 kwh) Showerhead 384, ,733 (152,325 kwh) Total 497, ,352 (195,297 kwh) 35 Parmenter, K., Ehrhard, R., Cook, G. and Williamson, C. Embedded Energy Savings from Water Saving Measures: Electricity Savings Due to Avoided Water Supply and Wastewater Treatment, Jan Applied Energy Group, Inc. 35

255 Faucet Aerators APSC FILED Time: 5/2/2016 Residential 2:38:38 Program PM: Recvd Findings 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 The per-unit installed impacts for faucet aerators were calculated using the TRM 5.0 algorithms described in Equations 1 and 2 of the residential impact evaluation approach of this report. Table 3-15 Faucet Aerator Evaluated Gross Impacts Type of Aerator Evaluated Gross Impact Per Unit Installed Per Participant Participant Total First Year kwh ,823 Lifetime kwh ,234 Kitchen Aerator Annual Peak kw Reduction Annual Therms Lifetime Therms ,235 First Year kwh ,148 Bathroom Aerator Lifetime kwh ,482 Annual Peak kw Reduction Annual Therms Lifetime Therms ,583 First Year kwh ,972 Lifetime kwh ,716 Aerators Total Annual Peak kw Reduction Annual Therms ,082 Lifetime Therms ,818 We calculated the per-participant and total measure impacts using the TRM 5.0 algorithm and the following information: Each kit contained one kitchen faucet aerator and one bathroom faucet aerator. Kitchen faucet aerators were rated at 1.5 GPM and were calculated to save gallons of water per year per aerator. Bathroom faucet aerators were rated at 1.0 GPM and saved gal/year per aerator. The per-participant and total measure impacts utilize the per-unit installed impacts shown in Table 3-10 plus program-specific information from the PY2015 participants. This includes: Kitchen aerator installation or in-service rate (ISR) = 47%. Bathroom aerator installation or in-service rate (ISR) = 44%. Applied Energy Group, Inc. 36

256 APSC FILED Time: 5/2/2016 Residential 2:38:38 PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 Water heat fuel shares = 55% electric and 33% natural gas. 36 o The saturation of heat pump water heaters within the electric water heater population was derived to be 5%. 37 All measure impacts also include drinking water and wastewater treatment impacts accrued from the reduced water use, as identified in the report on embedded energy intensity. 38 Energy intensity of treatment = 2,397 kwh and 0.27 kw per million gallon water. Embedded impacts total from aerators = 2,054 kwh/yr and 0.23 peak kw. Treatment is electric and applies to all water used by installed units, regardless of water heating fuel. Low-Flow Showerheads The per-unit installed impacts for low-flow showerheads were calculated using TRM 5.0 algorithms described in Equations 3 and 4 of the residential impact evaluation approach section of this report. Table 3-16 Low-Flow Showerhead Evaluated Gross Impacts Evaluated Gross Impact Per Unit Installed Per Participant Participant Total First Year kwh ,325 Lifetime kwh 3, ,523,251 Annual Peak kw Reduction Annual Therms ,843 Lifetime Therms ,429 We calculated the per-participant and total measure impacts using the TRM 5.0 algorithm and the following information: Each kit provided one low-flow showerhead rated at 1.5 GPM. Each low-flow showerhead was calculated to save 3,245.6 gallons of water per year. The per-participant and total measure impacts utilize the per-unit installed impacts shown in Table 3-10 plus program-specific information from the PY2015 participants. This includes: Low-flow showerhead installation or in-service rate (ISR) = 48%. Water heat fuel shares = 55% electric and 33% natural gas. 39 The saturation of heat pump water heaters within the electric water heater population was derived to be 5% The remaining 12% of water heaters were found to be fueled with propane and other fuel. Water heating savings for propane tanks are not included in the TRM and were not included in the PY2015 analysis. 37 Based on 2.9% overall heat pump water heater saturation (within all fuel types) in Arkansas single family homes from the 2015 Energy Efficiency Potential Study, as previously referenced. 38 Ibid. 39 The remaining 12% of water heaters were found to be fueled with propane and other fuel. Water heating savings for propane tanks are not included in the TRM and were not included in the PY2015 analysis. 40 Based on 2.9% overall heat pump water heater saturation (within all fuel types) in Arkansas single family homes from the 2015 Energy Efficiency Potential Study, as previously referenced. Applied Energy Group, Inc. 37

257 APSC FILED Time: 5/2/2016 Residential 2:38:38 Program PM: Recvd Findings 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 All measure impacts also include drinking water and wastewater treatment impacts accrued from the reduced water use, as identified in the report on embedded energy intensity. 41 Energy intensity of treatment = 2,397 kwh and 0.27 kw per million gallon water. Embedded impacts total from showerheads = 6,969 kwh/yr and 0.78 peak kw. Treatment is electric and applies to all water used by installed units, regardless of water heating fuel. Compact Fluorescent Lamps The per-unit installed impacts for CFLs were calculated using TRM 5.0 algorithms described in Equations 5 and 6 of the residential impact evaluation approach section of this report. Table 3-17 CFL Evaluated Gross Impacts CFL Evaluated Gross Impact Per Unit Installed Per Participant Participant Total First Year kwh ,619 Lifetime kwh ,075 CFL #1 First Year Peak kw Reduction Weighted Average Peak kw Reduction First Year kwh ,028 Lifetime kwh ,583 CFL #2 First Year Peak kw Reduction Weighted Average Peak kw Reduction First Year kwh ,659 Lifetime kwh ,832 CFL #3 First Year Peak kw Reduction Weighted Average Peak kw Reduction First Year kwh ,306 Lifetime kwh ,490 CFL Total First Year Peak kw Reduction Weighted Average Peak kw Reduction Ibid. Applied Energy Group, Inc. 38

258 APSC FILED Time: 5/2/2016 Residential 2:38:38 PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 Each kit provided three 13 watt CFLs. The per-participant and total measure impacts utilize the per-unit installed impacts shown in Table 3-10 plus program-specific information from the PY2015 participants. This includes the participant-reported CFL installation or in-service rate (ISR): ISR for CFL #1 = 70%. ISR for CFL #2 = 57%. ISR for CFL #3 = 50%. Fully complies with TRM 5.0, including: Calculation of different impacts in homes with electric heat and gas heat (weighted average is reported in table above). o o o Share of heating and cooling system type pairs, with electric resistance/central AC system types broken out to include heat pump systems in order to estimate weighted interactive effects factors for electric and gas (direct-fuel) heat types: Electric Heat Interactive Factors: IEFE = 0.84; IEFD = Gas Heat Interactive Factors: IEFE = 1.09; IEFD = Share of homes with electric and gas heat reported by participants: 65% electric space heat and 35% gas and other space heat types. Assessment of the baseline, based on participant survey response about wattage of lamp replaced; the first year baseline is the existing replaced lamp wattage, the baseline until PY2022 is compliant with Tier 1 of the Energy Independence and Security Act (EISA) of 2007, and the baseline starting in PY2023 is compliant with Tier 2 of EISA. CFL #1 Baselines: Existing = 61.9W; EISA Tier 1 = 44.4W; Tier 2 = 21.9W. CFL #1 Baselines: Existing = 62.3W; EISA Tier 1 = 44.7W; Tier 2 = 22.1W. CFL #1 Baselines: Existing = 61.8W; EISA Tier 1 = 44.3W; Tier 2 = 21.9W. o TRM annual operating hours (AOH) = Non-electric heating penalties were not evaluated for CFLs due to the various non-electric fuels present in the population and the fact that OG&E does not claim any non-electric fuel savings. Net Impacts As in past years, we did not conduct an independent assessment of net-to-gross (NTG) ratios to calculate net impacts for SEE. In PY2014, with full approval of the IEM, we used the NTG estimates from another, completely comparable, student education program in Indiana. The programs in Indiana supplied student take-home kits and teaching curriculum exactly the same as the OG&E program and were implemented by the same contractor. These results were used for this PY2015 evaluation as well. The Indiana report 42 provides NTGs by measure, based on collection of primary data and considerable analysis. The values incorporate estimates of both free ridership and spillover, as shown in Table The free ridership values for showerheads and aerators confirm widespread agreement that few residential customers install them in the absence of a program that provides Energizing Indiana: EM&V Report, prepared by The Indiana Statewide Core Program Evaluation Team, June 20, Applied Energy Group, Inc. 39

259 APSC FILED Time: 5/2/2016 Residential 2:38:38 Program PM: Recvd Findings 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 them. The 22% spillover rate supports the hypothesis that providing education and no-cost kits to students encourages households to take additional energy efficiency actions on their own. Table 3-18 Energizing Indiana Education Program Free Ridership, Spillover, and NTG Summary Measure Free Ridership Percent Spillover Estimate NTG Ratio Low-Flow Showerhead 13.8% 21.5% 107.7% Faucet Aerators 7.9% 21.5% 113.6% CFLs (13W) 27.6% 21.5% 93.9% Since the NTG values differ by measure type, we applied them to the first year SEE LivingWise measure impact totals and calculated the net impacts for the program as the sum of the net measure impacts. Table 3-19 shows the effects of applying these NTG ratios to the evaluated gross impact estimates for each measure and the total program. The overall program NTG ratio is Table 3-19 Measure SEE LivingWise Evaluated Net Program Impacts by Measure NTG Ratio First Year kwh Savings Lifetime kwh Savings First Year kw Reduction Weighted Average Annual kw Reduction Faucet Aerators , , Low-Flow Showerhead ,993 1,639, W CFLs , , Total ,745 2,768, The final evaluated net impacts from the PY2015 SEE LivingWise program are shown in Table As in the past, participation was capped by the allocated budget, though the PY2015 budget reportedly did allow RAP to reach more than the 80% of the 6 th grade students served by OG&E in Arkansas. Since OG&E set goals and reports impacts based on net savings (using the previous year s NTG ratio) the realization rate is calculated as net impact as a percent of reported impact. Table 3-20 SEE LivingWise Program Goals, and Reported and Evaluated Impacts Metric Goal Reported Impact 44 Evaluated Net Impact Realization Rate Participants 1,840 1,885 1, % First Year Energy Savings (kwh/yr) 288, , , % First Year Demand Reduction (kw) % 43 The total weighted average annual kw reduction is the sum of the kw reductions per measure per year divided by the maximum measure lifetime in the program (10 years). Thus, it is indicative of the average annual kw reduction expected over the course of 10 years. 44 These values were taken from the AR LivingWise 2015 YE Summary for Review_ xlsx file provided by OG&E and further confirmed in final "2015 Arkansas Program Tracking Information xlsx" file provided by RAP. Applied Energy Group, Inc. 40

260 APSC FILED Time: 5/2/2016 Residential 2:38:38 PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 Evaluated impacts surpassed goals. The program achieved and slightly surpassed the goals OG&E set for participation, annual kwh savings and kw reductions. Added funding enabled the 2% increase in participation. This resulted, however, in evaluated kwh impacts of 13% over the goal. This is very likely due to changes in the TRM calculations over time that resulted in higher per-unit savings than anticipated when the goals were set. Evaluated impacts met and exceeded reported impacts. The evaluated net first year (i.e., 2015) kwh savings and kw reductions notably exceeded OG&E s tracked (reported) impacts, as shown in the realization rates in Table 3-20 above. Possible explanations for the evaluated impacts exceeding the reported impacts are: The reported impacts are based on estimated per-participant impacts of kwh and kw which are notably lower than even the gross (pre-ntg) evaluated impacts that AEG calculated. AEG s calculations for evaluated impacts used the measure installation rates (aka in-service rates or ISRs) reported in the student surveys for the whole year. The PY2015 ISRs were at least as high as PY2014 for every measure in the kit, as shown in Table Table 3-21 SEE LivingWise In-Service Rate Comparison between PY2015 and PY2014 Measure PY2015 PY2014 Aerator - Kitchen 47% 44% Aerator - Bathroom 44% 39% Showerhead 48% 45% CFL #1 70% 66% CFL #2 57% 55% CFL #3 50% 50% The reported impacts may have used a lower NTG than the one used for the PY2015 evaluated impacts. Applied Energy Group, Inc. 41

261 APSC FILED Time: 5/2/2016 Residential 2:38:38 Program PM: Recvd Findings 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Residential Programs Adherence to Protocol A The tracking systems for each of the databases conform reasonably well to the tracking system protocol developed for use in Arkansas. Table 3-22 shows a summary of how well the residential program tracking systems meets the components of the protocol. Table 3-22 Multi-Family Program Adherence to Protocol A: Residential Programs Participating Customer Information includes all information required including customer contact information, customer identifier (account number), location of building, and date completed Measure Specific Information includes type and quantity of measures installed. Could capture type of lighting replaced with CFL and location in the home. Measure Codes n/a; description fields could be used for a measure description such as aerator, CFL, etc. Vendor Specific Information not included. Could capture which contractor performed service. Marketing and Outreach Activities One-on-one outreach made by implementation contractor with building owners/property managers continues to be effective form of marketing. Student Energy Education Participating Customer Information not provided for individual participants; only for teachers. Customer milestone tracked is the date kits are shipped. Measure Specific Information not applicable as all kits the same and info provided by implementer on spec sheets. Estimated savings are included as well as equipment useful life. Kits are provided by OG&E at no cost to participants. Reported measure type of equipment replaced is tracked by participant surveys from implementer. Measure Codes individual measures not identified; all kits provided to participants are supposed to be the same. Vendor Specific Information n/a measures self-installed Program Tracking Information date of the initial program contact provided. Rebate information n/a; provided at no cost to participants Marketing and Outreach Activities RAP conducts a well-established pattern of outreach activities. It is not known whether OG&E keeps records of how many outreach letters the staff sends each year or to whom. RAP handles all other marketing. Applied Energy Group, Inc. 42

262 SECTION 4 Commercial & Industrial Program Findings Process Evaluation for C&I Programs In-Depth Interviews In fall 2015 AEG conducted in-depth interviews with the program manager for the C&I Lighting and Standard Offer programs (see Appendix A for the interview guide). The following are some of the key points from the interview: Program Goals The Commercial programs are doing very well in PY2015. The Commercial Lighting program met its goal in the autumn of 2015, and was on track to exceed the savings goal at the time of the interview. The C&I SOP (including direct install) was also on track to slightly surpass its savings goals. These programs have had trouble in the past meeting their goals. The program manager feels that the longevity of the program and the consistency of marketing both by himself and the OG&E marketing team has helped inform the public and trade allies that rebates are available. Marketing and Outreach OG&E has done a very good job of marketing the program in 2015: radio, newspapers, bill stuffers, presentations at trade and civic organizations, and the internet have all been utilized to market the programs. Figure 4-1 Example Print Ad for OG&E Commercial Programs The program manager also conducts one on one calls/visits to both customers and trade allies. OG&E works with an implementation contractor (CLEAResult) who provides support for the entire program, implements the direct install portion of SOP and helps with marketing. According to the program manager it s taken about a year and a half for customers to understand that CLEAResult is representing OG&E. Now that they ve reached the point where customers are aware of the relationship the co-marketing is working very well. Applied Energy Group, Inc. 43

263 APSC FILED Time: Commercial 5/2/2016 2:38:38 & Industrial PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 QA/QC OG&E has been on every job that is completed through the program. For every lighting upgrade OG&E and the implementer have gone on-site before the project is started and after it has been completed. OG&E has done some spot checking and billing analysis to make sure they are getting an accurate picture of the customers savings from the project. The program manager has found that doing implementer EMV on the project can be helpful in getting customers to move forward with a project. Seeing the actual savings in this way can help get customers off the fence. Market Barriers For HVAC the program manger does not believe the rebates are helpful in moving projects forward. He does not consider them to be an incentive because they are not large enough to make a difference. For HVAC measures trade allies do not understand how to use the incentive to help m ake the sale. This has been a challenge for the program. Trade Allies The program manager does extensive outreach with many trade allies, including manufacturers, distributors, contractors, electricians, architects and engineering firms. He attends meetings to explain the programs and has one on one visits as well. Outreach with trade allies has paid off in Many trade allies are now aware of the program and reach out proactively to the program manager when preparing a bid. Customer Satisfaction According to the program manager, customers are tickled to death to get money back from their electric company. Changing the lighting program to providing incentives for kwh savings rather than kw savings has been very successful. Customers have responded really well to this and participation has increased. Program Improvements OG&E is changing the program so both lighting and SOP are under one umbrella ; it will be presented to customers as one program. There will still be individual lighting and HVAC goals, but if, for example, it s a particularly strong lighting year more funds can be used for that measure. The program manager would like to add LED bulbs to the direct install portion of the program. C&I Programs Adherence to Protocol A The tracking systems for each of the databases conform reasonably well to the tracking system protocol developed for use in Arkansas. Table 4-1 shows a summary of how well the commercial program tracking systems meets the components of the protocol. Applied Energy Group, Inc. 44

264 APSC FILED Time: 5/2/2016 Commercial 2:38:38 & PM: Industrial Recvd 5/2/2016 Program 2:33:07 Findings PM: Docket TF-Doc. 298 Table 4-1 Adherence to Protocol A Commercial Programs Commercial Lighting Participating Customer Information contains all information required including customer identifier (account number), and customer contact information. Also includes milestone dates in terms of the date the project was created, approved, and the rebate check was issued. Measure Specific Information collects type of equipment (and measure name), quantity, existing and new wattage, hours of operation, building type, temperature range and estimated savings. Would like to see more detail in the tracking file about the specific lamps installed and replaced. Categorical measure names can be too general. Measure Codes includes fixture identification number for installed equipment. Vendor Specific Information contractor information included on application such as contact name and phone number; could collect type of contractor (equipment or installation) in online application. Program Tracking Information tracks dates of program contacts (application, approval, rebate sent, etc.), amount of incentive, and application status. Marketing and Outreach Activities OG&E has done extensive marketing over the years which is paying off in the long-term resulting in high participation rates this year. Standard Offer Participating Customer Information includes all information required including customer contact information, customer identifier (account number), location of building site, date approved and date completed. Measure Specific Information includes new equipment type, description size, quantity, and efficiency level; includes estimated savings, hours of operation, and some information on replaced equipment. Measure Codes n/a; fields could be used for a measure description such as motors, chiller, HVAC, etc. Vendor Specific Information contractor information included in online portal such as contact name and phone number; could collect type of contractor. Program Tracking Information tracks dates of program contacts (application, approval, rebate sent, etc.), amount of incentive, and application status. Marketing and Outreach Activities OG&E has done extensive marketing over the years which is paying off in the long-term resulting in high participation rates this year. Standard Offer Direct Install Participating Customer Information includes all information required including customer contact information, customer identifier (account number), address, and date installed. Measure Specific Information includes type and quantity of measures installed. Measure Codes n/a; includes description for each measure installed. Vendor Specific Information not included. Program implemented by third party. Marketing and Outreach Activities one on one contacts are the most successful. Applied Energy Group, Inc. 45

265 APSC FILED Time: Commercial 5/2/2016 2:38:38 & Industrial PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 Commercial Lighting Impact Evaluation Gross Impacts OG&E provided AEG with a detailed tracking database summarizing the PY2015 Commercial Lighting projects. There were three sets of projects: 1) custom projects implemented by OG&E; 2) projects implemented by Direct Options; and 3) projects implemented by CLEAResult. AEG used the provided datasets to replicate savings calculations for the entire population of PY2015 projects. Then, AEG designed and selected a representative sample for documentation review of projects implemented by Direct Options (as discussed in Section 2). For each sampled project, we reviewed the reported impacts, gathered and reviewed detailed project documentation, and used TRM 5.0 algorithms to develop adjusted energy and demand impacts. We then extrapolated the sample results to the population using a combined ratio estimate as described in Section 2. Table 4-2 summarizes the gross energy and demand impacts for the program for the first year (2015) and the lifetime of the measure. Table 4-2 Commercial Lighting: Summary of Evaluated Gross Impacts Implementer Record Count Evaluated First Year kwh Savings Evaluated Lifetime kwh Savings Evaluated First Year kw Reduction Evaluated Weighted Average Lifetime kw Reduction Direct Options 192 5,534,013 74,117, CLEAResult ,289 8,306, OG&E Custom 8 296,053 2,989, Total 263 6,595,356 85,413,820 1, The subsections below describe results from the replication and documentation review steps in greater detail. Replication of Savings Calculations Table 4-3 provides a summary of findings from the PY2015 replication effort for the Commercial Lighting program. AEG conducted the replication process in two phases. In Phase I, AEG reproduced calculations using TRM 5.0 algorithms along with data that was provided for the algorithm variables in the tracking databases. In Phase II, AEG used best judgment to make corrections and adjustments to tracked data, and then recalculated impacts. Applied Energy Group, Inc. 46

266 APSC FILED Time: 5/2/2016 Commercial 2:38:38 & PM: Industrial Recvd 5/2/2016 Program 2:33:07 Findings PM: Docket TF-Doc. 298 Table 4-3 Commercial Lighting: Summary of Savings Replication Results Direct Options CLEAResult (Partial Year)* OG&E Custom Replication Summary Energy Savings (kwh) Demand Reduction (kw) Energy Savings (kwh) Demand Reduction (kw) Energy Savings (kwh) Demand Reduction (kw) First Year Impacts Reported 5,739, , , Replication Phase I 5,781, , , Replication Phase II 5,633, , , Replication Realization Rate 98% 110% 100% 103% 93% 83% Lifetime Impacts ( )** Replicated Phase II 75,534, ,128, ,989, *Partial year results were evaluated during the replication step for projects implemented by CLEAResult. The partial year results were then extrapolated to full year impacts. **Lifetime energy savings are the total energy savings over the measure lifetime; lifetime demand reduction values are the average demand reduction over the full measure lifetime. The subsections below describe the replication results, organized by implementer. Direct Options Database Replication The Direct Options tracking database for PY2015 consisted of 192 individual measure entries: 181 fixture retrofits, six lighting control additions, four delamping measures, and one new construction measure. All Direct Options projects were included in the savings replication step. First Year Impacts For the Phase I replication, AEG reproduced energy savings and demand reductions using variables provided in the Direct Options tracking database. Provided parameters included baseline and installed case fixture wattages, fixture counts, annual operating hours (AOH), building type, and temperature range. These were used along with CF and IEF values from TRM 5.0 in an attempt to replicate reported savings values for each measure entry. The Phase II replication involved adjusting the reported values in the database based on AEG s best judgment and TRM guidance. For each measure entry, AOH values were adjusted to 8,760 hr/yr if the retrofit type was an exit sign; otherwise, the AOH values were kept as reported since the TRM does not mandate adherence to Table 362 (TRM 5.0) if the AOH is known. The CF values were sourced from the TRM and based on the reported building type; these were changed to a value of 0.26 if the retrofit type was a lighting control and to a value of 1 for retrofits with 8,760 hr/yr operation. Interactive effects factors were based on the reported temperature ranges; a few measure entries were changed from ACSpaceMinimumTemps (refrigerated freezer space) to ACSpaceMediumTemps (refrigerated cooler space) to match the originally reported energy savings values (the demand reduction values did not match in any case). AEG then used the Standard Wattage Table in Appendix E of the Arkansas TRM to match the reported wattages and fixture types to deemed fixture codes and standard fixture wattages as well as possible. While only about one-third of measure entries provided exact fixture specifications, AEG was able to find matches for most entries. For example, some fixtures like Applied Energy Group, Inc. 47

267 APSC FILED Time: Commercial 5/2/2016 2:38:38 & Industrial PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 those with F54T5HO lamps were clearly reported as only the lamp wattages without the total fixture wattage. Overall, the replication process resulted in a total energy savings realization rate (RR) of 98% and a total demand reduction RR of 110%. Lifetime Impacts Lifetime energy savings and weighted average peak demand reductions were based on the Phase II replication. To calculate lifetime impacts, it was necessary to determine fixture lifetimes and any applicable baseline changes. Each retrofit was classified into different lamp type categories based on Table 360 of TRM 5.0 to determine the average EUL for each retrofit type. For measures that replaced incandescent lamps 100 W or less, first year impacts were based on the reported baseline; impacts for all following years of the measure life were based on the corresponding Tier 1 Energy Independence and Security Act of 2007 (EISA) baseline wattage from Table 358 of TRM 5.0. LEDs that replaced screw-in lamps (i.e. halogens, incandescents, or CFLs) were assumed to have an integrated ballast and a lifetime of 9 years; LEDs that replaced HID or linear fluorescent fixtures (as well as pin-based and exit light fixtures) were assumed to have remote drivers and a lifetime of 15 years, per TRM 5.0. It is unclear why the Commercial Lighting Efficiency section of the TRM does not provide Tier 2 EISA wattages for screw -in lamps as they are provided for Residential Lighting Efficiency; only Tier 1 EISA equivalent wattages are provided. For consistency with residential lighting, impacts in 2023 (9 th year of measure life and last year of measure life for screw-in LEDs with integrated ballasts) were evaluated with Tier 2 EISA baselines from Table 174 of the TRM 5.0. For T12 fixtures replaced with T8 or T5 fixtures, the Arkansas TRM specifies an EUL of 8.5 years based on the estimated RUL of 8.5 years for T12 fixtures (in 2012). 45 Thus, the T12 baseline is applicable for the full 8.5 years for these measures. The lifetime of LED measures that replace T12 fixtures, however, is 15 or 9 years (depending on presence of integrated ballast). Thus, the applicable baseline after 8.5 years is a T8 fixture wattage (corresponding FxxLL fixture code in the TRM Standard Wattage Table). Impacts for T12 to LED measures were evaluated with this reduced baseline after 8.5 years. AEG also devised a consistent approach to estimate lifetime impacts for delamping measures. The first year (2015) impacts were based on the stated baseline wattage, and the most likely substitute fixture (what the fixture would have been replaced with if it was not removed) was determined. 46 The delamping impacts from a baseline of the substitute fixture s wattage were then applied for the remaining years of the original fixture s lifetime. CLEAResult Database Replication To help meet OG&E s reporting timelines, AEG performed the savings replication process in the Fall of 2015 on partial year results for projects implemented by CLEAResult. AEG then used realization rates from the partial year results to estimate full year evaluated impacts. As of early September, the CLEAResult tracking database for PY2015 consisted of 37 records. These are the projects replicated. The replication results were then applied to the full year set of projects, which included a total of 63 records. 45 Frontier Associates. Petition to Revise Existing Measurement & Verification Guidelines for Lighting Measures for Energy Efficiency Programs: Docket No Public Utility Commission of Texas. Approved June 6, T8 equivalents for existing T12 fixtures were found in the AR TRM s Standard Wattage Table. A delamped 400 W metal halide was assumed to be replaced by a 156 W LED equivalent based on the schotopic/photopic multiplier method from Howard Lighting Products, HID to LED Wattage Cross Reference Table. A delamped 1000 W metal halide was assumed to be replaced by a W (375W average) LED based on typical equivalent replacement information from CREE and RAB Lighting. Applied Energy Group, Inc. 48

268 First Year Impacts APSC FILED Time: 5/2/2016 Commercial 2:38:38 & PM: Industrial Recvd 5/2/2016 Program 2:33:07 Findings PM: Docket TF-Doc. 298 For the Phase I replication, AEG reproduced energy savings and demand reductions using variables provided in the CLEAResult tracking database. Provided parameters included baseline and installed case fixture codes, fixture counts, AOH and CF values, building type, and type of heating and cooling. These were used along with IEF values from TRM 5.0 to replicate the reported savings values for each measure entry. Reported fixture codes were used to look up fixture wattages in Appendix E of the AR TRM 5.0. Since occupancy sensor controls were included with the fixture retrofits, impacts were calculated separately for fixture retrofit and controls and then combined to result in final impact values. 47 The Phase II replication involved correcting reported values as needed in the database. The only adjusted variable in the algorithm was the CF. The CF was changed to 1 for retrofits with reported 8,760 hr/yr operation. Overall, the replication process resulted in a total energy savings RR of 100% and a total demand reduction RR of 103%. Lifetime Impacts There were no measure entries that replaced EISA-regulated incandescent lamps or resulted in delamping. LEDs that replaced screw-in lamps (i.e. halogens, incandescents, or CFLs) were assumed to have an integrated ballast and a lifetime of 9 years, per TRM 5.0. For T12 fixtures replaced with T8 or T5 fixtures, the Arkansas TRM specifies an EUL of 8.5 years based on the estimated RUL of 8.5 years for T12 fixtures (in 2012). 48 Thus, the T12 baseline is applicable for the full 8.5 years for these measures. The lifetime of LED measures that replace T12 fixtures, however, is 15 or 9 years (depending on presence of integrated ballast). Thus, the applicable baseline after 8.5 years is a T8 fixture wattage (corresponding FxxLL fixture code in the TRM Standard Wattage Table). Impacts for T12 to LED measures were evaluated with this reduced baseline after 8.5 years. OG&E Database Replication There were eight custom projects implemented by OG&E in PY2015: five new construction projects and three retrofit projects. All PY2015 custom projects were included in the savings replication step. A breakdown of the replication results for each project is provided in Table 4-4 for energy savings and Table 4-5 for demand reduction. AEG assigned project numbers NC1-NC5 to represent the new construction projects and R1-R3 to represent the retrofit projects. 47 Occupancy sensor controls were calculated with a Power Adjustment Factor of Frontier Associates. Petition to Revise Existing Measurement & Verification Guidelines for Lighting Measures for Energy Efficiency Programs: Docket No Public Utility Commission of Texas. Approved June 6, Applied Energy Group, Inc. 49

269 Table 4-4 Project APSC FILED Time: Commercial 5/2/2016 2:38:38 & Industrial PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 Energy Savings Replication Results for Custom Lighting Projects Reported Energy Savings (kwh/yr) Phase I Energy Savings Replication (kwh/yr) Phase II Energy Savings Replication (kwh/yr) Realization Rate (%) Lifetime Energy Savings (kwh) NC1 25,521 29,008 4,794 19% 74,307 NC2 27,035 27,035 27, % 419,043 NC3 24,226 24,226 24, % 375,503 NC4 31,733 31,733 31, % 491,862 NC5 45,946 45,946 45, % 712,163 R1 44,000 44,000 44, % 218,228 R2 34,219 34,219 34, % 207,102 R3 84,100 84,100 84, % 491,188 Total 316, , ,053 93% 2,989,395 Table 4-5 Project First Year Demand Reduction Replication Results for Custom Lighting Projects Reported Demand Reduction (kw) Phase I Demand Reduction Replication (kw) Phase II Demand Reduction Replication (kw) Realization Rate (%) Weighted Average Lifetime Demand Reduction (kw) NC % 0.88 NC % 4.19 NC % 6.29 NC % 8.23 NC % R % 5.39 R % 4.18 R % Total % First Year Impacts For the Phase I replication, AEG reproduced energy savings and demand reductions using variables provided in each project s documentation tab. Provided parameters for new construction projects included installed case fixture codes, fixture counts, building type, code allowed lighting power density (LPD) from Table F-1 of TRM 5.0, gross lighted floor area, and type of heating and cooling. Provided parameters for retrofit projects included baseline and installed case fixture codes, fixture counts, AOH and CF values, building type ( Custom ), and type of heating and cooling. These were used along with IEF values from TRM 5.0 to duplicate reported impacts for each measure entry. AEG used reported fixture codes to look up fixture wattages in Appendix E of the TRM. Numerous issues were found during the Phase I replication of new construction projects. For NC1, the replicated energy savings during Phase I were higher than those reported. The replicated energy savings only matched reported savings for the reportedly Retail space if the AOH values were changed to the deemed hours for the Office building type while keeping the baseline LPD at Retail levels. All the provided photographic evidence showed that this was an Applied Energy Group, Inc. 50

270 APSC FILED Time: 5/2/2016 Commercial 2:38:38 & PM: Industrial Recvd 5/2/2016 Program 2:33:07 Findings PM: Docket TF-Doc. 298 office space instead of a stand-alone retail space; building type was adjusted to Office in the Phase II replication. The replicated demand reduction for NC1 disagreed with the reported reduction by 0.08 kw; the reason is not clear. The reported energy savings for other new construction projects were replicated at 100%, though peak demand reductions for the three retail stores in projects NC3-NC5 were replicated with an RR of 76.6%. These three projects were verified to be stand-alone retail stores, but reported demand reductions were found to have been calculated based on the CF for Retail: Strip Shopping & Non-enclosed Mall building type. All energy savings and demand reductions for retrofit projects were replicated at 100%; however, the CF of 0.89 for the Chicken House2 space type in both retrofit projects R1 and R2 is questionable since the reported AOH for this space are 592 hr/yr. 49 Since documentation was provided for all projects except retrofit R3, the Phase II replication involved adjusting reported values in the database based on evidence in the provided documents. The only adjustment made within the population was changing the building type for the NC1 project from Retail to Office, which changed the corresponding LPD requirements, AOH, CF, and IEF values, and resulted in a drastic difference in savings for NC1. Lifetime Impacts For measures that replaced incandescent lamps 100 W or less (retrofit projects R1 and R2 replaced 60 W incandescent lamps), first year impacts were based on the reported baseline, while impacts for all following years of the measure life were based on the corresponding EISA 2007 baseline wattage from Table 358 of TRM 5.0. One measure for delamping of 200 W incandescent lamps was also present in project R1. A measure life for halogens (1.5 years) was used for this measure since the TRM did not specify a measure life for incandescent lamps. A substitute wattage of 55 W was used to calculate the impacts for the remaining 0.5 years of measure life. LEDs that replaced screw-in lamps (i.e. halogens, incandescent lamps, or CFLs) were assumed to have an integrated ballast and a lifetime of 9 years, per TRM 5.0. For consistency with residential lighting baselines, impacts in 2023 (9 th year of measure life and last year of measure life for screw-in LEDs with integrated ballasts) were evaluated with Tier 2 EISA baselines from Table 174 of the TRM 5.0. Complete Documentation Review Direct Options Sample Review AEG reviewed worksheets and additional detailed documentation as available for a sample of 30 Commercial Lighting projects implemented by Direct Options. Information in the documentation was used to verify values recorded in the worksheets and in the tracking databases. AEG then adjusted parameters as necessary based on the documentation review to develop a set of evaluated energy savings and demand reductions for the sample. There were several issues and discrepancies found during the documentation review for both baseline and installed case fixtures, summarized in Figure Due to lack of evidence to the contrary, the CF and AOH were left as reported for these measure entries. Applied Energy Group, Inc. 51

271 APSC FILED Time: Commercial 5/2/2016 2:38:38 & Industrial PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 Count Discrepancies Space Type Categorization Wattage Discrepancies Invoice Issues Number of Rebate IDs Figure 4-2 Summary Counts of Main Issues in Commercial Lighting Documentation Review The following quality assurance and quality control (QA/QC) issues expand on the findings in Figure 4-2: Incorrect categorization of space types. Inadequate documentation of baseline equipment (i.e. number of fixtures, exact fixture type, fixture input wattage, etc.). Lack of documented fixture input wattage for some proposed lighting systems (i ncluding lack of OEM specification sheets). Lack of documentation for delamping measures (e.g. lack of labor quote via invoice or lack of specification of removed baseline equipment). Appendix C describes the documentation review findings in detail for each sample point. First Year Impacts Table 4-6 depicts average replicated and evaluated first year energy savings and demand reduction for the sampled projects, by stratum. The table also shows the number of sample points in each stratum, the stratum weights, and the overall combined ratios used to extrapolate sampled results to the population for 2015 (first year). Applied Energy Group, Inc. 52

272 APSC FILED Time: 5/2/2016 Commercial 2:38:38 & PM: Industrial Recvd 5/2/2016 Program 2:33:07 Findings PM: Docket TF-Doc. 298 Table 4-6 C&I Lighting Documentation Review Results: Sampled Projects Stratum # of Sample Points Stratum Weights Average kwh Savings for Sample, 2015 Average kw Reduction for Sample, 2015 Replicated Evaluated Replicated Evaluated 1. Retrofit: Low Savings 2. Retrofit: Medium Savings 3. Retrofit: High Savings 4. Retrofit: Highest Savings (census) 5. Sensor Projects (census) 6. Delamp Projects (census) 7. New Construction Projects (census) ,201 5, ,621 33, , , ,076,697 1,076, ,171 41, ,338 54, ,844 15, Combined Ratio Table 4-7 reports the population-level evaluated gross impacts estimated by extrapolating the sample points to the population of PY2015 Commercial Lighting projects implemented by Direct Options. It compares these evaluated findings with OG&E s reported impacts and the replicated impacts AEG developed for the population of Direct Options projects during the tracking system replication step. Table 4-7 Expansion of Sample Data to Population Using Ratio Estimation: Commercial Lighting Projects Implemented by Direct Options First Year Impact Reported Gross Impacts Replicated Gross Impacts Combined Ratio Evaluated Gross Impacts* Energy Savings, kwh/yr 5,739,129 5,633, ,534,013 Demand Reduction, kw *Evaluated impacts equal replicated impacts multiplied by combined ratio 50 As described in Section 2, the combined ratio is used to derive the verified savings from the reported, which is used in the calculation of the realization rate. Applied Energy Group, Inc. 53

273 Achieved Precision of the Sample APSC FILED Time: Commercial 5/2/2016 2:38:38 & Industrial PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 AEG designed the sample for the Commercial Lighting documentation review to achieve the best precision possible given the sample size of 30. Once we completed the documentation review and expanded the sample to the population, we calculated the achieved precision of the estimates. Table 4-8 reports the achieved precision at the 90% confidence level. The 90% confidence level relative precision for the projects implemented by Direct Options was 1.0% for energy savings and 1.2% for demand reduction. The uncertainty in the estimates is only due to Strata 1, 2 and 3. There was no sampling error for Strata 4 through 7, since they were census strata. Table 4-8 Achieved Precision of the Impact Estimates: Commercial Lighting Projects Implemented by Direct Options First Year Impact Relative Precision of Evaluated Gross Impact Energy Savings, kwh/yr 1.0% Demand Reduction, kw 1.2% Lifetime Impacts AEG determined lifetime impacts for the population of Commercial Lighting projects implemented by Direct Options using the approach described in Section 2 (Evaluation Methods). Table 4-9 summarizes the findings. Table 4-9 Evaluated Lifetime Energy and Demand Impacts: Commercial Lighting Projects Implemented by Direct Options Lifetime Impact Evaluated Gross Impact Lifetime Energy Savings, kwh/yr 74,117,799 Weighted Average Demand Reduction, kw Net Impacts Table 4-10 summarizes the overall first year gross and net impact results for the PY2015 Commercial Lighting program, including impacts for projects implemented by Direct Options and CLEAResult, as well as OG&E s custom projects. AEG applied the NTG ratio of 0.99 to the gross impacts resulting in net impacts of 6,529,402 kwh per year and 1,076 kw. Table 4-10 Overall First Year Impacts for the PY2015 Commercial Lighting Program First Year Impact Reported Gross Impact Evaluated Realization Rate NTG Net Impact Annual Energy, kwh 6,821,191 6,595, ,529,402 Peak Demand Reduction, kw 1,020 1, ,076 Section 5 and Appendix D present the net lifetime impacts for this and other programs by measure. Applied Energy Group, Inc. 54

274 APSC FILED Time: 5/2/2016 Commercial 2:38:38 & PM: Industrial Recvd 5/2/2016 Program 2:33:07 Findings PM: Docket TF-Doc. 298 C&I Direct Install & Standard Offer Impact Evaluation Gross Impacts AEG applied TRM 5.0 algorithms and custom approaches to estimate the impacts for the C&I Direct Install and Standard Offer program components, using default input values for OG&E s service territory in Arkansas and program-specific data for participants. Table 4-11 depicts the first year and lifetime evaluated gross impacts for PY2015 program participants. Table 4-11 C&I Direct Install & Standard Offer Programs: Summary of Evaluated Gross Impacts Program Measure EUL Evaluated First Year kwh Savings Evaluated Lifetime kwh Savings Evaluated First Year kw Reduction Evaluated Weighted Average Lifetime kw Reduction CFLs 3 74, , C&I Direct Install Pre-Rinse Spray Valves 5 74, , Faucet Aerators ,939 1,919, Low-Flow Showerheads 10 53, , Vending Misers 5 66, , C&I Standard Offer All Measures 20 6,518,250 67,246, SOP Total 20 6,978,237 70,570, The subsections below describe results from the analysis of gross impacts for the C&I Direct Install and C&I Standard Offer programs in greater detail. C&I Direct Install There were five measures installed and evaluated in PY2015 through the C&I Direct Install program. Using data provided by OG&E and the implementer in the form of a tracking database and TRM-based constants, AEG estimated total program impacts for each measure in the PY2015 program. Table 4-12 summarizes the evaluated first year gross impacts for PY2015 projects. Applied Energy Group, Inc. 55

275 APSC FILED Time: Commercial 5/2/2016 2:38:38 & Industrial PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 Table 4-12 C&I Direct Install Measures: Summary of Evaluated Gross Impacts Measure Project Count Unit Count First Year kwh Savings kwh Realization Rate First Year kw Reduction kw Realization Rate CFL , % % Pre-Rinse Spray Valve , % % Faucet Aerator , % % Low-Flow Showerhead , % % Vending Miser , % % Total , % % The subsections below describe the impact results by measure type. Compact Fluorescent Lights AEG used TRM 5.0 algorithms and inputs from the tracking database to calculate the energy savings and peak demand reductions for the 52 projects representing 472 CFLs installed in PY2015. The analysis revealed that reported impacts were calculated with TRM 4.0 values. The only changes from TRM 4.0 to 5.0 were notable reductions in the CF values for Office and Retail: Excluding Malls & Strip Centers building segments. Inputs from the tracking database included baseline and installed wattages, quantity installed per project, and an indication of whether the project site had electric cooling. First Year Impacts To calculate impacts for commercial lighting, the building segment and HVAC system type are needed to determine the annual operating hours, coincidence factor, and interactive effects factors from tables provided in the TRM. Since there was no building segment or specific HVAC system type identified in the program tracking database, AEG developed matrices of per-unit impacts for each building segment and HVAC system in TRM 5.0 for each of the four baseline wattages in (60, 65, 72, 100W). The Tier 1 (EISA 2007) baselines used for each reported baseline are shown in Table 4-13 (65W was assumed to be in the 60W bucket). The calculated impacts were then used to match the reported impacts per unit installed and back-calculate the assumed building segment and HVAC system type per measure entry. Back calculations showed that many of the CFL projects (43 of 52) had an Office or Retail: Excluding Malls & Strip Centers building segment, meaning they had lower CF values than used in the reported impact calculations. Table 4-13 Mapping of EISA 2007 Wattages to Baseline Wattages Baseline Wattage Project Count EISA Wattage Applied Energy Group, Inc. 56

276 APSC FILED Time: 5/2/2016 Commercial 2:38:38 & PM: Industrial Recvd 5/2/2016 Program 2:33:07 Findings PM: Docket TF-Doc. 298 The final evaluated first year energy savings and demand reductions are 74,241 kwh/yr and kw, respectively. These impacts correspond to realization rates of 100% for energy savings and 80% for demand reduction. The changes in CF values in TRM 5.0 relative to TRM 4.0 led to the lower evaluated demand impacts. Lifetime Impacts Commercial CFLs were assumed to have a three-year measure life based on a 10,000 hour manufacturer rated life for 13 W CFLs and a weighted average 3,230 annual operating hours across the program. 51 The first year impacts were based on the reported 60 W existing baseline since this is a direct install program. For the remaining two years of the CFL s EUL (2016 and 2017), a baseline of 43 W from EISA 2007 requirements was used. The total lifetime impacts for this measure are energy savings of 169,014 kwh and a weighted average peak demand reduction of kw. Pre-Rinse Spray Valves There were 11 spray valve measure installed in PY2015. AEG evaluated this measure using the TRM 5.0 algorithm. Though the TRM specifies average baseline and installed flow rates of 2.25 GPM and 1.28 GPM, installers measured the actual flow rates following Option A IPMVP M&V analysis guidelines. First Year Impacts All but two baseline and installed flow rate pairs were unique in PY2015. AEG used these flow rates to make a savings matrix and back-calculate the building segment type; all segments were reported as Casual Dining. Though it looks like most of these business types were grocery stores (with one lodging facility and one unknown), the casual dining segment specification makes sense given the other options in the TRM. All the grocery stores have some sort of deli where they do cook to-go food and need spray valves for cleaning pots and pans. The resulting calculated impacts using this approach were consistent with reported values. The faucet aerator measure saved 535,784 gallons of water per year. Adding regional average embedded energy and demand intensities to the analysis yielded total first year energy savings of 74,255 kwh and a total first year peak demand reduction of 8.15 kw. The resultant evaluated energy savings and demand reductions are 103% of the reported values. Lifetime Impacts The EUL for pre-rinse spray valves is 5 years, resulting in lifetime energy savings of 371,277 kwh and a weighted average lifetime demand reduction of 8.15 kw. Faucet Aerators AEG evaluated the 298 faucet aerators installed in PY2015 with the appropriate TRM 5.0 algorithms. Inputs from the tracking database included the flow rate of the installed equipment and quantity installed. Since building segment type was not specified in the tracking database, AEG developed per-unit TRM savings based on installed flow rate, building segment, and weather zone. These were then used to back-calculate the applicable building segment for each project. First Year Impacts AEG found one issue with the tracking database for this measure. The flow rate of one aerator was recorded in the tracking database as 0.5 GPM, but reported impacts were based on a 1.0 GPM installed case. After confirming with the program implementer that the correct flow rate was 1.0 GPM, AEG s replicated impacts matched the reported impacts. 51 This approach is consistent with TRM 5.0, and the annual operating hours were weighted based on actual program building segments and installations. Applied Energy Group, Inc. 57

277 APSC FILED Time: Commercial 5/2/2016 2:38:38 & Industrial PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 In all, the faucet aerators reduced water use by 1,896,186 gallons per year. With the addition of embedded impacts, the total evaluated first year impacts are energy savings of 191,939 kwh and a peak demand reduction of kw. The realization rates for energy savings and demand reduction are 104% and 102%, respectively. Lifetime Impacts The EUL for faucet aerators is 10 years, resulting in lifetime energy savings of 1,919,395 kwh and a weighted average lifetime demand reduction of kw. Low-Flow Showerheads There were 152 low-flow showerheads installed in PY2015. AEG evaluated the impacts using the appropriate TRM 5.0 algorithms. Since only the quantity of installed showerheads was recorded in the tracking database, AEG developed per-unit TRM savings based on installed flow rate and building segment. We then back-calculated the rated flow rate for the installed equipment as well as the assumed building segment for the project site based on per-unit kwh savings. First Year Impacts Based on back-calculation for applicable segment and flow rate, AEG determined that the installed showerheads were 1.5 GPM and were installed in Commercial and Lodging/Hospitality segments. The installation of 1.5 GPM showerheads is consistent with findings from PY2014, where the implementer confirmed that all showerheads were rated at 1.5 GPM. The low-flow showerheads reduced water use by 385,736 gallons per year. With the addition of embedded impacts, the total evaluated first year impacts are energy savings of 53,460 kwh and a peak demand reduction of 3.22 kw. The realization rates for energy savings and demand reduction are 103% and 106%, respectively. Lifetime Impacts The EUL for low-flow showerheads is 10 years, resulting in lifetime energy savings of 534,600 kwh and a weighted average lifetime demand reduction of 3.22 kw. Vending Misers There were 41 vending machine occupancy controls installed in PY2015. AEG evaluated the impacts using deemed values in TRM 5.0 and assuming the controls were installed on refrigerated vending machines per information provided by the implementer. First Year Impacts The first year evaluated energy savings and demand reductions are 66,092 kwh and 1.23 kwh. Therefore, the realization rates for the evaluated impacts are both 100%. Lifetime Impacts The EUL for vending misers is 5 years, resulting in lifetime energy savings of 330,460 kwh and a weighted average lifetime demand reduction of 1.23 kw. C&I Standard Offer AEG conducted complete documentation review for a sample of 20 Rebate IDs associated with C&I Standard Offer projects. The projects evaluated consisted of six measures with deemed savings algorithms in TRM 5.0, as well as several fully custom measures. The six measures with deemed algorithms in the TRM are as follows: 1) premium-efficiency motor upgrades, 2) unitary and split system HVAC equipment upgrades, 3) chiller upgrades, 4) strip curtain installation for commercial refrigeration equipment, 5) door gasket installation for commercial refrigeration equipment, and 6) computer power management. C&I Direct Install projects were not included in the documentation review step. Applied Energy Group, Inc. 58

278 APSC FILED Time: 5/2/2016 Commercial 2:38:38 & PM: Industrial Recvd 5/2/2016 Program 2:33:07 Findings PM: Docket TF-Doc. 298 AEG reviewed worksheets and additional detailed documentation as available for the sample. Information in the documentation was used to verify values recorded in the worksheets and in the tracking databases. AEG also conducted research for additional information to verify and confirm key parameters in the savings calculations in cases where there was not adequate documentation to substantiate the reported information. For measures found in TRM 5.0, AEG reproduced energy and demand impact calculations according to the algorithms specified in the TRM using the verified inputs. For measures not found in TRM 5.0, AEG reproduced energy and demand impact calculations according to the implementer methodology and best practice engineering judgment using verified inputs. AEG then develop a set of evaluated energy savings and demand reductions for the sample. Appendix C describes the documentation review findings in detail for each sample point. First Year Impacts Table 4-14 depicts average reported and evaluated first year energy savings and demand reduction for the sampled projects, by stratum. The table also shows the number of sample points in each stratum, the stratum weights, and the overall combined ratios used to extrapolate sampled results to the population for 2015 (first year). Table 4-14 C&I Standard Offer Documentation Review Results: Sampled Projects Stratum # of Sample Points Stratum Weights Average kwh Savings for Sample, 2015 Average kw Reduction for Sample, 2015 Reported Evaluated Reported Evaluated 1. Low Savings ,119 3, Medium Savings ,047 23, High Savings ,189 71, Highest Savings (census) , , Combined Ratio Table 4-15 reports the population-level evaluated gross impacts estimated by extrapolating the sample points to the population of PY2015 C&I Standard Offer projects. It compares these evaluated findings with OG&E s reported gross impacts. Table 4-15 Expansion of Sample Data to Population Using Ratio Estimation: C&I Standard Offer Projects First Year Impact Reported Gross Impacts Combined Ratio Evaluated Gross Impacts* Energy Savings, kwh/yr 7,324, ,518,250 Demand Reduction, kw *Evaluated impacts equal reported impacts multiplied by combined ratio Achieved Precision of the Sample AEG designed the sample for the C&I Standard Offer documentation review to achieve the best precision possible given the sample size of 20. Once we completed the documentation review Applied Energy Group, Inc. 59

279 APSC FILED Time: Commercial 5/2/2016 2:38:38 & Industrial PM: Recvd Program 5/2/2016 Findings 2:33:07 PM: Docket TF-Doc. 298 and expanded the sample to the population, we calculated the achieved precision of the estimates. Table 4-16 reports the achieved precision at the 90% confidence level. The 90% confidence level relative precision was 3.5% for energy savings and 16.3% for demand reduction. The uncertainty in the estimates is only due to Strata 1, 2 and 3. There was no sampling error for Stratum 4 since that was a census stratum 52. Table 4-16 Achieved Precision of the Impact Estimates: C&I Standard Offer Projects First Year Impact Relative Precision of Evaluated Gross Impact Energy Savings, kwh/yr 3.5% Demand Reduction, kw 16.3% Lifetime Impacts AEG determined lifetime impacts for the population of C&I Standard Offer projects using the approach described in Section 2 (Evaluation Methods). Table 4-17 summarizes the findings. Table 4-17 Evaluated Lifetime Energy and Demand Impacts: C&I Standard Offer Projects Lifetime Impact Evaluated Gross Impact Lifetime Energy Savings, kwh/yr 67,246,200 Weighted Average Demand Reduction, kw Net Impacts Table 4-18 shows the first year impacts for the PY2015 C&I Direct Install and Standard Offer program components. AEG determined net impacts by applying the NTG ratios of 1.0 for direct installation projects and 0.96 for custom projects as described in Section 2. Evaluated first year net impacts for direct installation and standard offer measures are 6,717,507 kwh per year for energy and 672 kw for demand. Realization rates for energy and demand are 90% and 85%, respectively. Section 5 and Appendix D present the net lifetime impacts for this and other programs by measure. 52 The higher than expected value of 16.3% for the relative precision of the demand reduction is primarily due to the high variance in reported demand reduction values and the low correlation in stratum 2. Applied Energy Group, Inc. 60

280 Table 4-18 APSC FILED Time: 5/2/2016 Commercial 2:38:38 & PM: Industrial Recvd 5/2/2016 Program 2:33:07 Findings PM: Docket TF-Doc. 298 First Year Impacts for PY2015 C&I Direct Install and Standard Offer Programs Program Measure Metric Direct Install C&I Standard Offer CFLs Pre-Rinse Spray Valves Faucet Aerators Low-Flow Showerheads Vending Misers All Measures Totals Energy Savings (kwh/yr) Demand Reduction (kw) Energy Savings (kwh/yr) Demand Reduction (kw) Energy Savings (kwh/yr) Demand Reduction (kw) Energy Savings (kwh/yr) Demand Reduction (kw) Energy Savings (kwh/yr) Demand Reduction (kw) Energy Savings (kwh/yr) Demand Reduction (kw) Energy Savings (kwh/yr) Demand Reduction (kw) Reported Gross Impact Evaluated Realization Rate NTG Net Impact 74,241 74, % 1 74, % ,943 74, % 1 74, % , , % 1 191, % ,796 53, % 1 53, % ,092 66, % 1 66, % ,324,803 6,518,250 89% ,257, % ,772,630 6,978,237 90% ,717, % Applied Energy Group, Inc. 61

281 SECTION 5 Lifetime Impacts This section shows the net lifetime impacts by measure and by program. Full details are shown in Appendix D. Net Lifetime Impacts by Program As shown in Table 5-1 and Figure 5-1, the Commercial Lighting Program provided the most lifetime energy (kwh) savings and weighted average lifetime demand (kw) reduction. However, this proportion was much less than in PY2014 where this program provided about 75% of savings. C&I Standard Offer and the Multi-Family Direct Install program provided another 30% and 28% of energy savings respectively. Table 5-1 Lifetime Energy (kwh) Savings and Weighted Average Lifetime Demand (kw) Reduction by Program Program Lifetime Energy (kwh) Savings Weighted Average Lifetime Demand (kw) Reduction MF Direct Install 60,362, SEE LivingWise 2,768, Commercial Lighting 84,559, C&I Direct Install 3,324, C&I Standard Offer 64,556, Total 215,572,219 1, AEG defined Weighted Average Lifetime Demand Reduction as the sum of all annual demand reductions for each measure divided by the maximum lifetime in that program. So for MFDI, for example, we divide total by 18 even if some measures only last 9/10/11 years. The total reduction is not determined by simply adding the demand reduction for each program. Applied Energy Group, Inc. 62

282 APSC FILED Time: 5/2/2016 2:38:38 Lifetime PM: Recvd Impacts 5/2/2016 2:33:07 PM: Docket TF-Doc % 28% Multi-Family Direct Install SEE LivingWise 2% 1% Commercial Lighting C&I Direct Install C&I Standard Offer 39% Figure 5-1 Lifetime Energy (kwh) Savings by Program Net Lifetime Impacts by Measure As shown in Table 5-2 and Figure 5-2, C&I lighting measures provided 39% of net lifetime energy (kwh) savings and C&I Standard Offer (not including Direct Install Measures) provided a further 30% of net lifetime energy (kwh) savings. In PY2014, 75% of these savings were from just the C&I lighting measures. C&I Lighting and Standard Offer also result in a total of and weighted average lifetime demand (kw) reduction respectively for this year. Combined with duct sealing, these three measures provided 91% of lifetime energy savings. Table 5-2 Lifetime Energy (kwh) Savings and Weighted Average Lifetime Demand (kw) Reduction by Measure Measure Lifetime Energy (kwh) Savings Weighted Average Lifetime Demand (kw) Reduction Effective Useful Life (TRM) Advanced Power Strips 3,116, Air Infiltration 3,821, C&I Lighting 84,559, C&I Standard Offer (not Direct Install) 64,556, CFLs 2,108, Duct Sealing 46,339, For all CFLs installed in Arkansas in PY2015, this is the weighted average lifetime demand reduction expected over the course of 9 years (residential CFL measure EUL). CFLs installed through the C&I Direct Install Program had an EUL of 3 years. Applied Energy Group, Inc. 63

283 APSC FILED Time: 5/2/2016 2:38:38 Lifetime PM: Impacts Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Faucet Aerators 3,847, Low Flow Showerheads 6,520, Pre-Rinse Spray Valves 371, Vending Misers 330, Total 215,572,219 1, % 3% 0% 0% 1% 2% Advanced Power Strips Air Infiltration 22% C&I Lighting 39% C&I Standard Offer (Non-DI) CFLs 1% Duct Sealing Faucet Aerators Low-Flow Showerheads Pre-Rinse Spray Valves 30% Vending Misers Figure 5-2 Lifetime Energy (kwh) Savings by Measure Applied Energy Group, Inc. 64

284 SECTION 6 Recommendations Multi-Family Program Recommendation 1: Continue to emphasize non-energy benefits such as water savings in all marketing materials. Rationale: Water savings is a large benefit of the program and including this in marketing materials could help increase participation in the program. Recommendation 2: Ensure the consistent application of embedded peak demand impacts for measures with water savings. Rationale: In PY2014 and PY2015, AEG found that the embedded demand reductions (kwh savings multiplied by a factor of kw/kwh) for all installations were calculated using energy savings that already included the embedded energy savings. This is the incorrect approach; embedded demand reductions should be calculated based on the direct energy impacts only. Recommendation 3: Use Fort Smith-specific values for embedded water impacts. Rationale: In prior years, the weighted average embedded energy and demand intensity values for OG&E s service territory were used for the Multi-Family Direct Install Program evaluations. AEG has provided OG&E with location-specific factors for indirect water impacts in Fort Smith, which should be used going forward. This approach will lead to more accurate impact estimates since most Arkansas participants are located in Fort Smith. 56 It will also allow for greater consistency across the programs, since the Fort Smith values are already being used for water saving measures in the SEE LivingWise program. Recommendation 4: Ensure the absence of rounding errors in parameters that are recorded and tracked in the tracking databases. Rationale: Reported values for the HVAC HSPF and HVAC Tonnage fields in the tracking database were shown as rounded or truncated integer values; this resulted in replication errors for duct sealing measures. AEG recommends reporting full values as recorded in the direct install measure calculators. Recommendation 5: Determine in-service rate for advanced power strips. Rationale: There was a significant increase in the amount of advanced power strip units installed from PY2014 (only 2 units) to PY2015 (1,373 units). AEG recommends that OG&E investigate and develop in-service rate factors for advanced power strips. An inservice rate study would help quantify persistence of the measure, including assessing if customers use the advanced power strips for applications other than home entertainment systems. Recommendation 6: Record system and device type plugged into installed advanced power strip. 56 Using embedded rates for Fort Smith results in realization rates of 98% and 93% for energy savings and demand reduction values, respectively, for faucet aerators; these realization rates would be 98% and 94% for low-flow showerheads. The realization rates quoted are based on PY2015 reported savings that include the double-counting error present in demand reduction calculations. Applied Energy Group, Inc. 65

285 APSC FILED Time: 5/2/2016 2:38:38 Recommendations PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Rationale: OG&E targets installations of advanced power strips on home entertainment systems. However, the AR TRM has deemed savings for a wide variety of peripheral devices on two general system types: 1) home entertainment, and 2) home office. The TRM also has average deemed values that represent an application with average kwh and kw impacts. Because the deemed savings vary significantly with system and device type, and since the impacts are considerably smaller for the home office and average system applications, actively recording and tracking the system application for all installations will streamline the evaluation process 57 and will lead to more accurate reported and evaluated savings estimates. SEE LivingWise Program Recommendation 1: Conduct an independent survey of teacher activities to obtain better information on the likely installation of the measures in the take-home kits. Rationale: Overall, the student survey response rate each year is over 50%, with responses for many classes in PY2015 each exceeding 80%. Each year, however, many teachers do not return any surveys. AEG has not been able to confirm that students in these classes are taught the curriculum, are given kits, and install measures at rates similar to those who did return surveys. A teacher survey could provide evidence to support the claim or information to develop more accurate installation rate estimates. Recommendation 2: Consider developing more complete heat pump water heating and HVAC technology saturations, all from the same source and for all residents served by OG&E in Arkansas. Rationale: While OG&E was able to provide some information for PY2015, we could not determine consistent saturations for HVAC heat pumps. Therefore, we had to use a different source to determine saturations for HVAC heat pumps in the service territory. To fully comply with the TRM specifications for measure savings, OG&E needs to develop information on these electric technology HVAC shares consistent with the technology classifications for HVAC systems. Commercial Lighting Program Recommendation 1: Establish consistent conventions for precision of reported results. Rationale: Having a sufficient number of significant digits in the tracking database for all energy and demand impacts will improve the accuracy of reported results. AEG suggests reporting values to at least two decimal places for energy savings and three for demand reduction. Recommendation 2: Record variables for new construction in the tracking database and ensure the reported building type is correct. Rationale: If worksheet parameters are recorded in the tracking database for new construction projects, impacts can be replicated for the whole population and the documentation review used only to fine-tune the impacts. AEG suggests recording important values for new construction projects such as baseline and proposed building wattages in the tracking database, as well as ensuring that the building type and temperature range is correct. 57 If the system type is clearly indicated in the tracking database for each installation, evaluators would not have to request additional documentation or confirmation of the application. Applied Energy Group, Inc. 66

286 APSC FILED Time: 5/2/2016 2:38:38 Recommendations PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Recommendation 3: Improve photographic documentation practices and delamping documentation. Rationale: Having more informative photographs would help evaluators verify measures, thereby improving the accuracy of evaluated impacts. OG&E should encourage contractors to take clear photographs of nameplates of baseline and installed lamps and ballasts, not just shots of the space and functioning lights from afar. For spaces with refrigerated (medium or low) temperatures, contractors should be encouraged to take photographs of thermostats, if possible. More information on delamped fixtures is also required for verification. Contractors should be encouraged to take nameplate photographs of baseline fixtures and include delamping (i.e. labor required) in the submitted itemized invoices. Recommendation 4: Correct space temperature designations in the rebate database to align with designations used in the AR TRM. Rationale: The temperature ranges for the Low/Medium space temperature designations in the online rebate worksheets do not align with the ranges stipulated by the AR TRM. Correcting the discrepancy would ensure that the correct interactive effects factors from the TRM are used and allow for more accurate impact estimates. There should be three different types of designations in the TRM Air Conditioned Space Normal Temps. (>41 F), Refrigerated Space Med. Temps. (33-41 F), and Refrigerated Space Low Temps. ( F). C&I Standard Offer Program C&I Direct Install Recommendation 1: Consider the direct installation of LEDs instead of CFLs. Rationale: LEDs have several advantages over CFLs, including higher efficiency, longer life, better light quality, and superior control capability (e.g., dimming). As the costeffectiveness proposition continues to improve for LEDs, this technology better ensures persistent, long-term savings for lighting programs in Arkansas. Recommendation 2: Record additional parameters in the tracking database for direct install measures. Rationale: The tracking database provided to AEG for direct install measures was missing several parameters. Though most information is kept in other program documentation files managed by the program implementer, being able to readily output this information into one central tracking spreadsheet for evaluators to review would help evaluators conduct the impact analysis much more efficiently. We recommend including the following in the tracking database: Building segment for all measures HVAC system type for CFLs Water heater type for pre-rinse spray valves, faucet aerators, and low-flow showerheads Installed flow rate for low-flow showerheads Confirmation that vending misers are installed on refrigerated vending machines Recommendation 3: Use Fort Smith embedded energy and demand intensities to take into account indirect energy and peak demand impacts for measures with water savings. Rationale: Adding embedded impacts increases the program s energy savings and demand reductions. OG&E applies these impacts to other program measures with water Applied Energy Group, Inc. 67

287 APSC FILED Time: 5/2/2016 2:38:38 Recommendations PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 savings and it would be consistent to include them for the C&I direct install measures as well. C&I Standard Offer Recommendation 1: Increase the amount of EM&V done for customers in the project planning stage, particularly for larger projects. Rationale: EM&V conducted by the implementer which demonstrates savings that can be achieved has helped convince customers to move forward with upgrades. Recommendation 2: Develop documentation checklists that require the participants to submit documentation (e.g., manufacturer s technical specification sheets, photographs of nameplate with model and serial number, itemized invoices with equipment details, etc.) to substantiate parameters for the new and replaced equipment. Rationale: Including adequate documentation allows the evaluator to develop robust impact estimates by verifying operating parameters and making sure the impacts are calculated correctly. While AEG saw substantial improvement in this area in PY2015, the completeness of documentation was inconsistent from project to project. Implementing this recommendation will simplify and expedite the eventual engineering review task and potentially improve savings realization rates. Recommendation 3: For projects where the existing equipment still has remaining useful life per the TRM, Early Retirement project documentation should include supporting evidence (e.g., letter from customer and/or contractor) indicating that the original equipment was in working condition or repairable when it was replaced and it has since been decommissioned. Rationale: Additional documentation supporting Early Retirement will improve the defensibility of reported impacts. Recommendation 4: For Early Retirement projects, compare the age of existing equipment with the TRM maximum ages based on equipment type. If the age is older than the maximum allowable for Early Retirement, use the Replace on Burnout method to estimate impacts. Rationale: Following the TRM rules for Early Retirement and Replace on Burnout will improve realization rates. Recommendation 5: Ensure that the newly installed HVAC and motor equipment exceeds the minimum federal standard efficiency level. Rationale: OG&E should only provide rebates for equipment that is more efficient than the minimum efficiency requirements. Equipment installed in projects like Rebate # and OPP had efficiencies exactly equal to the federal minimum efficiency requirements. Exceeding minimum standards would result in a significant increase in lifetime impacts. Recommendation 6: Calculate energy and demand impacts for the door gasket measure according to the correct deemed impact values from the Arkansas TRM. Rationale: Implementing this recommendation would ensure the reported impacts for the door gasket measure are consistent with the requirements set out in the TRM, and could improve realization rates. Recommendation 7: Conduct quality assurance and quality control (QA/QC) review of motor replacement measure calculators if it continues to be used. Rationale: In one of the calculators used by for motor replacement (Rebate # ), the algorithm used to calculate energy savings erroneously included the coincidence factor. If this error is part of the calculator s default formulas, systematic and significant estimation errors could arise. Applied Energy Group, Inc. 68

288 APSC FILED Time: 5/2/2016 2:38:38 Recommendations PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Recommendation 8: For fully custom measures, require analysis calculations that show how all reported impacts are calculated; documentation of all assumptions and raw, unprocessed logged data should also be provided. Rationale: Implementing this recommendation would allow the evaluator to follow and replicate the original analysis, while improving estimation methods and the accuracy of impact estimates wherever possible. When the analysis calculations were requested for custom projects (such as Rebate # OPP and OPP235706), the evaluator was able to find the error in the original analysis and adjust the impact estimates accordingly. Recommendation 9: For fully custom measures, emphasize that demand reduction calculations be as consistent as possible with the defined OG&E system peak period (4:00 to 5:00pm, third consecutive hottest day of the year). Rationale: Following this approach for demand impacts would ensure that calculated demand reductions are representative of their impact on OG&E s system peak demand. Recommendation 10: When analyzing a custom project based on logged amperage data, use the appropriate engineering algorithm to estimate true power (kw). Rationale: Applying the correct formula to estimate true power would help ensure that reported impacts are not drastically reduced due to errors from the incorrect application of a standard engineering algorithm. In PY2015, the evaluator found that the implementer was estimating true power (kw) from amperage using a factor of 3 instead of the square root of 3 (1.73). If this error is systematic, significant estimation errors could arise. Recommendation 11: For projects that use regression analysis in an effort to produce estimates that are representative of typical first year and lifetime impacts, the following approach is recommended: A regression analysis should be performed which includes both pre- and post-retrofit data based on the respective historical weather and production data. The resultant regression should then be applied to a full year of typical weather and production data. The peak demand reduction would then be calculated specific to the OG&E system peak period. More importantly, at a minimum, key regression model specifications such as p-values and coefficients of determination (r 2 ) should also be reported. Rationale: Following this approach would assist the implementer to develop calculation methods consistent with best practice, helping to ensure reported impacts are verifiable and defensible. Having the model specifications available to the evaluator would facilitate model review and development of evaluated impacts, as well as potentially improve savings realization rates. Applied Energy Group, Inc. 69

289 SECTION 7 Response to PY2014 Recommendations SEE LivingWise Program Recommendation1: The TRM now makes a distinction between heat pumps and other electric heating systems for the calculation of savings from measures that reduce water use and for CFL interactive effects. OG&E does not currently have any information on the saturation of heat pumps or heat pump water heaters but needs to collect it. This could be done in one of the following ways: by having RAP add heat pumps to the to the water and space heating technology options listed on the student surveys, from which the heating technology shares are currently calculated, along with explicit instruction on how to identify a heat pump, or alternately by OG&E conducting a market study to assess the prevalence of heat pumps and heat pump water heaters in the area. OG&E Response: The student surveys were not modified to distinguish heat pumps from other electric water and space heating technologies, citing lack of confidence that students would be able to accurately identify them. OG&E did provide information on heat pump water heater and heat pump HVAC saturations from the Arkansas Energy Efficiency Potential Study, prepared Navigant Consulting, Inc., June 1, 2015 (but only the water heater saturation was usable). Recommendation2: Update the values in the tracking system to reflect the newer TRM that becomes available before the end of the program year. While OG&E s implementer, RAP, utilizes the TRM to calculate savings used in the Saratoga tracking system, the values are based on the TRM version available at the beginning of the program year rather than the TRM version in place at the end of the program year and upon which the claimed savings should be based. OG&E Response: This recommendation was implemented. The program implementer did recalculate its savings using TRM 5.0 for all of PY2015. Multi-Family Direct Install Program Recommendation1: Ensure the consistent application of embedded energy and peak demand savings for measures with water savings. OG&E Response: This recommendation was accepted and for energy savings the application of embedded savings for measures with water savings was done consistently. However, for demand reduction this was not the case; AEG adjusted for double counting of these impacts. Recommendation2: Ensure that low-flow showerheads have flow rates of 1.5 gpm rather than 2.0 gpm or use the correct values in calculating claimed savings. OG&E Response: This recommendation was accepted. OG&E does not install any low flow devices with a flow rate higher than 1.5 gpm. Commercial Lighting Program Recommendation1: Consistently collect information on existing and new equipment for Commercial Lighting projects and supporting documentation for projects claimed as early replacement. OG&E Response: This recommendation was accepted. Information on new equipment was collected consistently; however, documentation regarding existing equipment was not available for all projects. Applied Energy Group, Inc. 70

290 APSC FILED Time: 5/2/2016 2:38:38 Recommendations PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Recommendation2: Modify the system to ensure transfer of all the values recorded in the online rebate worksheet to the tracking database, specifically: building segment, space type, and annual operating hours; and baseline and installed fixture types, counts, and wattages. OG&E Response: This recommendation was accepted. However, OG&E will have a new database in 2016 and did not consider it necessary to add additional expense to modify the existing transfers from the primary to the secondary database when the primary houses all the information. Current processes will be phased out when converting to the new database which will track all the required information. Recommendation3: Modify the system to match baseline fixture types to the Arkansas TRM Standard Wattage Table (Appendix E of the TRM). Consistently use fixture wattages that include the ballast (instead of just lamp wattages). Another approach would be to expand the fixture type fields in the worksheet to be more precise: e.g. a 400 HID to High Bay 6L-8L T8 or 4L-6L T5 should specify the number (4, 6 or 8) and type of lamp (T5 or T8) installed. If a T12 fixture is replaced, there should be more clarity on the lamp count, length, and ballast type of the baseline fixture. OG&E Response: This recommendation is not applicable. OG&E s Direct Options online worksheet was dismantled in 2015 and CLEAResult is now implementing the commercial lighting program. In 2015 Clearesult began the process of managing the entire program, including payment of rebates to consumers. The recommended action was implemented in the spreadsheet used to calculate rebates. Recommendation4: Record specifications and model numbers of installed fixtures in the database and include the information in the attached project documentation. OG&E Response: This recommendation was accepted. As discussed above, the online worksheet is being dismantled and CLEAResult will implement the program. Invoices with the type of lamp are required for each application to ensure model numbers and types of lamps are correctly used in the program. Recommendation5: Ensure that participants provide itemized invoices for the whole project. The program manager could check a sample of 10% of projects against invoices and include a note about this on the program application. OG&E Response: This recommendation was accepted. The online worksheet is being dismantled and CLEAResult will implement the program and is collecting itemized invoices for all projects. Recommendation6: Create a procedure to check the space type for any project with a space type categorized as Minimum or Low Temperature. OG&E Response: This recommendation was accepted. In PY2015, AEG did find issues in the online rebate worksheet with the categorization of space types but not this specific issue. Recommendation7: Modify the database to check annual operating hours against the hours deemed in the TRM. If the hours do not match, OG&E should require participants to substantiate. OG&E Response: This recommendation was accepted. As discussed above CLEAResult is implementing the program and both OG&E and the implementer check with the participant to ensure proper operating hours are being applied. Standard Offer Program Recommendation1: Document the building segment and HVAC system type in the program tracking database for each CFL measure entry. Document the building segment for all other measures in the program tracking database. Document the installed flow rate of the low-flow showerheads in the program tracking database. OG&E Response: This recommendation was accepted. CLEAResult is now implementing the SOP and is aware of this issue. While the PY2015 tracking database did document the Applied Energy Group, Inc. 71

291 APSC FILED Time: 5/2/2016 2:38:38 Recommendations PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 nominal flow rate of the installed low-flow showerheads, it did not document the building segments for measures. This resulted in the evaluator having to back-calculate building segment types based on deemed savings for any building segment-dependent measures in the program. Recommendation2: Use regional average embedded energy and demand intensities to take indirect energy and peak demand impacts for measures with water savings into account. OG&E Response: This recommendation was accepted. However, embedded energy and demand intensities were not taken into account in the PY2015 reported impacts for Direct Install faucet aerators, pre-rinse spray valves, or low-flow showerheads. Recommendation3: Require the participants to submit documentation (e.g. manufacturer s technical specification sheets, photographs of nameplate, itemized invoices with equipment details, etc.) to substantiate the new and, as far as possible, replaced equipment efficiencies and age of the replaced equipment. OG&E Response: This recommendation was accepted. CLEARresult is implementing the SOP program in In all cases the nameplates, photos, and any cut sheets that are available are being captured and entered into the information provided to the evaluator. In a number of the cases the old nameplate is missing, painted over, or ineligible to read. Consumer is asked for age of existing equipment being upgraded. OG&E and the implementer are working on a checklist to provide proof that the equipment is operational or repairable as well as the estimated age if name plate is missing. Recommendation4: Provide evidence for program influence and equipment decommissioning when early retirement is claimed. The TRM does not provide guidance for equipment older than 23 years and, based on the lifetime curve, this equipment should not be functional. If early retirement is claimed for such equipment, the project documentation should include information on the existing operating conditions and maintenance schedule that shows the equipment would still be operating for another year if it was not replaced. OG&E Response: This recommendation was accepted. OG&E makes every effort to ensure the equipment is in working order (or is repairable) before considering rebates on the new equipment being installed. The implementer is developing a checklist to collect more information on unit. No information regarding program influence or equipment decommissioning was provided in PY2015. Recommendation5: Update the measure calculator to accommodate units older than 23 years and ensure that equipment age matches with the Remaining Useful Life value. OG&E Response: This recommendation was accepted. However, AEG did find projects in PY2015 that claimed an Early Retirement baseline when the existing equipment age was greater than the maximum vintage allowed by the Arkansas TRM (i.e., there was no Remaining Useful Life for that particular vintage). Recommendation6: Modify program procedures to follow up when hours of use for motors are claimed as 8,760 by either requiring documentation to substantiate the motor annual operation hours, or using the hours specified in the TRM corresponding to the given facility type. OG&E Response: This recommendation was accepted. Some of the participating customers have plants that do not shut down with motors running 24/7, 365 days a year. As needed OG&E will get documentation from participants to substantiate the operation of the motors on a continuous basis. Applied Energy Group, Inc. 72

292 SECTION 8 Comprehensive Factors This section describes AEG s assessment of how effectively OG&E Arkansas programs (Multi-Family Direct Install, Student Energy Education LivingWise, Commercial Lighting, and Standard Offer) have addressed the following comprehensive factors: Factor 1: Whether the programs and/or portfolio provide, either directly or through identification and coordination, the education, training, marketing, or outreach needed to address market barriers to the adoption of cost-effective energy efficiency measures; OG&E s original portfolio contained some programs that met this need well and some that did not. In 2014 the utility dropped programs that struggled to address market barriers and increased the customer incentive in other programs to address barriers in the Commercial and Industrial class. The addition of the Multi-Family Direct Install program in late 2014 directly addresses the hard to reach customer barrier for residential customers. In 2015, OG&E engaged a new implementer for both the Commercial Lighting and Standard Offer program which enhanced the education, training, marketing and outreach for these programs resulting in much higher reported savings for both programs in PY2015. Factor 2: Whether the programs and/or portfolio, have adequate budgetary, management, and program delivery resources to plan, design, implement, oversee and evaluate energy efficiency programs; OG&E s new portfolio in 2014 increased budgets to address specifically this factor, including allocating funds for program design and third-party implementation for the multi-family program and commercial and industrial programs. OG&E also increased the evaluation budget for these programs. Factor 3: Whether the programs and/or portfolio, reasonably address all major end-uses of electricity or natural gas, or electricity and natural gas, as appropriate; OG&E s portfolio consists of programs designed to be cost effective and reasonably address all major end-uses of electricity. The SOP program was designed with flexibility to address changing commercial needs as the market may dictate; illustrated by the addition of more complex projects in PY2015. The programs reasonably address end uses for the residential, commercial and industrial sectors as shown below. In the new planning cycle OG&E will include new cost-effective measures identified by the statewide potential study which should improve compliance. Table 8-1 End Uses Addressed by OG&E Arkansas Portfolio of Programs OG&E Program Residential End Uses Commercial End Uses Industrial End Uses Multi-Family Direct Install SEE LivingWise Water Heating, Lighting, HVAC Water Heating, Lighting Commercial Lighting Lighting Lighting Standard Offer HVAC, Cooking, Refrigeration, Water Heating, Motors, Air Compressors, Process, Lighting Refrigeration, Motors, Air Compressors, Pumps, Process Applied Energy Group, Inc. 73

293 APSC FILED Time: 5/2/2016 Comprehensive 2:38:38 PM: Recvd Factors 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Factor 4: Whether the programs and/or portfolio, to the maximum extent reasonable, comprehensively address the needs of customers at one time, in order to avoid cream-skimming and lost opportunities; OG&E s portfolio addresses the need to address customer needs by providing several options for measures to the Multi-Family participant (e.g. water heating measures and HVAC duct sealing measures). Both the Commercial Lighting and Standard Offer programs are specifically designed to ensure that participant needs are addressed one customer at a time. A specific example of avoiding cream-skimming for residential customers is the new Multi-Family Direct Install program which addresses a hard to reach underserved customer segment. Factor 5: Whether such programs take advantage of opportunities to address the comprehensive needs of targeted customer sectors (for example, schools, large retail stores, agricultural users, or restaurants) or to leverage non-utility program resources (for example, state or federal tax incentive, rebate, or lending programs); Both the Standard Offer and Commercial Lighting programs target a variety of sectors and with the addition of CLEAResult as the implementer in 2015, compliance with this factor has improved. The direct installation of measures in C&I facilities by CLEAResult also focuses on specific sectors such schools, retails stores and restaurants to implement measures such as vending misers and faucet aerators. CLEAResult also implements programs specifically for schools and small businesses. OG&E s portfolio of programs was designed with enough flexibility to address the needs of the different sectors of customers. Aligning the customer incentive (rebates) with energy savings and has made the C&I programs available to even more customer sectors. Factor 6: Whether the programs and/or portfolio enables the delivery of all achievable, cost-effective energy efficiency within a reasonable period of time and maximizes net benefits to customers and to the utility system; AEG is not responsible for cost-effectiveness analysis. Factor 7: Whether the programs and/or portfolio, have evaluation, measurement, and verification "EM&V") procedures adequate to support program management and improvement, calculation of energy, demand and revenue impacts, and resource planning decisions. The EM&V procedures are continuing to work quite well. The portfolio includes both resources and procedures for EM&V that adequately support management of the programs and programs have been improved based on recommendation(s) from evaluators. During the program year, OG&E solicits advice and guidance from its evaluator to ensure compliance with the TRM and that reported savings, especially for the standard offer projects are estimated using industry-standard approaches. Applied Energy Group, Inc. 74

294 APPENDIX A Program Manager and Implementer Interview Guide Contact Name (s) Interviewer Interview Date Phone Background 1. Please give me a brief description of the program. 2. Please provide a brief description of the programs you are directly responsible for? 3. Please describe your job responsibilities related to the program. 4. How long have you been responsible for administration of this program? 5. On average, what percent of your time is spent on the program you are directly responsible for? 6. What other staff work on the program? What is their role regarding the program (i.e. what are the responsible for with regard to the program)? Who, if anyone, provides you with support? What support do they provide? 7. What do you hope to learn from the evaluation? Next, I d like to discuss the program s goals. Program Goals 8. What are your program goals for 2015? 9. Are you on target to reach you goals? Why or why not? What are some of the issues that will keep the program from meeting its potential? 10. Are your goals expected to change in 2016 how? Applied Energy Group, Inc. A-1

295 APSC FILED Time: 5/2/2016 Comprehensive 2:38:38 PM: Recvd Factors 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Program Implementation and Marketing 11. Do you work with an implementation contractor or trade allies to help deliver the program? Who is/are the implementation contractor and/or trade allies you work with on this program? Do you have any issues or concerns about the implementation contractor or trade allies? If yes, what are your concerns? 12. What is the target market for the program? How are potential customers identified? 13. What are the main barriers to participation? How does the program address these barriers? 14. What has been done to market the program? How successful have these strategies been? Are some marketing strategies or messages more beneficial than others? How is success of the marketing strategies or messages measured? Enrollment Process 15. Once the customer is informed about the program what is the enrollment process? 16. Have you heard any feedback either positive or negative from customers about the enrollment process? 17. Have you made any changes to the program in the last year? If so, why? 18. How do you coordinate the efforts of this program with other programs OG&E offers? Program Operations 19. What are the participation steps from the customers perspective? Have these changed over time? If so, why were they changed? 20. Are there any specific aspects of the program that are working very well? Any not working well? (Probe for details) What could be done to improve? 21. What quality control/quality assurance procedures are in place? Are these documented? Is any verification done? What does this entail? Who does the verification? 22. Do the incentive levels seem appropriate? If not, why not? What, if any, changes in the incentive levels do you think may be needed? 23. What is your opinion of free ridership for this program? Meaning do you think customers would pay for the measures on their own, outside of the program. Why do you say that? 24. Do you think the program is changing customers' energy efficiency attitudes and actions? (Probe for specifics) 25. Are customers satisfied with the program overall? With the measures installed? Are they satisfied with the incentive amount? With the savings achieved? How is satisfaction with the program measured? (i.e. is this based on survey information or is it anecdotal?) Applied Energy Group, Inc. A-2

296 APSC FILED Time: 5/2/2016 2:38:38 Comprehensive PM: Recvd 5/2/2016 Factors 2:33:07 PM: Docket TF-Doc. 298 Lessons Learned/Future plans 26. Are there any changes planned for the program in the next couple of years? 27. Are there changes you would like to implement but cannot? What are the obstacles to implementing those changes? 28. How do you see the program evolving in the next 3-5 years? Thank you again for taking the time to discuss this program. Applied Energy Group, Inc. A-3

297

298 APPENDIX B Multi-Family Property Manager Interview Guide Company Contact Name Interviewer Interview Date Phone Respondent Background If not already provided via ; introduce yourself, AEG, and explain that we are evaluating OG&E s Multifamily Efficiency Program in Arkansas and want to talk to property managers who participated in the program in Verify that we are talking with the appropriate person - We understand your company participated in OG&E s Multifamily Efficiency Program in Arkansas in The program involved a contractor installing several energy efficiency products in your buildings, including low flow showerheads, faucet aerators, power strips and CFLs. Are you the best person to speak to about participation in the program? - If not, ask for contact information for the best person. - If yes, ask if this is a good time or should reschedule say how long it will take (about minutes) Thank you for taking the time to talk with me today about OG&E s Multifamily Efficiency Program. The goal of this discussion is to talk about your experience with the program, find out what you think works in the program and any suggestions you have to improve it. All comments will remain confidential and we ll only provide OG&E the results combined across all participants we interview. First, I d like to get a little better understanding of your company s background and your role there. 1. Please briefly describe your company services, customers, size, time in business, geographical reach, owner/manager, etc. 2. How many multifamily buildings does your company own/manage in Arkansas? 3. What is your current title? What are your responsibilities? Applied Energy Group, Inc. B-1

299 APSC FILED Time: 5/2/2016 Comprehensive 2:38:38 PM: Recvd Factors 5/2/2016 2:33:07 PM: Docket TF-Doc Was this your company s first experience with OG&E s energy efficiency programs? If no, what other Arkansas programs has your company participated in? Has your company participated in programs in other jurisdictions? Next, I d like to discuss your experience with the Multifamily Efficiency Program Experience with Program/Satisfaction 5. How did you or your company hear about the program? 6. What motivated your company to participate? 7. Once you became aware of the program how easy or difficult was the enrollment process? 8. What is your overall satisfaction with the program? What aspects do you specifically value about the program? What do you dislike? 9. What are the benefits (if any) to your company for participating in the program? 10. How satisfied are you with the contractor who installed the equipment in your buildings? Did you have any problems? What worked well with the contractors? 11. Would you participate in the program again? Measures Installed 12. Have you installed low flow showerheads, CFLs, power strips or faucet aerators in your units in the past? If no, why not? Did you have duct sealing or air infiltration measures in any of your buildings? If so have you had experience with those types of improvements in the past? (Probe for cost of equipment, not aware of measure/benefit, time associated with doing this on their own, split incentive) 13. Were there any problems/challenges associated with the OG&E program involving the installation of the equipment? How were those problems/challenges overcome? 14. Are you satisfied with the equipment? Why or why not? Have you received any feedback from tenants regarding the equipment and/or the program? 15. Do you know if any of the equipment installed through the program has been removed? 16. Is there any other energy-saving equipment you would like to see added to the program? Applied Energy Group, Inc. B-2

300 APSC FILED Time: 5/2/2016 2:38:38 Comprehensive PM: Recvd 5/2/2016 Factors 2:33:07 PM: Docket TF-Doc Would your organization consider installing similar energy efficiency improvements in the future in this or other facilities? a. [IF YES] Would you consider installing similar energy efficiency improvements if a rebate was not available? b. [IF NO] Why not? Program Effectiveness 18. How likely would you have been to install this equipment if the program was not available? 19. [IF LIKELY] Would you have installed the same amount of equipment? Would you have installed it at the same time? (Probe: Did the rebate enable you to purchase additional equipment? Did the rebate encourage you to make these improvements sooner than expected?) 20. Has your experience with the program influenced you to buy other high efficiency equipment, either on your own without incentives, or through another OG&E program with incentives? If yes, probe for details. 21. Do you have any suggestions to improve the program? 22. Do you have any other comments you would like to add about your experience with OG&E s Multifamily Efficiency program? Thank you again for taking the time to discuss this program. Applied Energy Group, Inc. B-3

301

302 APPENDIX C Documentation Review Results Commercial Lighting Program The following sections provide a project-by-project summary of the reviewed measures administered by Direct Options in the PY2015 sample. Qualifications for eight fixture types were verified out of the 12 installed fixture types in the sample that were required by the AR TRM to be qualified by the Design Lights Consortium, Energy Star, or the Consortium for Energy Efficiency. Stratum 1 Rebate ID x1397 Fixture ID Stratum 1 Sample 1 T12 to T8 This project involved a T12 to T8 retrofit at an outpatient healthcare facility. The fixture retrofit was clearly marked in the Fixture Name field of the tracking database with ba seline and installed fixture codes from the Standard Wattage Table (SWT) provided by OG&E. Installed fixture counts were confirmed with provided invoices, and the installed fixture wattage was increased from the reported 108W to 111W based on OEM specifications; the T8 ballast was found to be qualified by the Consortium for Energy Efficiency (CEE), as required by the AR TRM. 58 Baseline fixture wattages and counts were accepted as reported based on the F44EE fixture code. Rebate ID x1439 Fixture ID Stratum 1 Sample 2 HID to T5 This project involved retrofitting high-bay high intensity discharge (HID) fixtures to 6-lamp T5 fixtures in a service facility. The installed fixture wattage (364W) was verified with OEM specifications to be higher than the originally reported fixture wattage (323W); this resulted in lower savings. The AR TRM does not require qualification for T5 fixtures, and the CEE does not qualify T5 fixtures. Fixture counts were confirmed with invoices. Baseline fixture wattages and counts were accepted as reported based on the MH400/1 fixture code. Rebate ID x0219 Fixture ID 3832 Stratum 1 Sample 3 T12 to T8 This project involved a T12 to T8 retrofit in a stand-alone retail facility. No invoices or specifications were provided for this project, but the wattages for this 4-lamp T12 to 4-lamp T8 retrofit were verified with the SWT. CEE fixture qualification could not be verified. No changes in fixture wattages and counts were made from the reported values. Rebate ID x1408 Fixture ID Stratum 1 Sample 4 Incandescent to LED This project involved an incandescent to LED retrofit at a coffee shop. AEG confirmed the wattage and count of the installed screw-in LEDs based on OEM specifications and provided invoices. The installed LEDs were found to be qualified in the Energy Star Certified Light Bulbs Product List (also referred to as a Qualified Product List QPL). 59 Baseline lamp wattages and counts were accepted as reported. In order to calculate lifetime savings, the applicable Tier 1 EISA baseline of 53W was used after the first year and a Tier 2 EISA baseline of 28W was used for the last year of the measure life. 60 Rebate ID x1371 Fixture ID Stratum 1 Sample 5 T12 to T5 58 Consortium for Energy Efficiency. Commercial Lighting Qualifying Product Lists. January W & 25W Lamps & Ballasts.xls 59 Accessed at: 60 Energy Independence and Security Act (EISA) of Applied Energy Group, Inc. C-1

303 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 This project involved a T12 to T5 retrofit in a service location. The wattage for the installed T5 fixture was confirmed with OEM specifications and lowered to 58W from the originally reported 61W. The AR TRM does not require qualification for T5 fixtures, and the CEE does not qualify T5 fixtures. No evidence was provided for the T12 baseline, though the most appropriate fixture code (F44EE/D4) from the SWT matched the recorded baseline wattage. Counts were confirmed with invoices. Stratum 2 Rebate ID x1395 Fixture ID Stratum 2 Sample 6 T8 to LED This project involved a T8 to LED retrofit at a stand-alone retail location. The fixture retrofit was clearly marked in the Fixture Name field of the tracking database with baseline and installed fixture codes from the SWT. The count and wattage for the installed LED fixtures were confirmed via invoices and OEM specifications. The LED fixture was confirmed to be qualified in the Design Lights Consortium (DLC) Qualified Products List (QPL). 61 Though the baseline appeared to be an 8-ft T8 fixture based on notes in the invoice, the baseline fixture code of F44ILL was accepted as reported in the tracking database. Rebate ID x1436 Fixture ID Stratum 2 Sample 7 T8 to T8 This project involved the relamping of (63) 4-foot 4-lamp F32T8 fixtures with two (2) F28T8 lamps per fixture. The resulting proposed wattage was verified with provided OEM specifications for the ballast and the fixture counts were verified with invoices. The installed T8 lamps were qualified by the CEE T8 Replacement Lamp Qualifying Product List. 62 Baseline fixture wattages and counts were accepted as reported based on the F44ILL fixture code. Rebate ID x1395 Fixture ID Stratum 2 Sample 8 T8 to LED This project involved a T8 to LED retrofit at a stand-alone retail location. The fixture retrofit was clearly marked in the Fixture Name field of the tracking database with baseline and installed fixture codes from the SWT. The wattages and counts for the installed fixtures were verified and changed via invoices and OEM specifications; while the original database entry reported (121) 114W LED fixtures, the invoice and specifications indicated (242) 58W LED fixtures. This adjustment resulted in a minimal reduction in savings from the replicated values. The installed LED fixture was confirmed to be qualified in the DLC QPL. Rebate ID x1408 Fixture ID Stratum 2 Sample 9 Incandescent to LED This project involved an incandescent to LED retrofit at a coffee shop. AEG confirmed the wattage and count of the installed screw-in LEDs based on OEM specifications and provided invoices. The installed LEDs were confirmed to be qualified in the Energy Star QPL. Baseline lamp wattages and counts were accepted as reported. In order to calculate lifetime savings, the applicable Tier 1 EISA baseline of 53W was used after the first year and a Tier 2 EISA baseline of 28W was used for the last year of the measure life. Rebate ID x1427 Fixture ID Stratum 2 Sample 10 T12 to LED This project involved a T12 to LED retrofit at a fast food restaurant. The fixture information provided in the invoice was not indicative of the total wattage per installed fixture. However, since the assumed baseline fixture (F44EIS) has four lamps and the replacement LED is an 18W T8 tube, it was reasonable to accept the reported 72W as the installed fixture wattage for four (4) 18W LED tubes. It was also confirmed that the 18W LED tube fixtures exist with a 72W option according to online specifications from foreign companies (the installed fixtures were imported directly from China). The installed LED fixture was not found to be qualified by any approved entity. In order to calculate lifetime savings, the T12 baseline was used for the first 8. 5 years of the measure life and an assumed 118W T8 replacement (F44LL fixture code) was used as the baseline for the remainder of the 15-year LED measure life. 61 Accessed at: 62 Consortium for Energy Efficiency. Commercial Lighting Qualifying Product Lists. CEE T8 Lamps November_2015_Posted.xls Applied Energy Group, Inc. C-2

304 Stratum 3 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Recvd Review 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 Rebate ID x0312 Fixture ID 4038 Stratum 3 Sample 11 T12 to T5 This project involved a T12 to T5 retrofit in a warehouse/manufacturing facility. No documentation was provided to substantiate the baseline or installed fixture types. Though there was one nameplate photo of the existing T12 fixture, it was illegible. Baseline and installed count and wattage values were accepted as replicated during the replication analysis. The AR TRM does not require qualification for T5 fixtures, and the CEE does not qualify T5 fixtures. Based on the photos provided (uninsulated walls, openings in garage doors, lack of equipment/processes that need refrigerated storage), this was not a refrigerated warehouse; this manufacturing facility is not expected to have a refrigerated space. The space type was changed to AC Space Normal Temps Unknown Heat. Rebate ID x0283 Fixture ID 3995 Stratum 3 Sample 12 HID to T5 This project involved an HID to T5 retrofit in an unconditioned manufacturing facility. Installed fixture counts were verified with invoices; installed fixture wattages were verified with OEM specifications (wattage slightly reduced from standard 234W to 229W). The AR TRM does not require qualification for T5 fixtures, and the CEE does not qualify T5 fixtures. The baseline could not be confirmed based on provided information, but 400W metal halide standard fixture wattages were used. Rebate ID x0294 Fixture ID 4006 Stratum 3 Sample 13 HID to LED This project involved an HID to LED retrofit in a manufacturing facility. Most of the entries and all fixture descriptions in the provided invoices for this project were a TYPING ERROR. Therefore, installed fixture types and wattage could not be verified. The space temperature range for this Fixture ID had been adjusted from AC Space Minimum Temps (freezer space) to AC Space Medium Temps (refrigerated space) during the replication review. The presence of a refrigerated space was confirmed via photographs for this poultry processing facility, though it is most likely medium temperatures as opposed to minimum temperatures. Furthermore, the OG&E Online Worksheet listed AC Space Minimum Temps as having a temperature range of 33 to 41 F. This is actually the temperature range for the AC Space Medium Temps, not the Minimum Temps space type (range of -10 to 10 F) in the AR TRM. AEG changed the temperature range (and corresponding IEF values) for this Fixture ID to a Medium Temp space. Rebate ID x1436 Fixture ID Stratum 3 Sample 14 T8 to LED This project involved a T8 to LED retrofit at a stand-alone retail location. Installed fixture counts were verified with provided invoices; the installed fixture wattage was slightly increased based on OEM specifications. The installed LED fixtures were confirmed to be qualified by the DLC QPL. Baseline fixture wattages and counts were accepted as reported based on the F43ILL fixture code. Rebate ID x1320 Fixture ID 4051 Stratum 3 Sample 15 HID to T5HO This project involved a retrofit from 1000W metal halide (MH) to 10-lamp T5HO fixtures in a manufacturing facility. No specifications were provided or procurable to verify the installed fixture wattage of 555W, but it was deemed reasonable based on SWT. The number of fixtures was confirmed via the project proposal, though final invoices were not submitted with the project. Rebate ID x1372 Fixture ID Stratum 3 Sample 16 HID to LED This project involved a metal halide high bay to LED retrofit in an office building. The baseline 400W metal halide fixtures were confirmed based on nameplate photos in the documentation. The provided invoice did not provide fixture counts or model numbers of installed fixtures; these were accepted as reported in the tracking database. Rebate ID x1323 Fixture ID 4061 Stratum 3 Sample 17 HID to LED This project involved retrofitting (83) 320 MH fixtures to (83) 146W LED fixtures in a manufacturing facility. Counts and wattages were confirmed with the provided invoice and OEM specifications for the installed fixture. The installed LED fixture was confirmed to be qualified by Applied Energy Group, Inc. C-3

305 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 the DLC QPL. Reported values were confirmed; it is unclear why the energy savings and demand reductions from the documentation review do not coincide with reported values. Rebate ID x1365 Fixture ID 9624 Stratum 3 Sample 18 T12 to T8 This project involved a T12 to T8 retrofit in a service (non-food) facility. The wattages for this T12 to T8 retrofit were confirmed with the SWT based on descriptions provided in the documentation. No specifications were provided for the installed fixtures, so they could not be verified with the CEE. Fixture counts in the provided invoice were large enough to cover the reported counts for the installed fixtures. Stratum 4 Rebate ID x1320 Fixture ID 4050 Stratum 4 Sample 19 HID to T5 This project involved a retrofit of 400W MH fixtures to 4-lamp T5HO fixtures in a manufacturing facility. Since no specifications were provided or were procurable to verify the reported fixture wattage of 216W, AEG changed the fixture wattage to 230W based on SWT (fixture code F44GPHL-H). The AR TRM does not require qualification for T5 fixtures, and the CEE does not qualify T5 fixtures. The number of fixtures was confirmed via the project proposal, though final invoices were not submitted with the project. Stratum 5 Occupancy Sensors Rebate ID x0222 Fixture ID 3838 Stratum 5 Sample 20 This project involved installing occupancy sensors to control newly installed LED fixtures at an elementary school. Wattages for the controlled LED fixtures were verified with OEM specifications. Invoices were used to verify sensor counts and that there were enough fixtures installed on the invoice to facilitate the control of 344 LED fixtures via 86 wall/ceiling occupancy sensors. It is unclear why the verified demand reduction differs so greatly from the reported demand reduction. Rebate ID x1320 Fixture ID 4052 Stratum 5 Sample 21 This project involved an occupancy sensor application on (335) of the (813) 4-lamp T5HO installed in Fixture ID 4050 of the same rebate ID (Stratum 4 Sample 19). As such, the controlled wattage was changed from 218W to 230W based on the SWT (fixture code F44GPHL- H) for consistency. No information on occupancy sensor application was provided in the project proposal. Final invoices were not submitted with the project. Rebate ID x1348 Fixture ID 9131 Stratum 5 Sample 22 This occupancy sensor project was completed in a non-refrigerated warehouse. AEG confirmed the reported wattage of the 4-lamp F54T5/HO fixtures, even though this wattage was increased during the replication review based on the SWT. AEG adjusted the verified wattage back to the reported wattage. Rebate ID x1395 Fixture ID Stratum 5 Sample 23 This project involved installing occupancy sensors to control a portion of the LED fixtures installed at a stand-alone retail facility in Fixture ID (this fixture was not in the sample). Though the measure was reported in the tracking database as (7) sensors controlling (4) 76W fixtures per sensor, AEG adjusted the values to reflect the actual installed LED fixtures. Thus, (7) sensors were calculated to control (8) 38W fixtures per sensor. The resulting savings were identical to reported values. Rebate ID x0221 Fixture ID Stratum 5 Sample 24 This project involved installing occupancy sensors to control newly installed LED fixtures at an elementary school. Wattages of LED fixture controlled under Fixture ID were confirmed with OEM specifications. Based on the invoice, there were more fixtures installed (108) than controlled (80). Forty (40) ceiling sensors were confirmed with invoices. Rebate ID x0221 Fixture ID Stratum 5 Sample 25 This project involved installing occupancy sensors to control newly installed LED fixtures at an elementary school. Wattages of LED fixture for Fixture ID were verified with OEM Applied Energy Group, Inc. C-4

306 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Recvd Review 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 specifications (wattage slightly reduced from 46 to 45. 5W). The count of LED fixtures and count of sensors were confirmed with provided invoices. Sensors were found to be integrated in the installed LED fixtures; based on the specifications these sensors had daylighting and continuous dimming capabilities. It is unclear why the verified demand reduction differs so greatly from the reported demand reduction. Stratum 6 Delamping Measures Rebate ID x0283 Fixture ID 3996 Stratum 6 Sample 26 This project involved delamping T12 fixtures in a manufacturing facility. There was no mention of delamping fourteen (14) T12 fixtures in the project invoices. AEG had matched the stated baseline wattage (123W) to an F82EE fixture code in the SWT during the replication review; no information was found to challenge any other variables for this Fixture ID. Since the baseline fixture was a T12 fixture, the assumed replacement fixture for the calculation of lifetime savings was a T8 replacement with an F82ILL fixture code (110W). Rebate ID x0312 Fixture ID 4039 Stratum 6 Sample 27 This project involved delamping HID fixtures in a manufacturing facility. Photographic evidence pointed to the delamping of HIDs, but the lighting nameplate was illegible. No other documentation outside of photographs was provided; values were accepted as reported. Based on the photos provided (uninsulated walls, openings in garage doors, lack of equipment/processes that need refrigerated storage), this facility was not a refrigerated warehouse; this manufacturing facility is not expected to have a refrigerated space. The space type was changed to AC Space Normal Temps Unknown Heat. Rebate ID x1374 Fixture ID Stratum 6 Sample 28 This project involved the delamping of (18) exterior T12 fixtures in a gas station/convenience store. No evidence was provided for the count or model of delamped fixtures; these were accepted as reported in the tracking database. Likely replacement fixtures for the T12 fixtures were assumed to be a fixture code of F42LHL with high output T8 lamps for calculation of lifetime savings. Rebate ID x1407 Fixture ID Stratum 6 Sample 29 This project involved delamping one (1) exterior metal halide fixture as part of a larger retrofit at an auto body shop. No evidence was provided for the count or model of the delamped fixture; these were accepted as reported in the tracking database. The likely replacement fixture for the 400W MH fixture was assumed to be a 156W LED for lifetime savings calculation purposes. 63 Stratum 7 New Construction Rebate ID x1328 Fixture ID 4070 Stratum 7 Sample 30 This was a new construction project in a fast food restaurant; savings originally could not be replicated based on data provided in the tracking database but were later replicated using values from the documentation. The allowed wattage was verified based on the Building Area Method Lighting Power Densities (LPD) in Table F1 of the TRM. The proposed wattage was verified based on the ComCheck web-based software screenshot provided in the documentation; no invoice or OEM specifications were provided. Since there were different types of fixtures (linear fluorescent, CFL, and LED) installed at the site, a weighted average lifetime of 9. 7 years based on fixture counts was determined in order to calculate the lifetime savings. 63 A delamped 400 W metal halide was assumed to be replaced by a 156 W LED equivalent based on the schotopic/photopic multiplier method from Howard Lighting Products, HID to LED Wattage Cross Reference Table. Applied Energy Group, Inc. C-5

307 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 C&I Standard Offer Program The following sections provide a project-by-project summary of the documentation review conducted for OG&E s Standard Offer Program in Arkansas for PY2015. This review consisted of 20 sampled Rebate IDs involving motor, chiller, HVAC rooftop unit replacements, refrigeration unit retrofits, and computer power management, as well as custom projects with measures not found in the Arkansas TRM 5.0. C&I Direct Install projects were not included in the documentation review step. In conducting the review of each project, AEG applied the following procedure: 1. Reviewed all project information documented in OG&E s Direct Options online portal or provided by the OG&E Program Manager. 2. Verified the inputs used to determine energy and demand savings documented in the various project documentation files. 3. Conducted research for additional information to verify and confirm key parameters in the savings calculations in cases where there was not adequate documentation to substantiate the reported information. 4. For measures found in Arkansas TRM 5.0, reproduced energy and demand impact calculations according to the algorithms specified in the TRM using the verified inputs. 5. For measures not found in TRM 5.0, reproduced energy and demand impact calculations according to the implementer methodology and best practice engineering judgment using verified inputs. The following tables are a summary of energy and demand impacts and realization rates fo r all sampled projects reviewed. Table C-1 Stratum Sample # C&I Standard Offer Documentation Review Results: Gross Energy Savings Rebate ID # Reported Energy Savings (kwh/yr) Verified First Year Energy Savings (kwh/yr) Realization Rate (kwh) Lifetime Energy Savings (kwh) ,329 1, % 4, , % 4, OPP % ,122 12, % 112, ,883 26,897 66% 403, OPP ,201 23,842 38% 95, ,947 9,516 14% 190, OPP ,154 32,572 72% 130, OPP ,895 43,578 36% 174, OPP ,335 41,031 49% 164, OPP ,573 44,745 41% 178, , , % 2,747, OPP ,963 46,398 63% 185, OPP , ,861 44% 2,082, , , % 9,229,735 Applied Energy Group, Inc. C-6

308 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Recvd Review 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 Stratum Sample # Rebate ID # Reported Energy Savings (kwh/yr) Verified First Year Energy Savings (kwh/yr) Realization Rate (kwh) Lifetime Energy Savings (kwh) 4 16 OPP , , % 6,002, OPP , , % 9,628, OPP , , % 12,655, OPP ,346,844 1,346, % 13,468, OPP ,417,303 1,417, % 5,669,212 Table D-2 Stratum Sample # C&I Standard Offer Documentation Review Results: Gross Peak Demand Reduction Rebate ID # Reported Demand Reduction (kw) Verified First Year Demand Reduction (kw) Realization Rate (kw) Weighted Average Lifetime Demand Reduction (kw) % % OPP % % % OPP % % OPP % OPP % OPP % OPP % % OPP % OPP % % OPP % OPP % OPP % OPP % OPP % Applied Energy Group, Inc. C-7

309 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 Stratum 1 Sample 1 Rebate # Project Description: This project consisted of the replacement of one split-system air conditioner (2.5 tons, providing space cooling only) with a new, higher efficiency unit. The retrofit was one-for-one and like-for-like, meaning that the type, quantity, and capacity of equipment in the pre- and post-retrofit cases were the same. Findings: Key Issue: The project documentation was posted in the Direct Options database. This project was specified as Early Retirement, and OG&E reported the impacts during first year of the remaining useful life (RUL) of the original (pre-retrofit) equipment. No documentation was provided to confirm that the original equipment was in working condition or repairable when it was replaced. The project documentation included a copy of the contractor s invoice for installation of the new unit, and photographs of the new and original units and nameplates. There was no specification sheet for the new unit. However, AEG was able to verify the capacity and efficiency of the new unit using the AHRI Directory of Certified Product Performance using the reported model number. Information posted in the Direct Options database indicated that the original unit was manufactured in 2000 with unknown efficiency. However, the reported serial number of the original unit indicated that it was manufactured in AEG used 2001 as the year of manufacture when determining the baseline efficiency of the original unit in the Early Retirement savings calculations (Tables 250 and 251 of TRM 5.0). The measure included in this project has an EUL of 15 years and a RUL of 3.3 years (per Table 254 of TRM 5.0) for the purpose of lifetime impact determination. The new split-system air conditioner has an efficiency of 13.0 SEER, which is exactly equal to the federal minimum efficiency requirements. As such, this project has no savings during the period beyond the end of the RUL in the lifetime impact calculations. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-3 Documentation Review Summary for Rebate # Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 1, Verified First Year Impacts 1, Realization Rate 100% 100% Verified Lifetime Impacts 4, Based on information from the Building Intelligence Center s HVAC Production Date/Age webpage, accessed here: 65 This is the total verified demand reduction over the life of the measure divided by the EUL of the installed equipment (15 years). Since savings were only calculated for the RUL of 3.3, this weighted average lifetime demand reduction is much lower than the first year demand reduction. Applied Energy Group, Inc. C-8

310 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Recvd Review 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 Sample 2 Rebate # Project Description: This project consisted of the retrofit of one split-system air conditioner (3 tons, providing space cooling only) with a new, higher efficiency unit. The retrofit was one-forone and like-for-like, meaning that the type, quantity, and capacity of equipment in the pre- and post-retrofit cases were the same. Findings: Key Issue: The project documentation was posted in the Direct Options database. This project was specified as Early Retirement, and OG&E reported the impacts during first year of the RUL of the original (pre-retrofit) equipment. No documentation was provided to confirm that the original equipment was in working condition or repairable when it was replaced. The project documentation included photographs of the new unit and nameplate. A copy of the contractor s invoice was also posted, but the invoice appeared to be for a different project. No specification sheet was provided for the new unit. Information posted in the Direct Options database indicated that the new unit has a capacity of 4 tons and efficiency of 14 SEER. AEG was able to confirm the efficiency of the new unit using the AHRI Directory of Certified Product Performance with the reported model number. However, the capacity of the new unit was found to be 3 tons based on the AHRI database and the unit s model number nomenclature. Information posted in the Direct Options database indicated that the original unit was manufactured in 1990 (25 years old) with unknown efficiency. No other equipment identifiers, such as model and serial numbers, were provided for the original unit. The measure included in this project has an EUL of 15 years and RUL of zero (0) years (per Table 254 of TRM 5.0) for the purpose of lifetime impact determination. Per TRM 5.0, split-system air conditioning equipment that is 25 years old no longer has an RUL. As such, AEG treated this project as Replace on Burnout and recalculated the impacts. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-4 Documentation Review Summary for Rebate # Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 1, Verified First Year Impacts Realization Rate 17% 21% Verified Lifetime Impacts 4, Sample 3 Rebate # OPP Project Description: This project consisted of the retrofit of two motors (5 HP each) with new premium efficiency motors. The retrofits were one-for-one and like-for-like, meaning that the type, quantity, and capacity of the motors in the pre- and post-retrofit cases were the same. Findings: The project documentation was provided by CLEAResult. Applied Energy Group, Inc. C-9

311 Key Issue: APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 This project was specified as Early Retirement, and OG&E reported the impacts during first year of the RUL of the original (pre-retrofit) equipment. However, no documentation was provided to confirm that the original motors were in working condition or repairable when they were replaced. Furthermore, no information was provided in the project documentation regarding the age of the original motors. The project documentation included a copy of the contractor s invoice for installation of the new motors. There were no photographs of the new and original motors and nameplate or specification sheets for the new motors. However, AEG was able to verify the capacity and efficiency of the new motors by conducting a search of the reported model number (as stated on invoice) of the motors on the manufacturer s website (Baldor). The project documentation included a spreadsheet showing the reported impact calculations. In this spreadsheet, AEG confirmed that the baseline efficiency of the original motors and operating hours used in the Early Retirement impact calculations are consistent with the values specified in Tables 342 and 344 of TRM 5.0. The measure included in this project has an EUL of 15 years. Since no information regarding the age of the original motors was provided, AEG assumed that the original motors were 18 years old the maximum age allowed for Early Retirement (per Table 345 of TRM 5.0) for the purpose of lifetime impact determination. The new 5 HP motors have an efficiency of 89.5%, which is exactly equal to the federal minimum efficiency requirements for motors of these capacity and type. As such, this project has no savings during the period beyond the end of the RUL in the lifetime impact calculations. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-5 Documentation Review Summary for Rebate # OPP Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts Verified First Year Impacts Realization Rate 100% 100% Verified Lifetime Impacts Sample 4 Rebate # Project Description: This project consisted of the retrofit of two motors (7.5 HP and 50 HP) with new premium efficiency motors. The retrofit was one-for-one and like-for-like, meaning that the type, quantity, and capacity of the motors in the pre- and post-retrofit cases were the same. Findings: The project documentation was posted in the Direct Options database. This project was specified as Early Retirement, and OG&E reported the impacts during first year of the RUL of the original (pre-retrofit) equipment. No documentation was provided to confirm that the original motors were in working condition or repairable when they were replaced. The project documentation included a copy of the contractor s invoice for installation of the new motors. There were no photographs of the new and original motors and Applied Energy Group, Inc. C-10

312 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Recvd Review 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 nameplate or specification sheets for the new motors. However, AEG was able to verify the capacity and efficiency of the new motors by conducting a search of the reported model number (as stated on invoice) of the motors on the manufacturer s website (Baldor). Information posted in the Direct Options database reported that the original 7.5 HP and 50 HP motors were 5 and 7 years old and had efficiencies of 85% and 89.2%, respectively. The operating hours of the original and new motors were also reported to be 7,460 hours per year. No other documentation was provided to support this information. AEG accepted the reported operating hours of 7,460 hours per year, since TRM 5.0 allows for operating hours to be estimated using interviews with facility operators and review of operation logs (it is assumed that the operating hours were estimated using this methodology). The measure included in this project has an EUL of 15 years. The RUL of the original 7.5 HP and 50 HP motors was of 10.0 years and 8.2 years, respectively (per Table 345 of TRM 5.0) for the purpose of lifetime impact determination. Key Issues: Since no documentation was provided to support the reported values for the original motors age and efficiency, AEG was unable to make any adjustments to these inputs and was only able to replicate the impact calculations using these input values. The algorithm used by OG&E to calculate the motor energy savings in the Direct Options database was incorrect. The coincidence factor was erroneously included in the energy savings calculation, when it should only be used in the demand reduction calculation. The new 7.5 HP and 50 HP motors have efficiencies of 91.7% and 94.5%, respectively, which are exactly equal to the federal minimum efficiency requirements for motors of these capacity and type. As such, this project has no savings during the period beyond the end of the RUL in the lifetime impact calculations. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-6 Documentation Review Summary for Rebate # Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 9, Verified First Year Impacts 12, Realization Rate 135% 100% Verified Lifetime Impacts 112, Stratum 2 Sample 5 Rebate # Project Description: This project consisted of the retrofit of a total of 27 split-system and packaged air conditioners (2 to 7.5 tons) with new, higher efficiency units. The new units also provide space heating. The retrofit was one-for-one, meaning that the quantity and capacity of equipment in the pre- and post-retrofit cases were the same. Findings: The project documentation was posted in the Direct Options database. Applied Energy Group, Inc. C-11

313 Key Issue: APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 This project was specified as Early Retirement, and OG&E reported the impacts during first year of the RUL of the original (pre-retrofit) equipment. No documentation was provided to confirm that the original equipment was in working condition or repa irable when it was replaced. The project documentation included photographs of the new units nameplate, as well as an incomplete set of photographs of the original units and nameplates. The contractor s final invoice for installation of the new units was also included, as well as specification sheets for the new units. However, the included sheets were incomplete and did not actually show the specifications of the new units. AEG was able to confirm the capacity and efficiency of the new units by obtaining the complete specification sheets from the manufacturer s website and conducting a search of the AHRI Directory of Certified Product Performance using the reported model numbers (from the final invoice). The efficiencies of the new units were verified to be higher than the value (13 SEER) reported for each unit in the Direct Options database. Information posted in the Direct Options database indicated that the original units were manufactured in either 1990 (25 years old) or 1993 (22 years old) with unknown efficiencies (with the exception of one unit where the nameplate photograph shows an efficiency of 9 EER). However, the photographs of some of the nameplates of the original units indicated that the units were manufactured in 1996 (19 years old). The measure included in this project has an EUL of 15 years and RUL of zero (0) years (per Table 254 of TRM 5.0) for the purpose of lifetime impact determination. Per TRM 5.0, unitary and split-system air conditioning equipment that is 19 years or older no longer has an RUL. As such, AEG treated this project as Replace on Burnout and recalculated the impacts. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-7 Documentation Review Summary for Rebate # Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 40, Verified First Year Impacts 26, Realization Rate 66% 55% Verified Lifetime Impacts 403, Sample 6 Rebate # OPP Project Description: This project involved the installation of a total of 333 linear feet of refrigerator door gaskets and 109 square feet of strip curtains on walk-in refrigerators in a supermarket facility. Both measures have deemed values specified by TRM 5.0. Findings: The project documentation was provided by CLEAResult, and consisted of a list showing the details of the door gasket and strip curtain installations as well as photographs of the installed measures. AEG replicated the savings for each measure using the information provided in the project documentation and the deemed impacts specified in TRM 5.0, and found the following: Applied Energy Group, Inc. C-12

314 Key Issue: APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Recvd Review 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 o o For the door gasket measure, the implementer used deemed impacts (kw per linear feet, kwh per linear feet) that were different than the values specified in TRM 5.0. The kw per linear feet value used by the implementer was lower than the value from the TRM, while the opposite was true for the kwh per square feet value (higher than the value from the TRM). For the strip curtain measure, AEG s verified energy and demand impacts matched the reported impacts. The deemed impacts used to determine the reported impacts for the door gasket measure were not consistent with the values specified in TRM 5.0. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-8 Documentation Review Summary for Rebate # OPP Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 63, Verified First Year Impacts 23, Realization Rate 38% 122% Verified Lifetime Impacts 95, Sample 7 Rebate # Project Description: This project consisted of the retrofit of an air-cooled scroll chiller (30 tons) with a new, higher efficiency unit. The retrofit was one-for-one and like-for-like, meaning that the type, quantity, and capacity of the units in the pre- and post-retrofit cases were the same. Findings: Key Issue: The project documentation was posted in the Direct Options database. This project was specified as Early Retirement, and OG&E reported the impacts during first year of the RUL of the original (pre-retrofit) equipment. No documentation was provided to confirm that the original chiller was in working condition or repairable when it was replaced. The project documentation included a copy of the contractor s invoice for installation of the new chiller, and photographs of the new and original chillers nameplates. There were no specification sheets for the new chiller, but AEG was able to obtain verify the capacity and efficiency of the new chiller by conducting a search of the reported model number on the manufacturer s website (Trane). The verified efficiency (10.1 EER) of the new chiller was higher than that reported (9.7 EER) in the Direct Options database. Information posted in the Direct Options database reported that the original chiller was manufactured in 1985 (30 years old) and had an efficiency of 5 EER. The reported serial number of the original chiller suggested that the unit was manufactured in The measure included in this project has an EUL of 20 years and RUL of zero (0) years (per Table 262 of TRM 5.0) for the purpose of lifetime impact determination. Per TRM 5.0, air-cooled chillers (non-centrifugal) that are 30 years old no longer have an RUL. As such, AEG treated this project as Replace on Burnout and recalculated the impacts. Applied Energy Group, Inc. C-13

315 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-9 Documentation Review Summary for Rebate # Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 66, Verified First Year Impacts 9, Realization Rate 14% 6% Verified Lifetime Impacts 190, Sample 8 Rebate # OPP Dale Moore s Apple Market Project Description: This project involved the installation of a total of 213 linear feet of refrigerator door gaskets and 131 square feet of strip curtains on walk-in refrigerators in a supermarket facility. Both measures have deemed impacts specified by TRM 5.0. Findings: Key Issue: The project documentation was provided by CLEAResult, and consisted of a list showing the details of the door gasket and strip curtain installations as well as photographs of the installed measures. AEG replicated the savings for each measure using the information provided in the project documentation and the deemed savings specified in TRM 5.0, and found the following: o o o For the door gasket measure, the implementer used deemed impact values (kw per linear feet, kwh per linear feet) that were different than the values specified in TRM 5.0. The kw per linear feet value used by the implementer was lower than the value from the TRM, while the opposite was true for the kwh per square feet value (higher than the value from the TRM). It was noted that the door dimensions (29.75" x ") used for one of the door gaskets was incorrectly transferred from the inspection sheet and used for the reported savings calculations. AEG corrected the dimensions (26.75" x ") for the verified savings calculations. For the strip curtain measure, AEG s verified energy and demand impacts matched the reported impacts. The deemed impacts used to determine the reported impacts for the door gasket measure were not consistent with the values specified in TRM 5.0. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-10 Documentation Review Summary for Rebate # OPP Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 45, Verified First Year Impacts 32, Realization Rate 72% 103% Verified Lifetime Impacts 130, Applied Energy Group, Inc. C-14

316 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Recvd Review 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 Stratum 3 Sample 9 Rebate # OPP43593 Project Description: This project involved the installation of a total of 602 linear feet of refrigerator door gaskets and 149 square feet of strip curtains on walk-in refrigerators in a supermarket facility. Both measures have deemed impacts specified by TRM 5.0. Findings: Key Issue: The project documentation was provided by CLEAResult, and consisted of a list showing the details of the door gasket and strip curtain installations as well as photographs of the installed measures. AEG replicated the impacts for each measure using the information provided in the project documentation and the deemed impacts specified in TRM 5.0, and found the following: o o For the door gasket measure, the implementer used deemed impact values (kw per linear feet, kwh per linear feet) that were different than the values specified in TRM 5.0. The kw per linear feet value used by the implementer was lower than the value from the TRM, while the opposite was true for the kwh per square feet value (higher than the value from the TRM). For the strip curtain measure, AEG s verified energy and demand impacts matched the reported impacts. The deemed impacts used to determine the reported impacts for the door gasket measure were not consistent with the values specified in TRM 5.0. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-11 Documentation Review Summary for Rebate # OPP43593 Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 121, Verified First Year Impacts 43, Realization Rate 36% 125% Verified Lifetime Impacts 174, Sample 10 Rebate # OPP Project Description: This project involved the installation of a total of 325 linear feet of refrigerator door gaskets and 184 square feet of strip curtains on walk-in refrigerators in a supermarket facility. Both measures have deemed impacts specified by TRM 5.0. Findings: The project documentation was provided by CLEAResult, and consisted of a list showing the details of the door gasket and strip curtain installations as well as photographs of the installed measures. AEG replicated the impacts for each measure using the information provided in the project documentation and the deemed impacts specified in TRM 5.0, and found the following: Applied Energy Group, Inc. C-15

317 Key Issue: o o APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 For the door gasket measure, the implementer used deemed impact values (kw per linear feet, kwh per linear feet) that were different than the values specified in TRM 5.0. The kw per linear feet value used by the implementer was lower than the value from the TRM, while the opposite was true for the kwh per square feet value (higher than the value from the TRM). For the strip curtain measure, AEG s verified energy and demand impacts matched the reported impacts. The deemed impacts used to determine the reported impacts for the door gasket measure were not consistent with the values specified in TRM 5.0. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-12 Documentation Review Summary for Rebate # OPP Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 83, Verified First Year Impacts 41, Realization Rate 49% 115% Verified Lifetime Impacts 164, Sample 11 Rebate # OPP Project Description: This project involved the installation of a total of 536 linear feet of refrigerator door gaskets and 184 square feet of strip curtains on walk-in refrigerators in a supermarket facility. Both measures have deemed impacts specified by TRM 5.0. Findings: Key Issue: The project documentation was provided by CLEAResult, and consisted of a list showing the details of the door gasket and strip curtain installations as well as photographs of the installed measures. AEG replicated the impacts for each measure using the information provided in the project documentation and the deemed impacts specified Arkansas TRM 5.0, and found the following: o o For the door gasket measure, the implementer used deemed impact values (kw per linear feet, kwh per linear feet) that were different than the values specified in TRM 5.0. The kw per linear feet value used by the implementer was lower than the value from the TRM, while the opposite was true for the kwh per square feet value (higher than the value from the TRM). For the strip curtain measure, AEG s verified energy and demand impacts matched the reported impacts. The deemed impacts used to determine the reported impacts for the door gasket measure were not consistent with the values specified in TRM 5.0. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Applied Energy Group, Inc. C-16

318 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Recvd Review 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 Table C-13 Documentation Review Summary for Rebate # OPP Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 108, Verified First Year Impacts 44, Realization Rate 41% 119% Verified Lifetime Impacts 178, Sample 12 Rebate # Project Description: This project involved the replacement of an existing 60 HP air compressor with a new 100 HP air compressor equipped with VFD controls at a manufacturing facility. This measure does not have deemed savings values or algorithms specified by TRM 5.0. Findings: The project documentation was provided by CLEAResult, and consisted of 15-second interval kw data for periods of 16 days for both pre- and post-retrofit, OEM specifications, invoices, OEM specifications for existing and installed compressors, and photographs of the existing and installed equipment. The documentation and data provided were adequate for AEG to complete a documentation review, and AEG verified that the reported equipment was installed. However, the nature and complexity of the impact calculations are such that while we can comment on the approach taken to estimate impacts, it is impractical given the time and resources available to replicate or improve the impact estimation methodology for this project. o AEG recommends using AIRMaster+ software from the U.S. Department of Energy for determining energy and demand impacts of similar compressed air projects. This compressor replacement and VFD measure was assumed by AEG to have an EUL of 15 years, consistent with the Database of Energy Efficiency Resources (DEER). 66 Impacts Summary: The following impacts were determined using the approach described above. Table C-14 Documentation Review Summary for Rebate # Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 183, Unadjusted First Year Impacts 183, Realization Rate 100% 100% Lifetime Impacts 2,747, Database of Energy Efficiency Resources (DEER) EUL values for variable speed drive applications on motors, February Applied Energy Group, Inc. C-17

319 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 Sample 13 Rebate # OPP Project Description: This project involved the installation of a total of 228 linear feet of refrigerator door gaskets and 209 square feet of strip curtains on walk-in refrigerators in a supermarket facility. Both measures have deemed impact values specified by TRM 5.0. Findings: Key Issue: The project documentation was provided by CLEAResult, and consisted of a list showing the details of the door gasket and strip curtain installations as well as photographs of the installed measures. AEG replicated the impacts for each measure using the information provided in the project documentation and the deemed impacts specified in TRM 5.0, and found the following: o o For the door gasket measure, the implementer used deemed impact values (kw per linear feet, kwh per linear feet) that were different than the values specified in TRM 5.0. The kw per linear feet value used by the implementer was lower than the value from the TRM, while the opposite was true for the kwh per square feet value (higher than the value from the TRM). For the strip curtain measure, AEG s verified energy and demand impacts matched the reported impacts. The deemed impacts used to determine the reported impacts for the door gasket measure were not consistent with the values specified in TRM 5.0. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-15 Documentation Review Summary for Rebate # OPP Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 73, Verified First Year Impacts 46, Realization Rate 63% 108% Verified Lifetime Impacts 185, Stratum 4 Sample 14 Rebate # OPP Project Description: This project involved the installation of a variable frequency drive (VFD) on one of three 60 HP positive displacement blowers in a process wastewater treatment aeration basin. This measure does not have deemed impact values or algorithms specified by TRM 5.0. Findings: The project documentation was provided by CLEAResult, consisting of calculations on logged amperage data for the two active blowers (with spot-measurements of voltage and power factor) for periods of 12 days for pre-retrofit and 24 days for post-retrofit, an M&V report, invoices, and photographs of the equipment. AEG replicated the impacts for each measure using the logged amperage data and found the following: Applied Energy Group, Inc. C-18

320 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Recvd Review 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 o o o The implementer calculated system power (kw) from amperage incorrectly. While the power should be calculated as (Voltage*Amperage*Power Factor* 3)/1000, the implementer used a factor of 3 instead of the square root of 3 (1.73). This error resulted in verified impacts that were lower than the originally submitted values. Since this application is a relatively stable load over the whole year and is not weather-dependent, the average kw from the pre- and post-retrofit data were multiplied by 8,760 hours/yr to result in annual electricity consumption. In order to calculate the peak demand reduction expected for this project, AEG used the adjusted system power calculations and averaged the hourly demand for each hour of the day across the logged data for pre- and post-retrofit periods. The difference in average hourly demand for the hour of 4:00 to 5:00 pm (consistent with the OG&E system peak period definition) was then taken as the demand reduction for the project. This measure was assumed by AEG to have an estimated useful life of 15 years, per DEER Impacts Summary: The following verified impacts were determined using the approach described above. Table C-16 Documentation Review Summary for Rebate # OPP Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 317, Verified First Year Impacts 138, Realization Rate 44% 48% Verified Lifetime Impacts 2,082, Sample 15 Rebate # Project Description: This project involved the installation of blowers with VFD control to replace constant speed blowers, and retrofitting existing oven exhaust and circulation fans with VFDs at a fiberglass manufacturing facility. This measure does not have deemed impacts or algorithms specified by TRM 5.0. Findings: The project documentation was provided by CLEAResult, consisting of calculations on logged amperage data for the two active blowers and post-installation circulation fan (with spot-measurements of voltage and power factor), logged power (kw) data for the exhaust fan and the pre-installation circulation fan, an M&V report, invoices, and photographs of the equipment. AEG replicated the impacts for each measure using the provided data and the implementer s analysis methodology, and found the following: o The implementer calculated system power (kw) from amperage incorrectly. While the power should be calculated as (Voltage*Amperage*Power Factor* 3)/1000, the implementer used a factor of 3 instead of the square root of 3 (1.73). This 67 Database of Energy Efficiency Resources (DEER) EUL values for variable speed drive applications on motors, February Applied Energy Group, Inc. C-19

321 o APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 error resulted in verified impacts that were lower for the blower replacement but higher for the circulation fan. AEG accepted the energy savings calculation methodology provided by the implementer with the changes to system power calculations mentioned above, but adjusted the calculation methodology for the demand reduction. In the submitted documentation, the implementer calculated the demand reduction using the pre- and post-retrofit maximum instantaneous demand from 15-minute interval data over the logging period for each fan. AEG determined that the optimal approach for calculating peak demand reduction consistent with the OG&E system peak period was to calculate the average demand during the hour of 4:00 to 5:00pm for pre- and post-installation cases. No demand reductions were reported for the exhaust fan. This measure was assumed by AEG to have an estimated useful life of 15 years, per DEER Impacts Summary: The following verified impacts were determined using the approach described above. Table C-17 Documentation Review Summary for Rebate # Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 580, Verified First Year Impacts 615, Realization Rate 106% 92% Verified Lifetime Impacts 9,229, Samples 16 and 19 Rebate # OPP and OPP Project Description: These projects involved installing actuators on the ammonia compressor slide valves as well as the addition of controls at a cold storage facility. These measures do not have deemed impacts or algorithms specified by TRM 5.0. Findings: The project documentation was provided by CLEAResult, consisting of monthly billed electrical energy for periods of one year pre-retrofit and six months post-retrofit, an M&V report, invoices, and photographs of the existing and installed equipment. The documentation and data provided were adequate for AEG to complete a documentation review, and AEG verified that the reported equipment was installed. However, the nature and complexity of the impact calculations are such that while we can comment on the approach taken to estimate impacts, it is impractical given the time and resources available to replicate or improve the impact estimation methodology for this project. We include feedback on such approaches, and recommendations to improve those approaches, but are not able to estimate adjusted impacts in this case. o The approach taken by the implementer is representative of the impacts during the historical measurement period (January through June 2015), but may not be representative of the typically expected impacts if the conditions that the 68 Ibid. Applied Energy Group, Inc. C-20

322 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Recvd Review 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 o regression analysis is based on (weather and production) during the measurement period are not representative of a typical year. For projects that use regression analysis in an effort to produce estimates that are representative of typical first year and lifetime impacts, the following approach is recommended: A regression analysis should be performed for both pre- and post-retrofit data based on the respective historical weather and production data. The resultant regressions should then be applied to a full year of typical weather and production data. The peak demand reduction would then be calculated specific to the OG&E system peak period. Key statistical variables such as p-values should also be recorded in order to determine the statistical significance of the independent variables used for the regression. Since this facility had two meters but the project was analyzed on a whole-building basis, the impacts were separated into two rebate IDs (OPP and OPP312424) based on relative energy consumption between the two meters in June The EUL for this project was estimated by the implementer to be 10 years, consistent with the Arkansas TRM 5.0 lifetime for HVAC controls. Impact Summary: The following impacts were determined using the approach described above. Table C-18 Metric Documentation Review Summary for Rebates # OPP and OPP Energy Savings (kwh) OPP Annual Demand Reduction (kw) Energy Savings (kwh) OPP Annual Demand Reduction (kw) Reported Annual Impacts 600, ,346, Unadjusted First Year Impacts 600, ,346, Realization Rate 100% 100% 100% 100% Lifetime Impacts 6,002, ,468, Sample 17 Rebate # OPP Project Description: This project involved the retrofit of three existing air compressors (totaling 275 HP) with one new 200 HP compressor equipped with VFD controls as well as leak repair at the tube mill and main plant of a metal manufacturing facility. These measures do not have deemed impacts or algorithms specified by TRM 5.0. Findings: The project documentation was provided by CLEAResult, consisting of logged electrical demand data for the existing and installed compressors for approximately two week periods for pre- and post-retrofit, an M&V report, invoices, OEM specifications for existing and installed compressors, and photographs of the existing and installed equipment. AEG replicated the impacts for each measure using the provided data and the implementer s analysis methodology, and found the following: o In the submitted documentation, the implementer increased the post-retrofit energy use based on pre-retrofit production levels to account for the fact that there was a considerable decrease in the post-retrofit production levels. This results in post-retrofit energy use that is representative of the pre-retrofit production levels. While this approach would not be appropriate if the postretrofit production levels are more representative of future production conditions than the pre-retrofit production levels, there was not enough documentation to Applied Energy Group, Inc. C-21

323 o APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 indicate which production levels were more representative. Therefore, AEG accepted the energy savings calculation methodology provided by the implementer. AEG did adjust the calculation methodology for the demand reduction, however. In the submitted documentation, the implementer calculated the demand reduction using the pre- and post-retrofit maximum instantaneous demand from 15-minute interval data over the logging period for each compressor. AEG determined that the optimal approach for calculating peak demand reductions consistent with the OG&E system peak period was to calculate the average demand during the hour of 4:00 to 5:00pm during production days for pre- and post-retrofit cases. This resulted in verified demand reductions that were higher than the reported values. This compressor replacement measure was assumed by AEG to have an EUL of 15 years, consistent with the Ohio Technical Reference Manual as well as a U.S. DOE Market assessment. 69 Impact Summary: The following verified impacts were determined using the approach described above. Table C-19 Documentation Review Summary for Rebate # OPP Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 641, Verified First Year Impacts 641, Realization Rate 100% 177% Verified Lifetime Impacts 9,628, Sample 18 Rebate # OPP Project Description: This project involved the full system retrofit of a chilled water system used for process cooling at a food manufacturing facility, including chiller replacement, multiple pump replacements, and VFD control. This full measure package does not have deemed impacts or algorithms specified by TRM 5.0. Findings: The project documentation was provided by CLEAResult, consisting of logged electrical demand data for the existing and installed chillers, logged amperage data for the cooling tower fan motors, chilled water supply pumps, and condenser water pumps (with spotmeasurements of voltage and power factor) for periods of 45 days pre-retrofit and 25 days post-retrofit, an M&V report, invoices, and photographs of the existing and installed equipment. The documentation and data provided were adequate for AEG to complete a documentation review, and AEG confirmed that the reported equipment was installed. However, the nature and complexity of the impact calculations are such that while we can comment on the approach taken to estimate impacts, it is impractical given the time and resources available to replicate or improve the impact estimation methodology for 69 Ibid. Applied Energy Group, Inc. C-22

324 APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Recvd Review 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 this project. We include feedback on such approaches, and recommendations to improve those approaches, but are not able to estimate adjusted impacts in this case. o o The approach taken by the implementer is representative of the impacts during the historical measurement period (portions of August, November, and December 2014), but may not be representative of the typically expected annual impacts if the conditions that the regression analysis is based on (weather and production) during the measurement period are not representative of a typical year. For projects that use regression analysis in an effort to produce estimates that are representative of typical first year and lifetime impacts, the following approach is recommended: A regression analysis should be performed for both pre- and post-retrofit data based on the respective historical weather and production data. The resultant regressions should then be applied to a full year of typical weather and production data. The peak demand reduction would then be calculated specific to the OG&E system peak period. Key statistical variables such as p-values should also be recorded in order to determine the statistical significance of the independent variables used for the regression. The implementer calculated system power (kw) from amperage incorrectly. While the power should be calculated as (Voltage*Amperage*Power Factor* 3)/1000, the implementer used a factor of 3 instead of the square root of 3 (1.73). Since the impacts for this project were calculated for the whole system and not per equipment retrofit, AEG assumed an EUL of 15 years for this set of measures; this is consistent with the EUL for motor replacement in TRM 5.0, VFD control EUL from DEER, and is shorter than the 20 year EUL for chiller replacement from TRM 5.0. Thus, we would expect full impacts for the set of measures to persist for the 15 yea rs. Impact Summary: The following impacts were determined using the approach described above. Table C-20 Documentation Review Summary for Rebate # Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 843, Unadjusted First Year Impacts 843, Realization Rate 100% 100% Lifetime Impacts 12,655, Sample 19 Rebate # OPP This rebate is covered in the prior discussion for Samples 16 and 19. Sample 20 Rebate # OPP Project Description: This project involved the installation of the Surveyor power management software in a total of 2,857 desktop and notebook computers at a hospital facility. The Surveyor software is an add-on software that switches off displays and enable computers to enter a low power setting during periods of non-use. The computer power management measure has deemed impacts specified by TRM This was the un-rounded demand reduction in the submitted documentation. Applied Energy Group, Inc. C-23

325 Findings: APSC FILED Time: 5/2/2016 Documentation 2:38:38 PM: Review Recvd 5/2/2016 Results 2:33:07 PM: Docket TF-Doc. 298 The project documentation was provided by CLEAResult, and consisted of the following: o o o Spreadsheet of the inventory of desktop and notebook computers at the facility. Spreadsheet showing the energy and demand impact calculations. Invoice from the contractor showing the installation of the Surveyor software in 2,857 computers at the facility. AEG replicated the impacts for each measure using the information (type and quantity of computers) provided in the project documentation and the deemed impacts specified in TRM 5.0. The energy and demand impacts calculated by AEG matched the reported impacts for this project. Impact Summary: The following verified impacts were obtained using the approach specified in TRM 5.0. Table C-21 Documentation Review Summary for Rebate # OPP Metric Energy Savings (kwh) Annual Demand Reduction (kw) Reported Annual Impacts 1,417, Verified First Year Impacts 1,417, Realization Rate 100% 100% Verified Lifetime Impacts 5,669, Applied Energy Group, Inc. C-24

326 APPENDIX D Lifetimes Impacts by Program, Measure, and Year Lifetime Gross Energy (kwh) Savings Program Measure EUL (TRM) Multi-Family Direct Install SEE LivingWise CFL 9 194, , , , , , , ,810 45, ,603,938 Faucet Aerator , , , , , , , , , , ,532,271 Low-Flow Showerhead , , , , , , , , , , ,673,057 Air Infiltration , , , , , , , , , , , ,246,142 Duct Sealing 18 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860, ,488,027 Advanced Power Strip , , , , , , , , , , ,462,706 Program Total 18 4,408,073 4,408,073 4,408,073 4,408,073 4,408,073 4,408,073 4,408,073 4,408,073 4,258,718 4,213,262 3,246,459 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860,446 2,860, ,006,141 CFL 9 120,306 77,173 77,173 77,173 77,173 77,173 77,173 77,173 21, ,490 Faucet Aerator 10 42,972 42,972 42,972 42,972 42,972 42,972 42,972 42,972 42,972 42, ,716 Low-Flow Showerhead , , , , , , , , , , ,523,251 Program Total , , , , , , , , , , ,635,456 Commercial Lighting Program Total ,595,356 6,411,781 6,410,436 6,410,436 6,410,436 6,410,436 6,410,436 6,410,436 5,555,334 4,516,513 4,516,513 4,516,513 4,516,513 4,516,513 4,507,343 1,298, ,413,820 C&I Direct Install C&I Standard Offer (Non-Direct Install) CFL 3 74,241 47,387 47, ,014 Pre-Rinse Spray Valve 5 74,255 74,255 74,255 74,255 74, ,277 Faucet Aerator , , , , , , , , , , ,919,395 Low-Flow Showerhead 10 53,460 53,460 53,460 53,460 53,460 53,460 53,460 53,460 53,460 53, ,600 Vending Miser 5 66,092 66,092 66,092 66,092 66, ,460 Program Total , , , , , , , , , , ,324,745 Program Total 20 6,518,250 6,517,725 6,512,999 6,500,604 4,671,997 4,671,997 4,671,997 4,671,997 4,671,997 4,563,069 2,624,179 2,624,179 2,624,179 2,624,179 2,624,179 30,536 30,536 30,536 30,536 30,536 67,246,200 Total All Programs 20 18,297,268 18,043,182 18,037,112 17,977,329 16,148,722 16,008,375 16,008,375 16,008,375 14,948,715 13,733,540 10,387,150 10,001,137 10,001,137 10,001,137 9,991,967 4,189,804 2,890,982 2,890,982 30,536 30, ,626,363 Lifetime Energy Savings (kwh) Lifetime Net Energy (kwh) Savings Program Measure EUL (TRM) Multi-Family Direct Install SEE LivingWise CFL 9 157, , , , , , , ,796 36, ,299,190 Faucet Aerator , , , , , , , , , , ,440,334 Low-Flow Showerhead , , , , , , , , , , ,345,943 Air Infiltration , , , , , , , , , , , ,821,528 Duct Sealing 18 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574, ,339,224 Advanced Power Strip , , , , , , , , , , ,116,435 Program Total 18 3,969,881 3,969,881 3,969,881 3,969,881 3,969,881 3,969,881 3,969,881 3,969,881 3,848,903 3,812,084 2,921,813 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574,401 2,574, ,362,655 CFL 9 112,931 72,443 72,443 72,443 72,443 72,443 72,443 72,443 20, ,653 Faucet Aerator 10 48,820 48,820 48,820 48,820 48,820 48,820 48,820 48,820 48,820 48, ,200 Low-Flow Showerhead , , , , , , , , , , ,639,932 Program Total , , , , , , , , , , ,768,785 Commercial Lighting Program Total ,529,402 6,347,664 6,346,332 6,346,332 6,346,332 6,346,332 6,346,332 6,346,332 5,499,781 4,471,348 4,471,348 4,471,348 4,471,348 4,471,348 4,462,269 1,285, ,559,682 C&I Direct Install C&I Standard Offer (Non-Direct Install) CFL 3 74,241 47,387 47, ,014 Pre-Rinse Spray Valve 5 74,255 74,255 74,255 74,255 74, ,277 Faucet Aerator , , , , , , , , , , ,919,395 Low-Flow Showerhead 10 53,460 53,460 53,460 53,460 53,460 53,460 53,460 53,460 53,460 53, ,600 Vending Miser 5 66,092 66,092 66,092 66,092 66, ,460 Program Total , , , , , , , , , , ,324,745 Program Total 20 6,257,520 6,257,016 6,252,479 6,240,579 4,485,117 4,485,117 4,485,117 4,485,117 4,485,117 4,380,546 2,519,211 2,519,211 2,519,211 2,519,211 2,519,211 29,314 29,314 29,314 29,314 29,314 64,556,352 Total All Programs 20 17,542,535 17,292,949 17,287,081 17,227,795 15,472,332 15,331,985 15,331,985 15,331,985 14,312,637 13,122,191 9,912,372 9,564,960 9,564,960 9,564,960 9,555,882 3,889,550 2,603,716 2,603,716 29,314 29, ,572,219 Lifetime Energy Savings (kwh) Applied Energy Group, Inc. D-1

327 Lifetimes Savings by Program, Measure and Year Lifetime Gross Weighted Average Demand (kw) Reduction Program Measure EUL (TRM) Multi-Family Direct Install SEE LivingWise CFL Faucet Aerator Low-Flow Showerhead Air Infiltration Duct Sealing Advanced Power Strip Program Total CFL Faucet Aerator Low-Flow Showerhead Program Total Commercial Lighting Program Total , , , , , , , , C&I Direct Install C&I Standard Offer (Non-Direct Install) CFL Pre Rinse Spray Valve Faucet Aerator Low-Flow Showerhead Vending Miser Program Total Program Total Total All Programs 20 2, , , , , , , , , , , , , , , , Lifetime Net Weighted Average Demand (kw) Reduction Program Measure EUL (TRM) Multi-Family Direct Install SEE LivingWise CFL Faucet Aerator Low-Flow Showerhead Air Infiltration Duct Sealing Advanced Power Strip Program Total CFL Faucet Aerator Low-Flow Showerhead Program Total Commercial Lighting Program Total , , , , , , , , C&I Direct Install C&I Standard Offer (Non-Direct Install) CFL Pre Rinse Spray Valve Faucet Aerator Low-Flow Showerhead Vending Miser Program Total Program Total Total All Programs 20 2, , , , , , , , , , , , , , , , Weighted Average Lifetime Demand Reduction (kw) Weighted Average Lifetime Demand Reduction (kw) Applied Energy Group, Inc. D-2

328 APPENDIX E Glossary A-B Accuracy - The relationship between the true value of a variable and an estimate of that variable. The term can also be used in reference to a model, a set of measured data, or to describe a measuring instrument s capability. The accuracy of an estimate based on a sample is defined by the level of confidence for the sample and the precision of the estimate. The confidence and precision levels are often stated together. For example, 90/10 precision level means 90% confidence that the error of the estimate (the difference between the estimate and the true value) is no more than 10%. Confidence - The certainty that the true value lies within an interval. If a 90% confidence interval of an estimate is +/- 0.5 kw, then we are 90% certain that the true value lies within 0.5 kw of our estimate. Precision - The size of the confidence interval. This can be absolute precision, in terms of the units measured (i.e. +/- 0.5 kw) or expressed as a percent of the estimate (i.e. +/- 8%). Adjusted Gross Savings - The change in energy consumption and/or demand that results directly from program-related actions taken by participants in an efficiency program, regardless of why they participated. It adjusts the savings relative to either ex-ante or reported savings (in this report, OG&E Reported Savings) for such factors as data errors, installation and persistence rates, and hours of use, but does not adjust for free ridership or spillover. For this report, the adjusted gross savings are annual kwh energy savings and peak kw demand reduced. Final adjusted gross savings (after all adjustments have been made) are also referred to as Adjusted Gross Impacts or Evaluated Gross Impacts. (See also AEG Adjusted Savings and Gross Savings.) AEG Adjusted Savings - The energy and demand impacts developed by AEG as part of the evaluation process using one or a combination of approaches including: billing analysis, simulation modeling, engineering review, on-site field verification and/or metering, and savings accounting. 71 In this report, this is the same as AEG Evaluated Impacts. Algorithm - Equation or set of equations, more broadly a method, used to calculate a number. In this case, it is an estimate of energy use or demand reduction tied to operation of a piece of equipment or a system of interacting pieces of equipment. An algorithm may include certain standard numerical assumptions about some relevant quantities, leaving the user to supply other data to calculate the use or savings for the particular measure or equipment. Annual Energy Savings - The reduction in electricity usage (kwh) or in fossil fuel use in thermal unit(s) from the savings associated with an energy saving measure, project, or program in a given year. 71 AEG Definition

329 APSC FILED Time: 5/2/2016 2:38:38 Glossary PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Attribution - Ascribing or establishing a causal relationship between action(s) taken by an entity and an outcome. For an energy efficiency program this usually means the portion of all gross savings that is actually attributable to the program, as opposed to occurring naturally. Baseline - Conditions, including energy consumption and related emissions, which would have occurred without implementation of the subject measure or project. Baseline conditions are sometimes referred to as business-as-usual conditions and are used to calculate program-related efficiency or emissions savings. Baselines can be defined as either project-specific baselines or performance standard baselines (e.g. building codes). Baseline Data - The baseline conditions of the facilities, market segment, generating equipment, or other area of focus of the subject project or program. Baseline Efficiency - A subset of "baseline" referring to the energy usage of the baseline equipment, process, or standard that is being replaced by a more efficient approach to providing the same energy service. It is used to determine the energy savings obtained by the more efficient approach. Baseline Period - The period of time selected as representative of the operations of the area of focus before the energy efficiency activity takes place. Bias - The extent to which a measurement or a sampling or analytic method systematically underestimates or overestimates a value. Some examples of types of bias include engineering model bias; meter bias; sensor placement bias; inadequate or inappropriate estimate of what would have happened absent a program or measure installation; a sample that is unrepresentative of a population; and selection of other variables in an analysis that are too correlated with the savings variable (or each other) in explaining the dependent variable (such as consumption). Billing Analysis - An analytic methodology used to estimate program savings. It compares billing data from program participants over a period of time before the energy efficient measures are installed at customer sites to billing data for a comparable period of time afterward. Commonly, monthly billing data are gathered for the year before and the year after installation. Also common is to compare the before-after difference for the group of participating customers to the corresponding before-after differences in bills for a comparable group of non-participants. Billing Data - Data obtained from the electric or gas meter that are used to bill the customer for energy used in a particular billing period. In an evaluation context, billing data also refers to the customer billing records over time that are used to conduct analyses of energy use before and after implementation of energy efficiency measures. Statistically Adjusted Engineering (SAE) Models - A category of statistical analysis models that incorporates the engineering estimate of savings as an independent variable. The regression coefficient in these models is the percentage of the engineering estimate of savings observed in changes in energy usage. For example, if the coefficient on the SAE term is 0.8, this means that the customers are on average realize 80% of the savings from their engineering estimates. Building Energy Simulation Models - Computer models based on physical engineering principles and/or standards used to estimate energy usage and/or savings. These models usually incorporate site-specific field-verified data and/or default values for customers and physical systems such as square footage, weather, surface orientations, elevations, space volumes, construction materials, equipment use, lighting, and building occupancy. Building simulation models can usually account for interactive effects between end uses (e.g. lighting and HVAC), part-load efficiencies, and changes in external and internal Applied Energy Group, Inc. E-2

330 APSC FILED Time: 5/2/2016 2:38:38 PM: Glossary Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 heat gains/losses. Examples of building simulation models include DOE-2, BEopt TM, RESNET, EnergyPlus, and Carrier HAP. Calibration - In economic planning, or engineering modeling, the process of adjusting the components of the model to reflect reality as best as possible, in order to prepare for the model s use in future applications. The term also applies to the process whereby metering and measurement equipment is periodically adjusted to maintain industry measurement standards. Simulation Model - An assembly of algorithms that calculates energy use based on engineering equations and user-defined parameters. Whole-Building Calibrated Simulation Approach - A savings measurement approach (defined in IPMVP Option D and ASHRAE Guideline 14) that involves the use of an approved computer simulation program to develop a physical model of the building in order to determine energy and demand savings. The simulation program is used to model the energy used by the facility before and after the retrofit. The pre or post-retrofit models are developed by calibration with measured energy use and demand data and weather data. C-D Coincident Demand - The demand of a device, circuit, or building that occurs at the same time as the peak demand of a utility s system load or at the same time as some other peak of interest, such as building or facility peak demand. The peak of interest should be specified (e.g. demand coincident with the utility system peak ). Examples of peak demand include: demand coincident with utility system peak load; demand coincident with ISO/RTO summer or winter peak, or according to performance hours defined by wholesale capacity markets; and demand coincident with high electricity demand days. Peak Demand - The maximum level of hourly demand during a specified period. The peak periods most commonly identified are annual and seasonal (summer and winter). Comparison Group - Also called Control Group. A selected group of individuals or organizations that have not had the opportunity to receive program benefits and that has been selected because its characteristics match those of another group of individuals or organizations that have had the opportunity to receive program benefits. In a randomized control trial (RCT), the control group matches the participant group because they are randomly assigned before program participation. In a Quasi-experimental design, a matched control group of customers similar to the participants is selected after program participation. The match should be based on as many characteristics as possible, with particular focus on characteristics associated with or related to the action or behavior that the evaluation is trying to examine. The comparison group is used to isolate program effects from other factors that affect energy use. Custom Program - An energy efficiency program intended to provide efficiency solutions to unique situations not amenable to common or prescriptive solutions. Each custom project is examined for its individual characteristics, savings opportunities, efficiency solutions, and often, customer incentives. Database for Energy-Efficient Resources (DEER) - A California database designed to provide well documented estimates of energy and peak demand savings values, measure costs, and effective useful life. Often used as ex-ante or deemed savings. See Deemed Savings - An estimate of energy or demand impacts for a single unit of an installed energy efficiency measure that (a) has been developed from data sources and analytical methods that are widely considered acceptable for the measure and purpose, and (b) is Applied Energy Group, Inc. E-3

331 APSC FILED Time: 5/2/2016 2:38:38 Glossary PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 applicable to the situation being evaluated. Individual parameters or calculation methods can also be deemed. 72 Demand Reduction - Reduction in at-site electrical demand associated with a program measure, measured in kw. Usually calculated to reflect the reduction of electrical demand during the customer s or the utility s peak period; the latter is referred to as coincident demand reduction. In prior reports, this has also been called Demand Savings. Defensibility - The ability of evaluation results to stand up to scientific scrutiny. Defensibility is based on assessments by experts of the evaluation s validity, reliability, and accuracy. Degree Days - For any individual day, degree days indicate how far that day's average temperature departed from a chosen base temperature, such as 65 F. Heating degree days is a measure to indicate how far the average temperature fell below 65 F. Similarly, Cooling Degree Days, which measure cooling energy demand, indicate how far the temperature averaged above 65 F. In both cases, smaller values represent less fuel demand, but values below 0 are set equal to 0, because no energy is needed to maintain those temperatures. Furthermore, since energy demand is cumulative, degree day totals for periods exceeding 1 day are simply the sum of each individual day's degree day total. Cooling Degree Days - The cumulative number of degrees and days in a month or year by which the mean temperature is above the selected base temperature, commonly 18.3 C/65 F. Also see Degree Days. Heating Degree Days - The cumulative number of degrees and days in a month or year by which the mean temperature falls below the selected base temperature, commonly 18.3 C/65 F. Also see Degree Days. E-F Early Retirement - Occurs when existing, functional, actively used or repairable equipment is replaced with similar, higher efficiency equipment. The equipment being replaced should have at least one year of Remaining Useful Life (RUL). Energy and demand impacts from the measure are calculated in two parts: 1) as the incremental impacts over existing equipment during the RUL period, and 2) as the incremental impacts over current code or standard equipment during the EUL-RUL period, where EUL is the Effective Useful Life of the installed measure. Effective Useful Life (EUL) - Also known as Measure Lifetime or Estimated Useful Life, this is the length of time (in years) at which 50% of the installed measures are expected to fail or no longer be in service. This metric takes into account the median number of years that a measure can operate as well as the amount of time that the measure lasts due to turnover, early retirement, or other reasons. Energy Efficiency Measure (EEM) - An installed piece of equipment or system, or modification of equipment, systems, or operations on end-use customer facilities that reduces the total amount of electrical or gas energy and capacity that would otherwise have been needed to deliver an equivalent or improved level of end-use service. Energy Efficiency Ratio (EER) - The ratio of the cooling capacity in Btu/h to the power input values in watts at any given set of rating conditions, expressed in Btu/(W h). The higher the EER, the more energy efficient the equipment (e.g., air conditioner, heat pump) is at the rated condition. 72 AEG definition. Applied Energy Group, Inc. E-4

332 APSC FILED Time: 5/2/2016 2:38:38 PM: Glossary Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 ENERGY STAR - A joint program of the U.S. Environmental Protection Agency and the U.S. Department of Energy designed to reduce energy use and the impact on the environment. The ENERGY STAR label is awarded to products that meet applicable energy efficiency guidelines and to homes and commercial buildings that meet specified energy efficiency standards. The program provides a range of energy management tools, primarily computer-based, for businesses. Engineering Methods - The use of standard formulas or models based on those formulas, typically accepted by ASHRAE, as the basis for calculating energy use. Engineering Model - Engineering equations used to calculate energy usage and savings. These models are usually based on a quantitative description of physical processes that transform delivered energy into useful work such as heat, lighting, or motor drive. In practice, these models may be reduced to simple equations in spreadsheets that calculate energy usage or savings as a function of measurable attributes of customers, facilities, or equipment (e.g. lighting use = watts hours of use). See the definition for Algorithm. Engineering Review and On-Site Field Visits - Engineering review (ER) includes examining exante or Deemed Savings (or in this report, OG&E Reported Savings) calculations to confirm/adjust number of installations recorded and/or the per-unit savings. It is best suited to programs that use deemed savings values or deemed calculated values, such as prescriptive measure programs. Applied to prescriptive measures, ER varies by program in terms of how actions are counted and information used to calculate per-unit savings, but adjusted or realized savings are always the number of confirmed actions multiplied by realized per-unit savings. Evaluators check for the appropriate use of deemed values and/or formulas contained in the program plan and calibrate values and/or formulas based on assumptions researched through study or secondary data and/or data collected in participant surveys or field verifications. From assessing and adjusting the components of deemed savings values, the evaluator develops realization rates by program and/or measure, and apply them to the population of participants in the program to obtain adjusted (realized) gross programlevel savings. 73 Evaluated Impacts - Another term for final Adjusted Savings or Realized Savings. First Year Impacts - These are the impacts (energy savings or peak demand reduction) accrued during the program year that is being evaluated. For instance, the first year impacts for PY2015 are the impacts in 2015 for the program that was implemented in PY2015. G-I Gross Savings - The change in energy consumption and/or demand that results directly from program related actions taken by participants in an efficiency program, regardless of why they participated and unadjusted by any factors. In this report, this is the same as Gross Impacts. Gross kw - Expected demand reduction based on a comparison of standard or replaced equipment, and the EEM installed through an energy efficiency program. Gross kwh - Expected kwh reduction based on a comparison of standard or replaced equipment, and the EEM installed through an energy efficiency program. 73 AEG Definition Applied Energy Group, Inc. E-5

333 APSC FILED Time: 5/2/2016 2:38:38 Glossary PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Heating Seasonal Performance Factor (HSPF) - The total space heating required during the space heating season (including supplementary electric heat), expressed in Btus, divided by the total electrical energy consumed by the heat pump system during the same season, expressed in watt-hours. The higher the HSPF, the more efficient the heat pump system. Heat Pump - A heating and cooling process that uses electricity to draw heat from a lower temperature source and transport it to a higher temperature location for heating purposes; or inversely, for cooling purposes. Heat pump technology is used for space heating and cooling, domestic water heating, and for process heating applications. There are various types of electric heat pumps and they are often defined by the content or location of the heat source: air, water, ground, and waste heat. Home Energy Rating System (HERS) - HERS is an indexing system used in residential new construction to rate the pre- and post- construction of new homes to indicate the degree of energy efficiency embedded in the construction. The HERS Index is a scoring system established by the Residential Energy Services Network (RESNET) in which a home built to the specifications of the HERS Reference Home (based on the 2006 International Energy Conservation Code) scores a HERS Index of 100; a net zero energy home scores a HERS Index of 0. The lower a home s HERS Index, the more energy efficient it is in comparison to the HERS Reference Home. Each 1-point decrease in the HERS Index corresponds to a 1% reduction in energy consumption compared to the HERS Reference Home. Horsepower (hp) - A unit for measuring the rate of doing work. One horsepower equals about three fourths of a kilowatt (745.7 watts). Hours of Use (HOU) - The average number of hours a measure is in use during a specified time period, typically a day or a year. This is also referred to as Annual Operating Hours (AOH). Impact Evaluation - An evaluation of the program-specific directly induced quantitative changes (e.g. kwh, kw, and therms) attributable to an energy efficiency program. Incentive - A financial strategy intended to encourage a change in behavior related to energy use. Incentives can take various forms. Customer incentives are commonly used in efficiency programs as rebates for EEMs or as buy-downs in upstream programs (aimed at retailers or manufacturers). In-Service Rate (ISR) - The percentage of measures that are incented by an efficiency program that are actually installed in a defined period of time. The ISR is more commonly used for measures that can be easily removed or disabled by customers, such as CFLs or automatic thermostats. The in-service rate is calculated by dividing the number of EEMs installed by the number of EEMs incented by an efficiency program in a defined period of time. Interactive Effects - The influence of one technology s application on the energy required to operate another application. An example is the reduced heat in a facility as a result of replacing incandescent lights with CFLs, and the resulting need to increase space heating from another source, usually oil or gas fired. J-P Load Factor - The load factor is defined by the Uniform Methods Project as the average demand divided by peak demand. Applied Energy Group, Inc. E-6

334 APSC FILED Time: 5/2/2016 2:38:38 PM: Glossary Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 LF = Average Load Maximum load in a given time period A load factor can be used to describe the ratio of the average load that equipment actually draws when it is in operation to the maximum (rated) load that it could draw. For example, if an oversized 35 hp motor drives a constant 25 hp load whenever the load is on, the motor load factor is 25/35 = 71.4%. The equipment load factor is often used in engineering algorithms to determine the annual energy consumption of motors or motor-driven HVAC equipment. The load factor can also be used in billing analyses. Lifetime Savings - The sum of energy savings over the course of the measure s effective useful life (EUL). This metric takes into account any baseline changes that may happen over the lifetime of a measure (e.g., efficiency increases for lighting due to EISA 2007 standards). Measurement and Verification (M&V) - A component of program impact evaluation that is associated with the documentation of energy savings at individual sites or projects using one or more methods that can involve measurements, engineering calculations, statistical analyses, and/or computer simulation modeling. M&V standards are outlined in the International Performance Measurement and Verification Protocol (IPMVP). AEG uses Options A-D in the IPMVP to develop AEG Adjusted Savings. Verification - An independent assessment of the reliability (considering completeness and accuracy) of reported energy savings or an emissions source inventory. Net Savings - The total change in load that is attributable to an energy efficiency program. This change in load may include, implicitly or explicitly, the effects of spillover, free riders, energy efficiency standards, changes in the level of energy service, and other causes of changes in energy consumption or demand. In this report, this is the same as Net Impacts. Naturally Occurring Efficiency - The effects of energy-related decisions that would have been made in the absence of the program administrator programs by both program participants and non-participants. Free Rider - A program participant who would have implemented the program measure or practice in the absence of the program. Free riders can be: 1) total, in which the participant s activity would have completely replicated the program measure; 2) partial, in which the participant s activity would have partially replicated the program measure; or 3) deferred, in which the participant s activity would have completely replicated the program measure, but at a future time than the program s timeframe. Free Ridership Rate - The percent of savings attributable to free riders. Spillover (Participant and Non-Participant) - Reductions in energy consumption and/or demand caused by the presence of an energy efficiency program, beyond the program-related gross savings of the participants and without financial or technical assistance from the program. There can be participant and/or non-participant spillover. Participant spillover is the additional energy savings that occur when a program participant independently installs energy efficiency measures or applies energy saving practices after having participated in the efficiency program as a result of the program s influence. Non-participant spillover refers to energy savings that occur when a program nonparticipant installs energy efficiency measures or applies energy savings practices as a result as a result of a program s influence. Net-to-Gross Ratio (NTGR) - A factor representing net program savings divided by gross program savings that is applied to gross program impacts to convert them into net program load impacts. The factor itself may be made up of a variety of Applied Energy Group, Inc. E-7

335 APSC FILED Time: 5/2/2016 2:38:38 Glossary PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 factors that create differences between gross and net savings, commonly including free riders and spillover. NTGR = 1 Free Rider % + Spillover %. OG&E Reported Gross Savings - The gross energy savings and peak demand reduced as reported by OG&E for each program. In this report, this is the same as OG&E Reported Gross Impacts. Participant - A consumer that received a service offered through the subject efficiency program, in a given program year. The term service is used in this definition to suggest that the service can be a wide variety of services, including financial rebates, technical assistance, product installations, training, energy-efficiency information or other services, items, or conditions. A participant could be a builder, a home, a facility, a company, a project, etc. Prescriptive Program - An energy efficiency program focused on measures that are one-for-one replacements of the existing equipment and for which fixed customer incentives can be developed based on the anticipated similar savings that will accrue from their installation. Q-Z Quality Assurance and Control (QA/QC) - QA refers to the planned and systematic activities implemented in a quality system so that quality requirements for a product or service will be fulfilled. It is the systematic measurement, comparison with a standard, monitoring of processes and an associated feedback loop that confers error prevention. QC is a procedure or set of procedures intended to ensure that a manufactured product or performed service adheres to a defined set of quality criteria or meets the requirements of the client or customer. QA is sometimes expressed together with QC as a single expression, QA/QC. Realization Rate - The term is used in the report for the comparison of AEG Adjusted Gross Savings to OG&E Reported Savings. The primary applications include the ratio of project tracking system savings data (e.g. initial estimates of project savings) to savings: 1) adjusted for data errors, 2) adjusted for correct application of savings algorithms, and 3) incorporating evaluated or verified results of the tracked savings. Realized Savings Another term for Evaluated Impacts or final Adjusted Savings. Remaining Useful Life (RUL) - This is the estimated remaining life of the existing equipment, or the length of time (in years) during which equipment replacement can be evaluated with an Early Retirement (existing) baseline. The RUL is based on the length of time at which 50% of existing equipment of a particular age is expected to fail and is typically evaluated with a survival function. To illustrate, a linear survival function for a commercial packaged terminal AC (PTAC) system from the AR TRM is used (see the figure below). While the EUL of this system is 10 years (50% of installed equipment is expected to fail at 10 years), a 12 year old PTAC system still has remaining useful life. Out of the PTAC systems still operating at the age of 12 years (40% of original), 50% (or 20% of original) are expected to fail by the age of 16 years. Thus, the difference between the 50% existing equipment failure age (16 years) and the age at which all equipment is expected to fail (20 years) is a remaining useful life of four (4) years. The Arkansas TRM does not allow claiming an RUL or Early Retirement baseline for this measure past 15 years, because 15 years represents the age at which 75% of existing equipment is expected to have failed. Applied Energy Group, Inc. E-8

336 APSC FILED Time: 5/2/2016 2:38:38 PM: Glossary Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Survival Function for Commercial PTAC/PTHP System 74 Replace on Burnout (ROB) - Occurs when existing equipment fails or is retired and is replaced with similar, higher efficiency equipment. Efficiency of the new equipment should exceed efficiency of current code or standard equipment. Energy and demand impacts from the measure are calculated as the incremental impacts over current code or standard equipment during the Effective Useful Life (EUL) of the installed measure. Sample Design - When conducting evaluations, it is often not possible to study the entire population of interest. As a result, evaluators use statistical samples to gather data, starting by defining a sampling frame that represents the population of interest from which a sample is to be drawn. The sample design provides the basic plan and method to select the sample, including the size of the sample. Sample designs can be based on maximizing precision for a given budget, or can be designed to achieve a particular level of precision. Sample - In program evaluation, the members of the population selected to represent the whole. Differing evaluation approaches rely on simple random or stratified (based on characteristic of the population) samples. Simple Random Sample - This is a subset of a population in which each member of the subset has an equal probability of being chosen. A simple random sample is meant to be an unbiased representation of a group. Stratified Random Sample - The population is divided into some number of subpopulations, called strata, which are non-overlapping and together span the entire population. A simple random sample is taken within each stratum. The strata are usually defined to reduce the overall variance, thereby improving the precision of the estimates. The results are analyzed first within each stratum, and then the stratum-level results can be weighted to appropriately represent the entire population. In some strata, all members of the population in a stratum are selected; this is called a census stratum. If properly weighted, a stratified sample will be an unbiased representation of the population. 74 Figure 10 of the AR TRM 5.0, sourced from U.S. DOE Technical Support Document for Packaged Terminal Air Conditioners and Heat Pumps. Applied Energy Group, Inc. E-9

337 APSC FILED Time: 5/2/2016 2:38:38 Glossary PM: Recvd 5/2/2016 2:33:07 PM: Docket TF-Doc. 298 Seasonal Energy Efficiency Ratio (SEER) - The total heat removed from the conditioned space during the annual cooling season, expressed in Btus, divided by the total electrical energy consumed by the air conditioner or heat pump during the same season, expressed in watt-hours. The higher the SEER, the more energy efficient the equipment (e.g., air conditioner, heat pump). Structured Interview - An interview in which the questions to be asked, their sequence, and the detailed information to be gathered are all predetermined. These are used where maximum consistency across interviews and interviewees is needed. Technical Reference Manual (TRM) - A resource document that includes information used in program planning and reporting of energy efficiency programs. It can include savings values for measures, engineering algorithms to calculate savings, impact factors to be applied to calculated savings (e.g., net-to-gross values), source documentation, specified assumptions, and other relevant material to support the calculation of measure and program savings. Weighted Average Lifetime Demand Reduction - The sum of annual demand reductions across the lifetime of a measure divided by the effective useful life (EUL, in years) of that measure. For weighted reductions across multiple measures, the sum of annual demand reductions is divided by the EUL of the measure with the longest lifetime within the measure package. This approximates the average demand reduction that would be expected each year across all years of interest. Applied Energy Group, Inc. E-10

338 Applied Energy Group, Inc. 500 Ygnacio Valley Road, Suite 450 Walnut Creek, CA P:

339 5.0 Appendix B: Promotional and Educational Materials

340 LivingWise Kit

341 Living Wise Education Materials

342 Weatherization Program Website Information

343 Bill Inserts That's right, OG&E is offering home energy efficiency upgrades for a limited number of qualified customers. All completely FREE. The OG&E Weatherization Program helps improve your comfort and reduce your energy costs, all while delivering increased protection for your residence. Some of the improvements may include: Adding attic insulation Sealing air leakage around windows and doors Caulking, air sealing and weather stripping throughout your home Installing energy-saving compact fluorescent lightbulbs (CFLs) For eligibility and participation, check the details on the back of this insert. Or contact us Monday through Friday, 8:30a.m. to 4 p.m.: with the help of myogepower. myogepower is a free online tool that gives you the ability to track and manage electricity use more efficiently, view current usage and see your estimated monthly bill. But, did you know it also includes a rate comparison tool? With the rate comparison tool, you can compare the different rates against the plan you are already participating in. This will allow you to see which rate is a better option and where you can see the most savings. For instance, this tool can show you how much you could have saved this past summer if you had switched to the SmartHours Price Plan. You can also see how shifts in your energy use could help you save even more. So, get to know your power by activating your free account now at myogepower.com and see how much you could save! OGl-E.

344 (Music up, under throughout) (ANNCR:) Radio Ad Script Reduce your home energy bills while increasing your comfort. AOG and OG&E are offering a weatherization program that provides FREE home energy efficiency upgrades to current customers in Arkansas. If your home was built before 2005, it may qualify for these upgrades - which include attic insulation, caulking, air sealing, weather stripping, and compact fluorescent lighting. To find out how you can benefit from these free home weatherization improvements, contact the Community Services Clearinghouse at (479) or visit aogc.com.

345 Letter to Customers OGl-E <Mr. John Q. Sample> <123 Main St.> <Oklahoma City, OK > Over $3,000 in Energy Efficiency Improvements at NO COST to You! Dear < John 0. Sample>: Would You Like to Get Something For Free From OG&E? It may sound too good to be true, but t here's no catch - real ly. OG&E's Weatherization Program in creases the comfort of your home and reduces your heating and coo ling b ills. We'll send a trained crew to complete these services, which have been performed for thousands of our customers. Weatherization se rvices may include: Adding insulat ion to t he attic Sealing around doors and windows Caulking, air sealing and weather stripping throughout your home Installing energy-saving compact fluorescent bulbs (CFLs) The services provided through t his program have an estimated value over $3,000, but are absolutely FREE to customers who meet the following criteria*: Own or rent a single-family or d up lex built prior to 2006 Current OG&E or AOG residential customer Please note this is a limited t ime of fer and will no longer be available after we reach our target goal this year. If you or someone you know could benefit from free home weatherization improvements, contact the Community Services Clearinghouse from 8:30a.m. to 4 p.m. Monday through Friday at The sooner you call, the quicker you can have your home weatherized. You'll be more comfortable and reduce your energy bills, both at no cost to you! Sincerely, Your friends at OG&E

346 Multi-Family Direct Install Program Website Information

347 Commercial and Industrial Programs Website Information SIGN IN CREATE AN ONUNE ACCOUNT CONTAG US CAREERS INVESTORS SYSTEM WATCH OGl-E Power at t/,e 1peeJ of Jife: I.. MyOnline Account Outage Center Support Safety& Education Save Energy & Money Corporate Commercial Lighting save Energy & Money 1 Arkansas- Business 1 commercial Lighting Save Energy & Money Residential Business Arkansas- Residential Arkansas - Business myogepower Wind Power Standard Offer CommerCial Lighting Multi-Family Efficiency Program SmartHours Weatherization Arkansas Business Customers Can See Serious Savings Receive a rebate from OG&E with the Commercial Lighting Program! Resources See your company's energy usage at myogepower Switch to SmartHours ~ OG&E's Energy Technology Center Need a great facility to host your next meeting? Get details here. ~ SmartHours Price Plans for Arkansas Business Customers Last year. over 80% of smarthours Arkansas business customers saved.

348 Commercial Lighting upgrades usually have the quickest payback for customers and may help decrease monthly bills while improving business efficiency. To help reduce the costs associated with purchasing and installing lighting upgrades. the OG&E Commeroal Lightmg program offers a rebate to all eligible customers who are upgrading their lighting systems. This rebate will usually cover 10-20% of the cost of the improvements. Most lighting improvements include: Installing energy efficient indoor and outdoor lighting Installing lighting controls Installing light emitting diode (LED) exit lights in both retrofit and new construction Rebates are based on kwh reductions from lighting upgrades. The OG&E Commercial Lighting program has paid almost 300 rebates since the program launch. Funding for lighting rebates is limited and may be stopped at any time. So start improving the energy efficiency of your business today! 8QQ!y for a rebate now. To take advantage of this program offer. please contact Robin Arnold at or by at arnoldrk@oge.com. Commercial Lighting rebates from OG&E Two well-known Fort Smith businesses have upgraded their lighting, reduced their energy costs and benefitted from OG&E's rebate program.

349 SIGNtH CREATEANONLINEACCOUHT I CONTACTUS CAREERS INVESTORS S'VSTEMWATCH OGlE. My Online Account Outage Center Support Safety & Education Save Eneq:y & Money Corpo.-ate Standard Offer Save Energy & Money Residential Business Atlcansas - Residential Arkansas - Business myogepower Wind Pow~r StarcMrd Offer Commerc al l.jghtmg Mult -Family Effidency Progra m Smart:Hours Weatherization An energy efficiency solution for your business OG&E has a program that will repay 10-20% of your equipment upgrades. Resources See yotk company's energy usage at myogfpower Switch to SmanHours ~ OG&Fs Ene<gy Technology Center Nted t srtiilt fac 1ty to host your next mttnnj 7 Get details here ~ SmartHoors Price Plans fo< Atlcansas Business Cust:om«s last ye r. over 80,.. o~ Sma"Hours Arlcanus business customers s ~ with n... erase of 0\ler S200. s.an up now. risk-free OG&E S offerh"'j r bate opponun1ty ~or alll rs coml"<"'erc,a; and ndustr l customers.n ArUnus throuzh the Standard Offer prozram. The Commeroal/1ndustnal Standard Of'er prozram providh lncentl~s ~or m. lnsta ation 0 ~. Wldt! ranz of me sures th t reduce customer enetjy cosu. reduce pe k dem nd. ndlor wve enero in non residenw! ~Kilitaes such as publtc uthority buildinzs, school.s. hosp ~Is nd other.ndustri l customers. tn th s prozram. ars individual customers. enerjy serv ce com pan u (ESCOs). and quaj fi~ contractors r t!i J ble or incent v payments of 50.12/kWh for enerjy eujc ency projects that S Jnifiundy reduce enerl)'. Tt'le S~ndard Offer prosram was des Jned co o"fer a l'le.x b e proaram to help r1er customers choeve enerjy and dt!mand uv1np. All commerci nd indvstnal cvstomers on the Pl nd LPl rates r e1j,ble or ttus prozram. Last year the averaze tebate tor customers parnopat nz n the Commerc,al and lndustnal Standard Of'l!r prozram was Cl ck ht f"t to t pp'y for 1 rtbftt now For additiontl nfonnt:ion. cont ct Robin A.rnohj t rnoldrk(jozt com« ,

350 Print Ads l l?~~ss=rs Installed First National Bank of Fort Smith invested in a bright future, and it paid off with a little help from OG&E. We helped them transform inefficient and outdated lighting into an energy efficient solution-for significant, long-term electricity savings. In fact, First National now has an estimated annual savings of over $44,000. ~,533 k\\11 1 Roductlon l!l~j83 Rebatl!s l ~~m1j12 Annual Savings The Story lntrodudng OG&E's Arkansas Commercial Ughtlng Program. We're providing lnci'qdiblelncentfves and rebates to commercial and Industrial customers who purchase and install energy e ff~elent indoor and outdoor lighting, lighting controls, sourcas, sensors and light emitting diode (LED) exit lights-both In retrofit and new construction applications. So why wouldn't First National Bank taikfl advant<lge7 They decided to update their century-old facility, Invest In newer, efficient technology and then cash In on the nasults. POSITIVE ENERGY TOO ITHi r For more information contact us at or 'lislt OGE.com/rl!bate

351 The Solution OG&E's expert team worked closely with First National to retrofit existing T12 fluorescent fixtures with high-efficiency T8 lamps and ballasts, while replacing all incandescent bulbs with hlgh-'lfficlency LED bulbs. Advanced fluorescent and LED technology means T8s can now produce the lumen output of T12s with significantly reduced energy consijtlptlon and LED lamps require significantly less power to provide the Incandescent equivalent The Success The lighting upgrade was a shining example of efficiency success, resulting In an estimated annua l energy savings of 631,533 kwh. That's like eliminating the annual greenhouse gas emissions from 85 passenger 119hlcles,* which means you're also ensuring a better environmental future. And of course, saving on energy bills for years to come. That's a program you can take to the bank. THEY SAID '7his is a project that you can fund using money saved on jour utility bill and the ((}bate for upgrading inefficient lighting. Short-term cost for long-term savings!' David Burton, AVP Purchasing Officer & Security Officer, First National Bank of Fort Smith Easy rebates. Short Rot. Efficiency partners to rely on. Join the growing list of customers receiving thousands In rebates plus reduced energy bils. Laarn more about the Commercial Lighting Program, plus other bright ideas, atoge.comltebate.

352 lmmedma.tfici ncy rob;rtec now, lower bills for yoars to come. OG8.E heps you ma1r.e easy busness energy efficiency impl CNel'l'lel'!:S for~ smlgs. These offumgs pay off in!he 5hort tsm too. wih hug& rebata. OG&E's Co~ I.Jg!lmg upgradft ha\oe!he quickest Ra. pkjs rebates amring 141 to 201(. ar!he mst And ow Standard Offer Progr.Jm allows OJstlllllE!r.; to rea:r.'l!f some ar!he casts 10 replace equ~rec:ljcing energy and peak oernands--f3the 1hal makilg inefficient and expal5m! repais. These rebates a1so CXM!f 141 to 2D'l' ar IN! msts on moto-s lers. a..mpl'liisiid u ~ l1ll. H'I.AC sys!!11m ~ u conditionlng and mrigl!r.ltjon rnproll'8fllents.. Last yo;><. aaacrun- ll'id<.ll;nds al dalorn., rgbiotds. not CD mon!dilho> rvdjdjor1 '"oncugy ~ Be ""OG& cf5ciclncy p;:nnor YDC.t1 dol:oy 1ho> noad lot f>cjq!nin1;ji poorw gan<orxjon b ~.. of.,._ ;nl tat :oct..n:..go ollong-.n. bat:lan lria liiicxrs: OG.canJ POSITIVE ENERGY TOOITHir

353 Big rebates right now. Lower bills for years. OG&E helps you make energy efficiency improvements for big savings- which pay off in the short term with huge rebates. OG&E's Commercial Lighting upgrades have quick ROI, plus rebates covering up t o 20% of the cost. And o ur Standard Offer Program rebates also cover up t o 20% t o re place motors, chillers, HVAC systems and much more. Thousands in rebates. Reduced energy bills. Be an OG&E efficiency partner for long-term, bottom line success: OGE.com/rebllte. POSITIVE ENERGY TOGET HER" OGlE. OGE.COM UNI_1.4 OG 274_0GE_C!IJ!tdd ~/\.Y, 1 o e11ej 1= - -=IIDII Make easy energy efficiency business improvements for significant savings and get instant cash payback. OG&E's Commercial Lighting upgrades have fast RO I, and o ur Standard Offer Program helps you replace motors, chillers, HVAC systems and m ore~all with rebates covering up to 20% on costs. It's efficiency to bank on. Be an OG&E efficiency partner: OG E.com/ rebate POSITIVE ENERGY TOG ETHER' OGlE. OOECOM

354 SMARTHOURS & ENERGY EFACIENCY PROGRAMS: Innovative rate plans offering more choices and savings, plus programs making energy efficiency upgrades easier and affordable can grow your bottom line. Designed for cornrnercial and industrial customers, reduce yoor energy bill by shifting some electric use to different times. And you can increase efficiency to lower your costs, too, as wei as address volatile prices, create new jobs, protect the environment and spur growth. Commercial Lighting Fast payback, plus great inoontm!s forcunomers installing energy-efficient indoor and outdoor lighting solutions, retrofit or new construction. Standard Offer Further cost-saving incentives for a wide range of efficiency improvents and solutions, all redudng customer costs and peak demands. Commercaal Tune-up 1m prow effi:iencies of commercial A/C, refngeration, wntilation systems and more (upgrades to tune-ups), with cost-cutting incentives. Time of Use Program Electricity costs more on s~nmer \Wekdays during peak demand periods. Shift energy out of 2 p.m. to 7 p.m. to off peak and stdrt saving. load Reduction Program Great savings for customers who curtail electrical load when asked. Just subscnbe to an amoum, then receive a discount for that reduction. Day-Ahead Pricing Proqram OG~E wid notify you the day before r ugher prices are posted, so adjust your demand the next day and save.,-.~~energy ~,('G- OG&E Business Advantage Group o.o"> i 1-tt-'s_y_o_u_r_o-ne ca_l_l so - lu_ti_on- fo_r_a_n_sw- er_s=:o_n_a_n_y_iss_u_e..: "{ BUSIN -S S ~ Call , Monday-FridayS a.m. to 5 p.m. 0 SUCC SS ~ Or at businessadvantage@oge.com. ~~ ~~ ~~ ~~.c-,clenc.'{ POSITIVE ENERGY T OGE'T HER'

355 Serious Savings OUR EXPERTISE CAN INCREASE THE ENERGY EFFICIENCY OF YOUR BUSINESS. THAT'S POSITIVE ENERGY TOGETHER-. Improving the efficiency of your business is one of the most cost-effective ways to grow your company's bottom line. Not only do these investments reduce costs and protect the environment, they also help addmss volatile energy prices, strengthen energy security, aeate new jobs and spur economic growth. OG&E has several flexible programs to help our business and industrial customers make energy efficiency upgrades easier and more affordable. Commercial lighting The Commercial Llghtirg program prollides incentiws to Arkansas commercial and industrial (C&i) customers v.t.o purchase and install energy c;ffkient indoor and outdoor lightlrg, hghting controls and light emitting diode (LED) BJtit lights In both retrofit and new construction applications. This program helps customers reduce monthly energy costs while reduclrg some of the Initial cost barrier. Commercia I buildings In the U.S. consume an estimated 18% of 1otal U.S. energy use and contribum ngarly 4% of global carbon dioxide emissions. Standard Offer The CommerclaVindustrlal Stardard Offer program provides incentil/95 fer the lnstallatbn of a wide ranga of measures that reduce customer energy costs, reduce peak demand, ancvor saw energy In non-resk:lentlal facfhties such as publk authority buildings, schools, hospitals and other industrial customers. In this program, large indfllidual cus1dmers, energy servke companies (ESCOsj, and qualified contractors are eligible for lncmtlve paymmts of $250/kW for energy effidency projects that significantly reduaa customer peak demand. The Stlndard Offer program was designed to offer a flexlble program to help larger customersachfev9 energy and demand savings. AH commercial and industrial customers on the PL and LR. rates are eligible for this program. Commercial Tune-Up The Commercial Tune-Up Offllr program provides incentlws to help customers by lmprollfng the effkiency of their commercial air conditioning, food servke, refi'lgeratlon ard.br ventilation systems to upgrade In efficiency or t!sie-up of commercial air conditbnlng.

356 No Peaking OUR RATE PLANS OFFER YOU NEW CHOICES AND MORE , SAVINGS, AS PART OF A POSITIVE ENERGY FUTURE. OG&E has several offerings that are specif~ea l ly designed for commercial and industrial customers. One of these programs may frt: your oompany's needs and help reduce your energy bill. Simply shifting some of your electric use to different times of day can save you money. Load Reduction Program Our performance-based Load Reduction program offers financial inc:entiws 1D oommercial and industrial customers that can curtail their electrical load when notified byog&e.. You can sobscribe 1D an amount of demand reduction, tell us how you ~h to be notified, and then rkeive a di9count for that promised reduction. Those who reduce their usage during a called ""l!nt receiw a discount on their next aectric bil. Time of Use Program It costa more for electricity in the summer during peak periods of high customer demand, trom 2 p.m. to 7 p.m. This program allows you to &aile when oosts are lower, or off peak.. Customers who are able to reduce or s hift their electricity use 1D off-peak hours on weekdays, June 1 through September 30, are rewarded with lower bills. It also helps OG&E manage its on-peak demand. Time of Use offers lo-r prices during off-peak time~. So enjoy off-peak pricing trom 7 at night until 2 p.m. the next day, plus au weekend. and national holidays. Day Ahead Pricing Program You can lower your energy costs by shifting your aectric use to the times of the day when rates are lower. When higher hourly prices are posted., cuslcmersenrolled in Day Ahead Pricing Cai!IO known as Reel T orne Pricing) can reduce their electric useduringthattime and saw monay. OG&E will notify you the day before, which allows you 1D adjust your demand the next day besed upon hourly pricing information. If demand reductions are not needed, stand:ard pricing is used. Business Advantage Group The BU!iiness Advantage Group offers a one~ I!IOiution to resolving i56ues and responding to inquiries tram bus ness custom us. These agl!nts receiw, track and communicate c:ustomer i56ues 1D the appropriate groups within OG&E and to customers. This dedicated group of represl!ntatives will also inform CUS1Dmers of important istues like rate changes, program offerings and other opportunities for customers. The Business Advantage Group can be reached by contacting the c ll center at 888-9' 747, Monday-Friday 8 a.m to 5 p.m., or via ema~ at '-.u. ~---t-n...,

357 Letter to Customers DriVEr compan~~om line, u p:~~ntinualy " 1 1'' 1 1' ' '1 1 '1 1 1 t Alllt)-~DIGrT999 Jorn 0. Sample OR VALU 0 CUSTOM tr 123.<lnyS11<10t Any Town US Dear You can get lmmedid.tfldancy.-.bate rraw, and lo-r bills foi' Y'lai'IID come, OG&E helps you male e;asy energy effi::lency l~ents IDyoorinfrastruc:ture forsigrificant 5a'Yings. Yfi1h offerings that payoff in the short tenn, too, indudingblg rebates. Canmerdal L.lghtingt OG~ r!'ijhtlrg l.f)grades have the quldcest ROI, plus rebate. COYering up ID 2~ of 1he CC6t. These ll!batbi hep yoo reocmtr some of 1he if5tallatfon on indoor and outdoor lghdng, mntrols, senscn and LED 8lcit lights in fetrofit and new oonstrucllons, which subsequently reduce yoor monthly mstl. And since fighting is185p01l!ible fer 1he vast majority of 1he electrical usage In ccmmertial buildings ~mast ~0 it makes a sigrificant inpact onyourl!lcpellse!6, so it's the best WJ1f to impact saw!gs. Stand.lrd Offer ProgramcCooling and ventilation canaooountforabout 25'lf. of your electric bill, so recownomeofthe up-front co;ts to replace equi>ment. rather than making ineffi::ient and expensiye repaf'5. Thase n:batas CXNer up td ::10'l' of the 006ts on new motors, ~ comprened 01ir syswms, HVAC syst~jm., ewn commercial 01ir conditioning and refrigeration upgr.~des. OG&E even offecs E!ldusiYe CXlrnfTlEf'dal and indus trial rate plans, from SmartHours for busln!!sses to Tme of Use and Load Reduction to Day Ahead Pricing. Sane are so easy that sirrply shiftir!'ij a lit1le enetgy use ID different times of the day can SiJ\E you big bucb. W.1,do a raw mmpariton a~ for you. For FREEl That\ what it means ID be an OG&E Effidency Partner. What\ mor~ yoo'l also helpdelay1he need for incremental po"'"ergeneratlonfcryea~s to come, which helps~ With rising enetgy oasts and with sevenll new EPA-mandlted expenses, an irlve!;tment in energy efficient technology or demand response rates can lower your bottom line. So let\ do it, together. Slll0el' 'ly, Your friends at OG&:E Contact our Busli\E!55 Advantage Grol.p at Q , ~kdays from 8 a.m. to 5 p.m. or emar For more lnfcrmation visit OGE. POSITIVE ENERGY T " l THiilr