Measure Cost Study (WO17) Task 5 Report: Recommended Data Collection Approaches (Deemed Measures)

Transcription

1 Measure Cost Study (WO17) Task 5 Report: Recommended Data Collection Approaches (Deemed Measures) Final Itron, Inc Broadway, Suite 1800 Oakland, CA (510) August 23, 2012

2 Table of Contents 1 Introduction Recommended Data Collection and Development Approaches by Measure Group Retail Unit Prices Large Sample of Actual Retail Price Observations Retail Price Build-up from Wholesale Prices Installation Labor Other Measure Cost Data O&M Labor and Materials RUL for ER Measures Lifecycle Setup Appendix A Attributes of NPD POS Data Appendix B - Sample Extract of C4A Invoice Data Appendix C - Final Disposition of DNV KEMA s Fall 2011 Shelf Surveys List of Figures Figure 2-1: Number of Rebates Issued under California s Cash for Appliances Program (source: CEC) List of Tables Table 2-1: Recommended Data Sources and Primary Data Collection Activities to Support Development of Large Samples of Actual Retail Price Observations by Measure Group Table 2-2: Recommended Data Sources and Primary Data Collection Activities to Support Development of Built-up Retail Prices by Measure Group Table 2-3: Recommended Data Sources and Primary Data Collection Activities to Support Development of Installation Labor Hours by Measure Group Table 2-4: Deemed Measures with Incremental O&M Costs Table 3-1: Equipment Attributes Available for NPD s Appliance POS Data Table 3-2: Equipment Attributes Available for NPD s Electronics POS Data Itron, Inc. i Table of Contents

3 Table 4-1: Sample Extract of California Cash for Appliances Program Invoice Database Table 5-1: Distribution of Completed Store Visits by Chain/Independent, Participating/Non-Participating Stores, Channel, and IOU (source: DNV KEMA) Table 5-2: Number of Total Advanced and Non-Advanced Lamps by Channel and Detailed Lamp Type (source: DNV KEMA) Itron, Inc. ii Table of Contents

4 1 Introduction This memorandum represents the primary deliverable for Task 5 of the Measure Cost Study (Work Order ITRON017). As presented in the approved research plan, the primary objective of Task 5 is to develop detailed data collection strategies and plans for each in-scope deemed measure group. As described in the research plan, the primary deliverables associated with Task 5 are as follows: A comprehensive memorandum presenting the preferred data collection strategies for all in-scope measure groups, including but not limited to explicit identification of data to be collected from subcontractors, data to be collected by Itron and/or KEMA staff, data to be purchased, and data to be collected from secondary sources. The limitations and key analysis issues associated with each preferred data collection strategy will be delineated and previewed (draft and final) Task orders for all data collection to be conducted by subcontractors, including written scopes of work, budgets, and schedules (draft and final) This memorandum is structured to satisfy the first of these two requirements. The second requirement will be fulfilled by having the subcontractors prepare detailed budget estimates for completing the recommended data collection approaches described in this memo. Each subcontractor s draft budget and scope will then be optimized to be consistent with the final priority rankings established in Task 4. Revised subcontractor budgets and scopes will form the basis of each final task order, which will be structured as phased budget authorizations linked to a series of interim deliverables. Itron, Inc. 1-1 Introduction

5 2 Recommended Data Collection and Development Approaches by Measure Group In this section, we present and describe the specific data collection and development approaches for each in-scope deemed measure group that the study team believes will allow robust estimation of ex ante incremental measure costs. These data collection and development approaches are based on the study team s assessment of the quality and applicability of costrelated data that can be readily and cost-effectively acquired by Itron and/or its subcontractors. These data collection and development approaches were also designed to meet the data collection and analysis objectives delineated previously in the Task 4 memorandum, which themselves are based on a detailed review and assessment of the data sources and methods underlying the current set of incremental cost estimates in DEER and the IOU workpapers. These objectives are: Use substantially larger sample sizes from highly representative sample frames Increase use and improve specification of regression-based cost models Use systematic, independent validation of results Incorporate anticipated interactions with future codes, standards, and labeling programs Develop additional lifecycle cost data Streamline data acquisition and development for future updates In the remainder of this section, we present and describe each of the recommended data collection activities and sources in detail. We describe the strengths, limitations, and key analysis issues associated with each preferred data collection strategy and identify the activities to be conducted by specialized subcontractors and those to be conducted by Itron. It is important to note that the recommended data collection activities described below will be designed to allow estimation of both full measure cost and incremental measure cost relative to either a code-defined baseline or a market average baseline. The proposed data collection activities for each measure group have been designed around the types of analysis we believe will lead to the most robust cost estimates based on factors such as delivery channel and the various cost influencing factors identified in Task 4. The primary analysis methods will be described at a high level in the sections that follow. Analysis methodologies specific to each Itron, Inc. 2-1 Recommended Data Collection

6 measure group will be developed and tested in parallel to data collection activities. Recommended cost cases and incremental cost analysis approaches for each measure group will be defined in the deliverable for Task 7 in close coordination with the DEER team. 2.1 Retail Unit Prices To develop estimates of the average retail unit price for in-scope deemed measures, we recommend two general approaches to data collection and development. For mass market measures that are primarily sold directly to final consumers through retail channels, we recommend collecting and developing large samples of actual retail price observations at the point of sale. For measures that are procured and sold to consumers primarily or exclusively via contractors, we recommend using a retail price build-up approach where unit price data is collected at the wholesale level and supplemented by explicit estimation of bulk purchase discounts, contractor mark-ups, warranties, and other factors that determine the average retail price faced by final consumers. Within these two general approaches to data collection and development, we identify and describe the specific data sources and data collection approaches that the study team recommends leveraging for each in-scope deemed measure group in the subsections below Large Sample of Actual Retail Price Observations For mass market measures that are primarily sold directly to final consumers through retail channels, we recommend collecting and developing large samples of actual retail price observations at the point of sale. 2 Collecting and developing such samples allows incremental costs due to efficiency to be estimated using regression-based cost modeling (also known as hedonic price modeling). As discussed previously in the Task 4 memorandum, regression-based cost modeling has many attributes that make it highly appealing for incremental cost estimation. First and foremost, it allows incremental cost estimates to be explicitly controlled for costinfluencing factors that are not related to efficiency performance. Second, these models allow incremental costs to be estimated across a continuum of technology specifications and are thus 1 Note that there are a few measures which are strictly services, and as such their incremental costs are entirely the labor costs associated with the service and do not include any retail technologies or materials. Specifically, the in-scope deemed measures that are strictly services are: refrigerator recycling, freezer recycling, room air conditioner recycling, AC coil cleaning, delamping, and BMS programming. 2 In support federal appliance standards rulemakings, the USDOE uses a manufacturing cost approach to estimate incremental costs due to efficiency improvements for mass market technologies. This approach involves isolating the specific equipment components that determine efficiency performance and using engineering analysis to estimate the incremental production costs of those components. This engineering analysis is then followed by a retail markup analysis to arrive at average retail price. This approach is well-developed and has proven to be acceptable to the USDOE s stakeholders but is also time and cost-intensive. In this respect, it is largely impractical and well beyond the resources of this study. Itron, Inc. 2-2 Recommended Data Collection

7 inherently flexible and applicable to both program-level and measure-level planning activities. Third, and perhaps most importantly, regression-based cost models allow for the explicit quantification of the uncertainty associated with the result for each independent variable, which is strictly not possible when using weighted or simple averages. In addition to the advantages provided by applying regression-based cost modeling to large samples of actual retail price observations, large samples of point of sales (POS) data for some measures can be readily purchased from third party marketing firms (e.g. ACNeilsen, Activant, NPD) which makes it possible to conduct regular, targeted updates for those measures. Indeed, large POS samples have been used as the primary datasets in nearly all of the Residential Market Share Tracking (RMST) studies sponsored by the CPUC and the IOUs since Conversely, there are several challenges associated with the approach of using large samples of actual retail price observations. The first of these challenges is that the price data must be recent enough to be relevant to the analysis, i.e. data collection is somewhat constrained to prices paid in the most recent 1-2 years. This is especially true for technologies that change rapidly, such as TVs and computer displays. Second, the use of large samples of actual retail prices often needs to be complemented by sales volume (or relative sale volume) data in order to properly account for any significant price differences observed across different retail channels. Finally, such data sets often need to be corrected for seasonal pricing (e.g. holiday sales), inflation, and changes in producer prices (e.g. for labor and commodities). The first two challenges described above are essentially additional data collection requirements. The last challenge described is primarily an analytic challenge for which econometric methods have already been developed and applied in a wide variety of contexts. The study team has identified several viable data sources that can be readily leveraged to develop large samples of actual retail price observations for specific measures, as well as a set of primary data collection activities that can be used to develop such samples and/or validate results. Table 2-1 shows the specific measure groups for which the study team recommends developing large samples of actual retail price observations and identifies the specific data sources and primary data collection activities recommended for each such measure group. Itron, Inc. 2-3 Recommended Data Collection

8 Table 2-1: Recommended Data Sources and Primary Data Collection Activities to Support Development of Large Samples of Actual Retail Price Observations by Measure Group Sector End Use Tech Group Measure Group Data Sources Residential Lighting Interior lighting CFL lamps IOU program data, Residential Lighting Interior lighting CFL fixtures POS data Residential Lighting Exterior lighting CFL lamps (ACNeilsen/Activant), Retail shelf surveys, Residential Lighting Exterior lighting CFL fixtures Supplier interviews Residential Appliances Laundry Clothes washers Residential Appliances Kitchen Dishwashers Residential Appliances Cold storage Refrigerators Residential Appliances Cold storage Freezers Residential Appliances Laundry Clothes dryers POS data (NPD) Residential Electronics Office Copier Residential Electronics Office Desktop computers Residential Electronics Office Monitors Residential Electronics Other plug load Televisions Residential Lighting Interior lighting LED Residential Lighting Controls Photocell Residential Lighting Controls Occupancy sensors Residential Water Heating HW distribution Pipe insulation Residential Water Heating HW distribution Faucet aerator Retail shelf surveys C&I Water Heating Liquid circulation Pipe insulation C&I Water Heating Liquid circulation Faucet aerators C&I Water Heating Liquid circulation Low-flow showerheads Residential Electronics Office PC power management Residential Electronics Other plug load Plug load sensors Web price search Data Validation Approach Web price search CA SEEARP invoice data Retail shelf surveys Web price search none Itron, Inc. 2-4 Recommended Data Collection

9 In the subsections below, we describe each of the recommended data sources and primary data collection activities shown in Table 2-1 in more detail and the rationale for the study team s recommendations. Note that the first three data sources described below are specific to interior and exterior CFLs. In these particular cases, there are multiple sources of large volumes of actual retail price observations. However, from Itron s previous Residential Market Share Tracking Studies (RMST), it is well established that retail prices for CFLs vary significantly across retail channels for identical products. In order to develop robust estimates of the average incremental cost of CFLs, it is therefore necessary to also acquire comprehensive data on relative sales volumes across all retail channels. Unfortunately, there is no single data source that has comprehensive sales data for CFLs from each retail channel. 3 In this sense, developing retail channel sales weights for CFLs requires piecing together best available information from several sources. The recommended data sources and data collection activities for CFLs described below were developed by DNV KEMA on behalf of the Itron study team in order to leverage their related data collection activities and analysis currently being conducted for WO28, as well as their previous data collection and development activities conducted for their evaluations of IOU residential lighting programs in the and program cycles. IOU program data The IOUs provide information on all products discounted through the current upstream lighting program. This information includes model number, manufacturer, retailer, product style, wattage and lumens, and rebate and pricing information. The study team (via DNV KEMA) has direct access to program tracking data from as far back as These data will support the development of retail channel weights for interior and exterior CFLs. POS data (ACNeilsen, Activant) Since the late 1990s, Itron has evaluated, procured, and analyzed POS data from various third party marketing firms as the primary datasets to support near annual updates to the RMST studies for lighting and appliances. For the current portfolio of EM&V studies, Itron plans to purchase similar datasets for 2010 and 2011 as part of the current update to the RMST studies under WO23 (which will also be leveraged for analyses being conducted in WO28). Specifically, Itron plans to purchase a POS dataset for CFLs from ACNeilsen that covers the food store, drug store, and mass merchandiser retail channels, and an analogous POS dataset from Activant that covers hardware stores which collectively account for all of the major retail channels for CFLs with the exception of national chain home improvement centers (such as Home Depot and Lowe s) and small mom and pop grocery stores. 3 As noted in Itron s 2007 RMST Lamp Report, national chain home improvement stores stopped providing their POS data to third party marketing firms starting in Itron, Inc. 2-5 Recommended Data Collection

10 In past RMST efforts, Itron has attempted to identify products discounted through the IOU upstream lighting programs. However, there is no flag in the ACNeilsen and Activant POS databases that isolates within-program versus outside-program sales. In the context of the MCS study, the ACNeilsen and Activant POS data will contribute to the development of retail channel weights for interior and exterior CFLs. Supplier interviews DNV KEMA has conducted hundreds of in-depth interviews with lighting suppliers, i.e. manufacturers, lighting buyers, sales representatives from national chain retailers, and retail store managers. The results from these interviews have been used since 2004 to estimate total CFL sales by retail channel, including estimates from within and outside of IOU programs. The supplier interview data provide a good high-level understanding of retail channel weights for interior and exterior CFLs and will supplement the IOU program and POS datasets. POS data (NPD) The NPD Group collects POS data for a large panel of major appliance and electronics retailers that cover roughly 80% of those respective retail markets. 4 Included in these POS datasets are nearly all of the appliance and consumer electronics measures in the scope of this MCS update. Specifically, the NPD POS data include clothes washers, dish washers, refrigerators, freezers, room air conditioners, clothes dryers, televisions, desktop computers, monitors, and copiers. In addition to brand, model number, unit price, and relative unit sales for all models sold by their partner retailers, the NPD POS data also include fields for an extensive set of variables that are critical to developing robust cost models and isolating price differences due strictly to efficiency performance. For example, in the case of refrigerators, each NPD POS record includes fields for color, depth, width, height, capacity (volume), door dispenser options, icemaker options, ENERGY STAR qualification, rated annual kwh, number of doors, and door type and configuration. A full list of the product attribute variables available in NPD s POS data for appliances and electronics measures are provided in Appendix A. Discussions with NPD revealed that, due to NPD s confidentiality agreements with their appliance retail partners, the unit price and relative sales volume data for major appliances that can be delivered to Itron would be national averages, not California-specific averages. However, NPD s confidentiality constraints do not restrict it from weighting the national averages by the 4 NPD s POS data for appliances includes the following retailers: Abt TV & Appliances, amazon.com, American TV, Bernies, Best Buy, BJ s Wholesale Club, Bloomingdale s, Boscov s, Dillard s, Fred Meyer, HH Gregg, JCPenney, Kmart, Kohl s, Lowe s, Macy s, Meijer, Nebraska Furniture Mart, PC Richard & Sons, Pamida, RC Wiley, Sears, Shopko, Target, Ultimate Electronics. NPD s POS data for electronics includes the following retailers: AAFES, ABT TV & Appliances, amazon.com, Apple Store, Best Buy, BJ s Wholesale Club, Dell, NewEgg.com, Office Max, Office Depot, PC Richards, Staples, RadioShack, Ritz Camera, Sears, Target Itron, Inc. 2-6 Recommended Data Collection

11 actual CA-specific mix of retailers. It is also possible to apply a CA-specific CPI or other appropriate index to further weight the national averages to account for CA-specific inflation and other macroeconomic factors that influence consumer prices for appliances in California. NPD has indicated that they have not observed CA-specific unit price differences for appliances and electronics that cannot be explained by CA macroeconomic factors. This approach is analogous to the procedure used in previous RMST efforts where the original POS dataset was composed of national or regional averages and then adjusted using CA-specific corrections for retailer mix (and their relative sales volumes). For the electronics POS data, NPD does not face the same confidentiality constraints and can provide POS to Itron specifically for California retailers. The electronics POS data is also available for each of the eight individual Designated Market Areas (DMAs) that NPD tracks in California. It should be noted that, in coordination with WO34 (BCE), KEMA has already acquired NPD s complete POS dataset for televisions, so the remaining potential data purchase would only need to include desktop computers, monitors, and copiers. CA SEEARP data As part of the USDOE s State Energy Efficiency Appliance Rebate Program (SEEARP), the CEC established the California Cash for Appliances (C4A) program that launched on April 22, 2010 and closed on December 31, Under this program, roughly 178,000 rebates were offered to customers to replace existing, inefficient equipment with new ENERGY STAR qualified units, including a host of technologies in the scope of the MCS update, including clothes washers, dishwashers, refrigerators, freezers, room air conditioners, gas furnaces, central heat pumps, central air conditioners, gas storage water heaters, solar thermal water heaters, and heat pump water heaters. The CEC recently released the final number of rebates issued under the C4A program, which are summarized by technology in Figure 2-1 below. Itron, Inc. 2-7 Recommended Data Collection

12 Figure 2-1: Number of Rebates Issued under California s Cash for Appliances Program (source: CEC) Through the RFQ process for this study (Task 2), D&R International informed the Itron study team of the forthcoming availability of the invoice data from the C4A program as well as other SEEARP programs (for which D&R is one of several firms evaluating the program impacts at a national level). The Itron team then inquired directly with the CEC s program manager and consequently acquired the entire C4A invoice dataset from the CEC. Each invoice record includes brand, model number, purchase price, rebate amount, date of purchase, zip code of purchase/installation, as well as information on the replaced equipment. A sample extract of the C4A invoice database is provided in Appendix B. As Figure 2-1 shows, this data set contains roughly 178,000 records and thus represents a very large sample of actual retail price observations. Additionally, since the dataset contains model numbers for each line item, it can be easily expanded to include the equipment characteristics (e.g. capacity and efficiency rating) required to conduct robust hedonic price modeling. However, due to the C4A program design, this dataset does not contain any retail price data for baseline efficiency units. Therefore, the C4A invoice dataset cannot be used as the exclusive primary data source on which to develop incremental cost estimates. That said, the C4A data still have significant value as a potential supplemental primary data source or, at a minimum, an important data source for validating NPD s POS data for appliances. 5 5 Unfortunately, the some of C4A data for HVAC and water heating equipment include installation labor and delivery charges (aggregated with retail unit prices), and these records cannot be parsed out from those that do not. Therefore, the C4A data for HVAC and water heating equipment cannot be used to validate the study team s estimates of full or incremental measure cost. This is not the case with the C4A data for appliances, where the invoices only include retail unit prices. Itron, Inc. 2-8 Recommended Data Collection

13 Retail shelf surveys One of the most direct ways to collect and develop large samples of retail price observations for mass market goods is to conduct in-store retail shelf surveys, i.e. visits to a statistically significant sample of retail stores to collect information on the specific products stocked in those stores. This type of data collection has been a central part of previous MCS efforts in California as well as other CPUC-funded market characterization and baseline studies. Indeed, in the specific case of upstream lighting measures, DNV KEMA has conducted four different waves of retail shelf surveys for lighting products since 2008 and recently completed another large shelf survey of retail lighting stores in November 2011 as part of WO13 and WO28. The final disposition of stores surveyed and unit prices observed is provided in Appendix C. The Itron study team intends to integrate DNV KEMA s most recent shelf survey data as a key supplement to the Activant/Nelisen POS data for CFLs and the IOU program tracking databases. One of the main challenges associated with using this data collection approach, however, is that it does not provide information on relative sales volumes across competing brands and manufacturers. This can be problematic when observed retail prices vary significantly within a certain product class, as it leads to large standard deviations for the estimated variable (incremental cost). In this sense, it is often not ideal to rely exclusively on retail shelf survey data for incremental measure cost analysis. Conversely, retail shelf surveys can be an effective and appropriate primary data collection approach when positioned as a way to strategically supplement other primary data collection activities or to validate measure costs estimated using other data sources. For this MCS update, we recommend developing and implementing retail shelf surveys primarily as a means to validate NPD s POS data for appliances and electronics. Validation through shelf surveys would both verify that the use of national averages (with California-specific retailer weights and CPI corrections) produces robust estimates of California-specific retail prices and ensure that NPD s POS data is representative of the prices at retailers known to be outside their sample. 6 We also recommend using retail shelf surveys as the primary data collection approach for a few specific residential measures, namely residential outdoor lighting controls (photocells and motion sensors), faucet aerators, low-flow showerheads, and hot water pipe insulation. For these specific measures, initial investigations suggest either limited variance in retail prices across brands (faucet aerators, pipe insulation) and/or a relatively limited number of competing products in the market (photocells and motion sensors) situations where the main challenges associated with relying on retail shelf surveys as the primary data source are largely mitigated. 6 NPD claims that their POS data covers > 85 percent of all appliance/electronics retailers, but the sample does not include a limited number major retailers such as Wal-mart and Home Depot. Itron, Inc. 2-9 Recommended Data Collection

14 Web price search Another direct and low-cost way to collect and develop large samples of retail price observations for mass market goods is to conduct online price searches, either using web crawler routines or leveraging the many publically-available search engines. As with retail shelf surveys, one of the main challenges associated with using this data collection approach is that it does not provide information on relative sales volumes across competing brands and manufacturers. In this sense, it is often not ideal to rely exclusively on web-based price searches for incremental measure cost analysis. Conversely, web price searches can be an effective and appropriate primary data collection approach when positioned as a way to validate measure costs estimated using other data sources or for measures where the primary retail channel is in fact online retailers (as opposed to brick and mortar stores) and competing product offerings are limited. For this MCS update, we recommend using web price searches to validate the final incremental cost estimates for CFLs and the retail shelf survey data and resulting cost estimates for residential outdoor lighting controls (photocells and motion sensors), faucet aerators, low-flow showerheads, and hot water pipe insulation. 7 We also recommend using web price searches as the primary data collection approach for two specific nonresidential measures plug load sensors and network power management software. For plug load sensors (sometimes called smart plug strips), the commercial product offerings are currently very limited, and it is unclear whether a retail shelf survey approach would successfully capture all products currently available in the market. 8 For network power management, the number of commercial product offerings is also very limited and distribution appears to be either direct from the software provider or via online retailers. 9 Web-based data collection will be most effective for these measures if the IOUs can provide detailed information on the specific products being incentivized Retail Price Build-up from Wholesale Prices For measures that are procured and sold to consumers primarily or exclusively via contractors, we recommend using a retail price build-up approach where unit price data is collected at the wholesale level and supplemented by explicit estimation of bulk purchase discounts, contractor mark-ups, warranties, and other factors that determine the average retail price faced by final consumers. This approach closely mirrors the equipment and project pricing practices used by 7 Note that this validation exercise is not meant to be a large-scale endeavor but a low-cost, systematic means by which to independently verify that the cost models are calibrated accurately. 8 See the most recent market assessment for plug load sensors conducted by Ecova (formerly Ecos) for a list of known products commercially available in the U.S.: Smart-Plug-Strips-DRAFT-Jul2009-v2x.pdf 9 The EPA maintains a list of commercially available network power management software packages in addition to its own free EZ GPO software tool in support of its ENERGY STAR Low Carbon IT campaign: Itron, Inc Recommended Data Collection

15 contractors, energy service companies, and program implementers who procure and install energy efficiency measures on behalf of customers. There are several important advantages to using a retail price build-up approach. First, because the population of wholesale equipment distributors is relatively small, the sample sizes required to form a representative share of the total market are also small, especially compared to the sample sizes required when sampling the population of final customers. Second, for measures typically procured and installed by third parties, the wholesale markets are highly competitive (e.g. packaged DX) and unit wholesale prices tend not to vary significantly across manufacturers for similar products, thus reducing and sometimes eliminating the need to acquire data on relative sales volumes in order to develop weighted average wholesale prices. Similarly, some wholesale markets have a very limited number of manufacturers (e.g. remote refrigeration), which also serves to reduce the complexity and data required to develop weighted average wholesale prices. Finally, and perhaps most importantly, this approach allows retail price estimates to be explicitly controlled for high volume vs. low volume purchasing practices, variation in contractor markup percentages, warranties, and other pricing factors which can influence variations in final retail unit prices as much or more than variations in wholesale unit prices. Of course, there are also several challenges associated with the approach of building up retail unit price estimates from wholesale prices. The first of these challenges is that this approach requires access to multiple distributor price lists, which are not often readily available and shared with the general public. Second, actual bulk purchase discounts and contractor markup percentages are highly variable and depend on distributor inventories and the specific relationship between a given distributor and contractor. Addressing this challenge requires leveraging a large volume of actual project records to develop robust averages. A third challenge is trying to account for both recent and historical changes in wholesale equipment prices. While there have recently been large fluctuations in many commodity prices (e.g. copper), initial discussions with the subcontractors have indicated that wholesale unit prices for equipment change much more slowly than other project material costs like those for piping, ducting, and wiring. Because these types of material costs cancel in an incremental cost analysis, the largest potential source of price fluctuation is obviated. The study team has identified five firms that regularly specify, procure, and install energy efficiency technologies on behalf of customers and have ready access to multiple distributor price lists for the measures of interest. These firms also have archives of a large number of actual project cost records. Additionally, the study team has identified publically-available raw data and a set of primary data collection activities that can be used to supplement the wholesale unit price data and/or validate results. Table 2-2 shows the specific measure groups for which the study Itron, Inc Recommended Data Collection

16 team recommends using a retail price build-up approach and identifies the specific data sources and primary data collection activities recommended for each such measure group While we are aware that the IOUs have extensive archives of rebate invoices for many of these measure groups, those invoices only provide unit price data for high-efficiency technologies, not baseline-efficiency technologies. With the exception of add-on measures like HVAC QM, we feel it is important to obtain both baseline and efficient case measure costs from a single source in order to establish the most robust incremental cost models. That said, IOU data may be appropriately used for data validation purposes. Itron, Inc Recommended Data Collection

17 Table 2-2: Recommended Data Sources and Primary Data Collection Activities to Support Development of Built-up Retail Prices by Measure Group Sector End Use Tech Group Measure Group Data Sources Data Validation Approach Residential HVAC DX Split CACs Residential HVAC DX Split HPs Residential HVAC Space heating Furnaces Residential HVAC Space heating Gas boiler Residential Water Heating Water heaters Heat Pump WHs Residential HVAC Air distribution Whole house fans Residential HVAC Air distribution Fan VSDs Residential HVAC Air distribution Fan motors Residential HVAC Air distribution Air filter alarm controls DEG (via AEP affiliate) Residential HVAC Controls Programmable thermostat Residential Building Shell Insulation Batt insulation Residential Building Shell Insulation Blow-in insulation Residential Building Shell Roof Cool roof Residential Building Shell Windows Windows Residential Water Heating Water heaters Storage WHs Residential Water Heating Water heaters Tankless WHs Residential Water Heating Controls Water heating controller Residential Pool Pump Pool pump C&I Lighting All All but CFL lamps, fixtures C&I HVAC All All but QM C&I Pool All All C&I Building Shell All All EMCOR, QuEST C&I Water Heating All All but pipe insulation, faucet aerators, low-flow showerheads C&I Process All All C&I Food Service All All ERS C&I Refrigeration All All VACOM IOU invoice data Actual project records, IOU invoice data, and/or representative bids Itron, Inc Recommended Data Collection

18 As Table 2-2 shows, the study team recommends leveraging the market position and expertise of five specific firms Davis Energy Group (DEG), EMCOR Energy Services (EMCOR), Quantum Energy Services & Technologies (QuEST), Energy & Resource Solutions (ERS), and VaCom Technologies (VACOM). These five firms were selected as a result of a Request for Qualifications (RFQ) that was developed and released by the study team under Task 2 of the approved research plan. 11 In all, 18 firms responded to the RFQ, and the study team vetted each firms stated qualifications and conducted multiple follow-up calls to determine each firm s specific areas of expertise, including but not limited to: established in-house cost estimation resources, processes, and expertise; access to distributor price lists; accessible archives of actual project records involving measures in the scope of this MCS update; and established relationships with a large network of contractors. Each firm s qualifications were then matched to the areas of greatest need within the specific measure scope and objectives of the MCS update. This process yielded the following specific recommendations: using DEG to support residential HVAC, water heating, and building shell measures; using EMCOR and QuEST to support nonresidential lighting, HVAC, water heating, and building shell measures; using ERS to support commercial food service and industrial process measures; and using VACOM to support commercial refrigeration measures. Each of these firms has extensive knowledge of the respective energy technology markets and established relationships with equipment manufacturers and vendors that can be leveraged to acquire multiple distributor price lists and develop a representative sample of wholesale unit prices for those respective measure groups. Note that due to the breadth and diversity of the nonresidential lighting, HVAC, water heating, and building shell measures in the scope of this MCS update, the study team recommends leveraging both EMCOR and QuEST s relationships with distributors and equipment vendors in order to ensure that an adequate number of distributor price lists can be acquired for those specific measure groups to support robust estimates of average wholesale unit prices. To develop estimates of average contractor markups, the study team recommends leveraging each firm s relationships with contractors and builders to solicit a representative sample of itemized project bids (i.e. with equipment separate from labor) for clearly defined prototypical retrofit projects involving the specific measures of interest. By using the wholesale unit price estimates derived from the distributor price lists, the study team will then be able to explicitly back out the contractor markups for each type of equipment included in each bid The final RFQ is available at: %20Final_ docx 12 Under this approach, warranties and other markups besides contractor profit markups would be included in the estimate. Itron, Inc Recommended Data Collection

19 To validate the final built-up retail unit price estimates, the study team recommends leveraging a combination of the C4A invoice data (for central air conditioners, air-source heat pumps, furnaces, and heat pump water heaters), each firm s records of completed projects, and/or IOU current project records involving the specific measures of interest. In cases where the available project records are not sufficient or comprehensive enough to support validation for a particular measure, we recommend leveraging the appropriate firm s relationships with contractors and builders to solicit a sample of representative project bids (separate from the exercise described previously) as a means of validating the built-up retail unit price estimates. 2.2 Installation Labor For deemed measures that are installed directly by the final customer, estimating full measure costs only requires estimating average retail unit prices. Examples of such measures include residential lighting, appliances, and electronics, as well as some residential water heating measures such as faucet aerators and low-flow showerheads. However, for deemed measures that are installed by third parties (e.g. contractors) on behalf of final customers, estimating full measure costs requires estimating average installation labor costs in addition to average retail unit prices. For this MCS update, the study team recommends using RS Means Cost Data publications as the primary source for average installation labor rates ($/hr) associated with different types of retrofit projects. The RS Means publications are a recognized source for developing construction cost estimates and a common benchmark used by contractors, developers, and builders. In the context of this study, the primary advantages of using the labor rates published by RS Means are that the labor rates are internally consistent (thereby reducing systematic bias in labor cost estimates), easily customizable to specific regions and locations via application of RS Means city cost indices and location cost factors, and consistent with the labor cost estimation procedures used by many contractors and implementers. IOU invoice histories will be used as a means of validating RS Means labor rate information. However, the RS Means estimates of the installation labor hours required for construction and retrofit projects are often too generic to be reasonably representative of energy efficiency interventions. For example, while RS Means provides labor hour estimates for the installation of various types of cooling systems, it does not provide labor hour estimates for several types of HVAC control measures (e.g. demand-control ventilation) and indeed all of the lighting and HVAC maintenance measures in the scope of this MCS update. The study team therefore proposes developing original estimates of average installation labor hours for each deemed measure in the scope of the MCS update that is typically installed by a Itron, Inc Recommended Data Collection

20 third party. Table 2-3 provides an overview of the data sources and data collection and validation approaches proposed by the study team for each in-scope deemed measure group. Itron, Inc Recommended Data Collection

21 Table 2-3: Recommended Data Sources and Primary Data Collection Activities to Support Development of Installation Labor Hours by Measure Group Sector End Use Tech Group Measure Group Data Sources Residential Lighting All All Residential Appliances All All but recycling Residential Electronics All All Residential Water Heating HW distribution All C&I Water Heating Liquid circulation Faucet aerators C&I Water Heating Liquid circulation Low-flow showerheads Residential HVAC All All Residential Water Heating Water heaters All Residential Water Heating Controls All Residential Building Shell All All Residential Pool Pump Pool pump C&I Lighting All All but CFL lamps, fixtures C&I HVAC All All C&I Pool All All C&I Building Shell All All N/A CATI-based survey of IOUapproved contractors Data Validation Approach N/A DEG project records and/or representative bids; IOU invoices EMCOR/QuEST project records and/or representative bids; IOU invoices All but pipe insulation, faucet C&I Water Heating All aerators, low-flow showerheads In-depth interviews w/contractors C&I Process All All ERS project records C&I Food Service All All C&I Refrigeration All All and/or representative bids; IOU invoices VACOM project records and/or representative bids; IOU invoices Itron, Inc Recommended Data Collection

22 Below we describe each of the recommended data sources and primary data collection activities shown in Table 2-3 in more detail and the rationale for the study team s recommendations. For residential HVAC, water heating, pool pumps, and building shell measures, as well as nonresidential lighting measures, the study team recommends conducting telephone surveys with large samples of the respective contractor populations in California using Itron s Computer- Aided Telephone Interview (CATI) survey center. Due to the relative homogeneity of new HVAC, water heating, pool pumps, and building shell equipment and materials installed in the residential sector (especially compared to those installed in the commercial and industrial sectors), the study team believes that it is possible to construct a representative set of site conditions and projects that can be effectively communicated in a CATI survey format in order to solicit installation labor hour estimates. For most nonresidential lighting measures (e.g. general service linear fluorescent lamp/ballast upgrades and fixture replacements), the study team also believes that it is possible to construct a representative set of site conditions and projects that can be effectively communicated in a CATI survey format. We acknowledge, however, that the site conditions for certain types of nonresidential lighting projects (e.g. HID fixtures and high-bay lighting applications) are too heterogeneous to be reasonably represented and communicated in a CATI survey format. For those types of nonresidential lighting measures, the study team recommends using in-depth interviews with contractors (described further below). The sample frames for the CATI surveys will be developed using a combination of the approved contractor lists maintained by each of the IOUs and the list of contractors that have submitted relevant incentive applications for the current program cycle (through Q4 2011) as shown in the SPT. The study team will leverage the expertise and experience of DEG (for residential HVAC, water heating, pool pumps, and building shell measures) and QuEST and EMCOR (for nonresidential lighting measures) to ensure that the prototypical site conditions and project types are defined in terms and metrics appropriate for each respective contractor population and consistent with the manner in which actual projects are described and defined when contractors produce cost estimates for competitive bids. For all other nonresidential measures, the study team recommends conducting in-depth interviews with contractors. Due to the diversity of equipment specifications and site conditions involved in these types of efficiency projects, it is difficult if not impossible to construct a representative set of site conditions and projects that can be effectively communicated in a CATI survey format. However, the study team believes that it is possible to leverage our subcontractors existing relationships with contractors to solicit reasonable installation labor estimates in an in-depth interview format, where a variety of site conditions and other sitespecific factors could be explored. To be successful, the study team recommends that these indepth interviews be conducted by qualified members of each subcontractor s staff who have direct experience in both the project types of interest and the contractors being interviewed. Itron, Inc Recommended Data Collection

23 As with the CATI surveys, the sample frames for the in-depth interviews will be developed using a combination of the approved contractor lists maintained by each of the IOUs and the list of contractors that have submitted relevant incentive applications for the current program cycle (through Q4 2011) as shown in the SPT. The study team will leverage the expertise and experience of QuEST/EMCOR (nonresidential lighting, HVAC, water heating, and shell), VACOM (commercial refrigeration), and ERS (commercial food service and industrial process) to ensure that the prototypical site conditions and project types are defined in terms and metrics appropriate for each respective contractor population and consistent with the manner in which actual projects are described and defined when contractors produce cost estimates for competitive bids. To validate the final installation labor hour estimates, the study team recommends leveraging each subcontractor s records of completed projects and/or IOU project invoice histories involving the specific measures of interest. In cases where the available project records are not sufficient or comprehensive enough to support validation for a particular measure, we recommend leveraging the appropriate firm s relationships with contractors to solicit a sample of representative project bids (separate from the in-depth interview exercise described previously) as a means of validating the installation labor hour estimates. Appliance Recycling and Direct Installation Measures Use of Program Data It should be noted here that Table 2-3 does not show a recommended data collection plan for appliance recycling measures. For these measures, the incremental measure cost is, at first blush, represented simply by the labor costs associated with removing old appliances from residences, and then decommissioning and disassembling the removed units. Conceptually, these quantities could be estimated in a fairly straightforward manner by conducting a large sample telephone survey of appliance recyclers in California. However, the IOUs appliance recycling programs are administered by turnkey contractors one for each electric utility, respectively. In this context, estimating incremental measure cost is not simply a question of estimating the labor hours associated with removing and decommissioning old units, since each turnkey contractor is also earning a profit (which is embedded in the turnkey arrangement) and potentially getting some return from the value of the recovered raw materials. From this perspective, the incremental cost of current recycling programs from a societal perspective could be estimated as simply the total value of each turnkey contract divided by the number of units removed under the program. A similar situation occurs for direct installation programs, where the IOUs engage turnkey contractors for program delivery (albeit in larger numbers than for appliance recycling). A more robust empirical assessment of these issues would require a detailed audit of each of the turnkey contracts and their performance to date. Such types of financial auditing activities are outside the scope of this MCS update. Itron, Inc Recommended Data Collection