Water utilities in the United States struggle over how to set

Transcription

1 emerging issues The typical utility bill provides a wealth of unexplored data on customer behavior and use of services. CHRISTINE E. BOYLE, SHADI ESKAF, MARY WYATT TIGER, AND JEFFREY A. HUGHES Mining water billing data to inform policy and communication strategies WITH THE DATA MINING METHODOLOGY DESCRIBED HERE, UTILITIES CAN USE DATA THEY ALREADY COLLECT TO TAILOR COMMUNICATIONS WITH CUSTOMERS AND INFORM MANAGEMENT DECISIONS. Water utilities in the United States struggle over how to set policies, design rate structures, and implement communication procedures that are effective and appropriate to their customer base. Adding to the complexity of these tasks is that water management requires local planning, and cookie-cutter strategies often fail to consider that customer characteristics and water demand vary from one town to the next, even within the same state or geographic region. For example, the same conservation initiative that succeeds at one utility may fail to reach its conservation goals or costs much more at a neighboring utility, solely because customers in the two communities have different water use patterns. Consequently, the revenue effects of the conservation initiative will be vastly different for the two utilities. Without reliable data, it is difficult for water providers to customize their demand management and communication strategies to their customer base. However, the reality is that utilities already collect much of the information they need to inform such decisions, but most, if not all, utilities do not use this valuable data set to its fullest potential. This article proposes that customer billing data offer a valuable tool one that is readily available to water utilities that can facilitate a new understanding of customer water use, inform BOYLE ET AL PEER-REVIEWED 103:11 JOURNAL AWWA NOVEMBER

2 water management decisions, and develop more effective means of communicating service-related messages to targeted groups of customers. UTILITIES ALREADY HAVE VALUABLE DATA AT HAND The most regular method of communication and data sharing between US water providers and their customers is the utility bill. The nation s utilities mail water bills to nearly 105 million households monthly, bimonthly, or quarterly, as estimated from data from the US Census Bureau (2009) and the US Environmental Protection Agency (2009). Typically, the sharing of data through the bill has been a one-way process; the utility informs customers about how much water they use and how much they need to pay. However, every billing period when they read the water meter, utilities are also collecting valuable data about each of their customers. Currently most water providers use these data to determine charges, collect revenue, monitor systemwide use, and conduct costof-service analyses, yet they can learn much about their customers simply by analyzing and tracking the water bill data at the customer level. This readily available information constitutes an effective tool for customizing strategies and procedures in order to provide more efficient service and achieve higher levels of customer satisfaction. Utility bill data can be used to inform many types of management decisions. Water utilities throughout the United States are grappling with various management decisions that are clearly influenced by customer behavior. By mining water billing data and uncovering details about their customer base, water providers can find the answers to key questions that will direct management decisions in various areas. Pricing. To maximize the effectiveness of block rate structures, where should the blocks be placed in order to target price signals to those customers who consume water in a specific pattern? How many customers never exceed the usage level within the first block? Conservation marketing. As with any marketing campaign, understanding the target audience and how Analyzing and understanding customer profiles may be difficult but are essential to utilities management decision-making framework and underpin the process required to create a continuous feedback loop between utility policies and customers behavior. it behaves is essential. Can efforts designed to reduce or change use patterns be made more efficient by incorporating customer sales analyses? How can a utility identify customer groups that would benefit from different communication messages? Peak planning. Many water providers use pricing and conservation marketing campaigns to try to reduce customers peak water use. How many customers contribute to peaking of water use? Can they be distinguished from those who use large amounts of water year-round? Innovative methodology helps provide a more refined view of the customer base. The methodology introduced here helps water providers analyze the variety of customers they serve and use that information to develop appropriate policies, strategies, and marketing programs customized to their customers water use behavior. This analytical process provides utilities with a more refined view of their residential customer base and creates analogous groups for comparison purposes. The innovation of this methodology, which represents a first-step attempt at using data-mining methods for water utilities, lies in tracking and analyzing readily available data on each customer on a disaggregated basis. This means tracking each customer s use over time and conducting an analysis, one customer at a time, in order to create profiles of different customer groups within the whole customer base. The Environmental Finance Center (EFC) at the University of North Carolina s School of Government in Chapel Hill developed the methodology and implemented it at five utilities in North Carolina between 2006 and By categorizing groups of residential customers, the research seeks to refine a customer class that is almost always lumped together in utility billing databases as residential (Mayer et al, 1999). The authors focus exclusively on residential customers in the current analysis for three reasons: residential customers represent the bulk of most utilities premises or connections, residential customers can more easily be compared from one utility to another, and the highly variable and seasonal nature of residential water demand drives system peaks and is the target of many utilities water-reduction strategies. However, this same methodology may be replicated, with some modifications, for nonresidential customers as well. The rest of this article is divided into four main sections. The first section describes the motivation behind using customer billing data to better understand a utility s customer base. Next, the authors introduce a datamining and personalization methodology that can be applied by any water provider with billing data and discuss possible developments when additional customer-level data are used. The third section presents 46 NOVEMBER 2011 JOURNAL AWWA 103:11 PEER-REVIEWED BOYLE ET AL

3 empirical examples and results of applying the data-mining methodology at five utilities in North Carolina. The final section describes limitations and concerns associated with this technique. MINING BILLING DATA HELPS HIGHLIGHT DIVERSITY OF CUSTOMER BASE Why use customer billing data? This approach to use customer billing data to inform water management decisions is driven by three factors. First, billing data are available to all utilities and are fairly accessible from the utility s finance or customer billing department. Second, using customer billing data to identify customer groups and target specific groups with customized service messages can be more cost-effective than using broadcast media and other broad public outreach approaches that blanket the entire customer base. Research has shown that customer understanding of a policy helps increase rates of compliance in dealing with water scarcity management (Chestnutt & Beecher, 1998). Thus, sending customized messages to different groups of customers on the basis of their unique water use behavior likely will increase rates of compliance. Third, each water provider is different, and understanding each specific customer base in depth leads to localized utility policies and strategies and helps achieve locally specific service results. Customer base is too diverse for a catchall category such as residential customers. When water providers conceive of their residential customers as a single large homogeneous mass (Harvey & Schaefer, 2009), they run into obstacles when designing rate structures, policies, and strategies for the average customer and applying such strategies across the board to all residential customers. Customers who differ significantly from the average may react to these strategies in ways unintended by the utility (Silva et al, 2010; Mayer et al, 1999). Studies on the relationship between utility performance and efforts to understand a utility s customer base support the notion that segmenting customers into groups related to specific strategies can aid in management decisionmaking and targeted conservation messaging (Silva et al, 2010; Chestnutt & Beecher, 1998). Diversity within and across utilities has been found to influence both use patterns and the effect of social marketing campaigns (Silva et al, 2010). For example, the households irrigating lawns and driving up peak water demand in Orange County, Fla., were identified as affluent and therefore better able to afford a higher water price, whereas high-use customers in Tempe, Ariz., were self-identified as nonenvironmentally conscious households. Because the high users in the two locales exhibited different sets of motivation behind their high water-use behavior, their respective utilities must rely on different strategies of communicating conservation policies to them (Silva et al, 2010). This large-scale study by Silva and colleagues revealed that the pockets of customers with the most significant potential to conserve water varied across US utilities. Such diversity of water-use behavior highlights the necessity of specific, targeted de mand management strategies and further underscores the importance of each utility understanding its unique customer base and the variations within it. Recognizing the need to identify the diverse user groups within the broad category of residential users is only a first step; reaching these different customer segments requires additional resources to develop and target messages appropriate to the various groups. Although recent efforts in broad-scale public outreach have succeeded in reducing water consumption and conveying water quality messages (Humm Keen et al, 2010; Meyer-Emerick, 2004), utilities continue to ponder how best to identify specific groups to relay service and other goals on the basis of location and customer-specific characteristics such as land use, climate, and demographics. Why is reaching diverse groups essential to utilities? Throughout the United States, growing utility concerns about water supply and scarcity, coupled with increasing costs of supplying water during peak seasons, have forced water providers into the difficult financial predicament of needing to encourage customers to use less of their product via conservation and efficiency rather than buy more. The range of options to foster less consumption of water resources is termed demandside management and is often accompanied by rate increases. The dual need that water providers present to their customer base use less water and pay higher rates continues to perplex customers. Rather than apply the identical strategies and rate increases to all customers, utilities could benefit from an approach that allows them to target conservation messages and initiatives to the portion of customers with high seasonal discretionary use of water. Even in the absence of mandated customer relationship management, researchers have found that when firms and utilities communicate good reasons for their actions, perceptions of price fairness tend to increase (Meyer-Emerick, 2004; Campbell, 1999). In fact, water suppliers across the United States are piloting new programs to improve the effectiveness of communication and dialogue with customers via customer outreach and public education programs (Harvey & Schaefer, 2009; Shridhar, 1999). Yet utilities continue to struggle to design cost-effective strategies that will help them make the most of a limited communications budget to reach their policy goals. In-house data-mining and analysis techniques answer this need for targeted communication strategies by offering an effective way to segment customer populations into water user groups using billing data. BOYLE ET AL PEER-REVIEWED 103:11 JOURNAL AWWA NOVEMBER

4 A DATA MINING METHODOLOGY FOR UTILITIES In 2006, the EFC engaged with the North Carolina Urban Water Consortium, an affiliation of water suppliers, through the North Carolina Water Resources Research Institute to use the utilities household-level water billing data to lead to a better understanding of customer water use behavior under a variety of rate structures and customer base characteristics. Operating under the core concept that public utilities possess a wealth of untapped information, EFC carried out a data-mining and customer profiling analysis for each participating water provider. EFC then worked with the individual utilities to determine how the data could be used to make more community-relevant decisions and sophisticated outreach efforts. This section introduces the water services customer data-mining and personalization methodology developed by EFC and explains how these profiles can be used to assess the effectiveness of rate structures on subsections of the customer base, the amount of discretionary water use that occurs in the FIGURE 1 A summer months, and the demand for irrigation meters; develop demand-side management programs customized to customer groups; identify which groups of customers responded positively (or negatively) to conservation strategies initiated by the utility improve additional aspects of planning for and managing a utility s resources, finances, and customer service programs; and identify which customers to invite to participate in focus groups The data-mining and personalization process is built on a feedback framework. In an article describing the links between marketing, customer perception, customer behavior, and financial performance for firms, Gupta and Zeithaml (2006) introduced a framework in which marketing actions drive customer behavior and firm performance (Figure 1, part A). Building on that structure, EFC introduced a water utility framework that creates a continuous loop in which customer-level data analysis underpins policy decisions (Figure 1, part B). Framework for customer metrics and their effect on firms financial performance (A) versus framework for data mining and personalization and their effect on utility performance (B) firms get customers do customers think firms do Financial performance (profits and firm values) Behavioral outcomes (observable metrics) Perceptual measures (unobservable metrics) Marketing actions utilities learn customers do customers think utilities do Part A of the figure is adapted from Gupta & Zeithaml, B Data mining and analysis from billing data Behavioral outcomes (observable metrics) Perceptual measures (unobservable metrics) Setting policies, rate structures, communications The basic framework for water services data mining and personalization begins with the data because these data describe actual customer usage patterns and are used to categorize customers into different use groups, such as high-volume users, seasonal-volume users, and nonirrigators. In this data-driven process, customer data are a key input to utility policy decision-making and inform financial and operational planning. The different customer-use categories tie directly back into water utility operations and financial performance outcomes related to capacity requirements, rate structures, conservation measures, and revenue stability. This feedback framework continues to develop and change as customers themselves change and adapt to new water service pricing and policies. The process of data-mining and personalization involves seven steps. Step 1: collecting data. The process of querying information from billing data begins with an assessment of what type of information is available and how to extract it. Data typically are requested from the billing or finance department in a specific and clear way for the finance technician to query the proper data for a specific time period. Data should be collected for every customer for all months in the study period. At the very least, the data must include a unique identifier for the customer, the billing month, and the water use for that month. Additional data can be added to supplement the basic analyses described later in this article. The key difference between this approach and the more common billing studies used by utilities to review rates and policies (such as cost-of-service analyses) is that the current approach entails tracking each customer s use over time, observing the individual patterns of use, and identifying trends within groups of similar customers. The usual billing study aggregates use across customers of each of the major customer classes (e.g., residen- 48 NOVEMBER 2011 JOURNAL AWWA 103:11 PEER-REVIEWED BOYLE ET AL

5 tial, commercial, industrial) for each month to denote changes in monthto-month usage systemwide. In contrast, the current approach disaggregates the data and determines changes in use at the customer level. To accomplish this, the water billing data must contain a unique identifier for each customer that allows the analyst to track individual customer use over time. A location or premises number is preferred over a customer account number as the unique identifier for three reasons. First, customer account numbers may change over time, such as when a utility introduces a new billing system. Second, as people move from one residence to another, they may carry their old account number to their new residence, and the data analyst will not be able to explain a significant change in water use patterns that may arise from a different house or lot size. Using the premises itself as the customer regardless of who is actually living at the location better controls for changes in house and lot size, which occur far less frequently than changes of residence, i.e., people moving to new homes. The third and most important reason for using an identifier linked to the premises rather than to a customer account number is that multiple account numbers may serve the same household. For example, at utilities that create a separate account number for irrigation meters, residents with such meters may receive two bills a month. Because the analysis detailed here calls for combining water use data from all meters for a particular household, data must be aggregated to the premises level, and therefore a premises identifier is preferred over a customer account number as the unique identifier. Step 2: preparing data. The data analyst (or in some cases, the utility billing software company representative) must next prepare the data for analysis. Before this step is un - dertaken, however, it is essential to assess the capability of the software and the staff to analyze the data and ensure that both software and staff are up to the task. The data preparation step also requires the determination of key variables for analysis, such as premises identifier, charge date or billing month, usage, service type, charged amount, and meter size. Data must then be adjusted to the requisite level for analysis. For example, to calculate the total monthly bill for individual premises, the water, irrigation, and sewer charges should be summed and late fees or other surcharges excluded in order to arrive at a total monthly service charge per customer. The goal is to produce a clean data set in which each record provides complete information on one customer s water use and total charges for one month or billing period. Step 3: segmenting and creating customer groups. Subsequent steps move into the analysis phase of data mining and personalization. Because the goals of the current approach are to fine-tune utilities understanding of their customers water use patterns and determine how customers react to certain strategy changes through their usage (and ultimately, how this affects revenues), the large, homogeneous customer base must be broken into separate groups of customers that can be analyzed independently. To accomplish this step, the analyst defines rules that split the customer base into a few mutually exclusive and exhaustive groups reflective of customer behavior (Adomavicius & Tuzhilin, 2001). Table 1 offers some examples of these rules and how they are applied. These rules can be mixed and matched to further split the cus- TABLE 1 Examples of rules that split the customer base into mutually exclusive groups Rule Basis Example Groups How the Rule May Be Applied Customer type Residential, commercial, institutional, Some utilities identify the type of customer in their billing accounts. industrial, wholesale, and other Others use meter size and maximum or minimum monthly use to categories make assumptions about whether the customer is residential or nonresidential. Residential customer s Residential low users (low demand), Calculations are based on average nonzero usage during the three or baseline demand medium users, high users four months of systemwide low usage (usually winter months). For (indoor only) example, residential low users average up to 3,000 gal per month in the winter, medium users 3,001 8,000 gal per month, and high users more than 8,000 gal per month. Seasonality of demand Seasonal users, stable users Rate is calculated for individual customers as the ratio of their average demand during their highest four or five months divided by their average nonzero demand during their lowest four or five months. For example, seasonal users may have a ratio greater than 2, indicating that they use more than twice as much water in part of the year as they do in another part of the year. Customer location Inside municipal limits versus Information for this rule is provided in the billing or customer records. outside or based on sections Some municipal utilities identify customer locations if the location of service area affects the rates being charged (e.g., inside municipal versus outside, county district). Zip codes may also be used to break up a large service area into subsections. BOYLE ET AL PEER-REVIEWED 103:11 JOURNAL AWWA NOVEMBER

6 tomer base into specialized groups. For example, the residential baseline demand rule and the seasonality of demand rule from Table 1 can be combined to create four groups of residential customers: seasonal users with low baseline demand (group 1), seasonal users with high baseline demand (group 2), stable users with low baseline demand (group 3), and stable users with high baseline demand (group 4). By breaking up the customer base into these four groups, the utility can differentiate between those residential customers whose high use is attributable to summer irrigation and those whose high use is attributable to many family members and protect the latter group from increasing block rate structures that penalize high water use year-round. (This example is explained in detail later in the article.) The use of defined segments moves the analysis toward the personalization aim of the data mining, i.e., to profile customers based on their use patterns. Identifying different customer groups helps utility decisionmakers understand the effect of various strategy adjustments on a wider range of their customers and not just the average customer. Furthermore, segmenting the customer base and determining how many customers fall within each group provide valuable information to utility managers as they assess different strategies for accomplishing their goals. For example, water providers will be better able to predict how successful their mandatory watering restrictions will be at achieving water conservation if they know how many customers demonstrate significant seasonality in their use patterns and can calculate the difference between those customers highseason use and low-season use. Without segmentation, the utility would have to estimate the potential water savings on the basis of its entire customer-base usage, without knowing how many customers use large amounts of discretionary water. Step 4: creating customer profiles. After the customer base has been segmented according to various rules, billing records can be analyzed to calculate distributions of usage patterns and charges for the different customer groups. These analyses should reflect the type of information needed to make appropriate Diversity within and across utilities has been found to influence both use patterns and the effect of social marketing campaigns. policy decisions and develop rate structure designs that are relevant to the utility s customer base. In order to conduct the analysis, the water provider will need a statistical software program, which could range from a basic spreadsheet program 1 for small utilities to more sophisticated software for larger data sets. 2,3,4 Table 2 provides examples of the types of analyses conducted by EFC for the five North Carolina utilities; this list does not include all the types of customized analyses that could be performed with monthly data on usage and bill charges for each customer over a long period. Step 5: adding context to the analyses. The data analyses can provide decision-makers with answers to many questions that could not be answered previously. Nonetheless, as with any data analysis, results must be placed in context of what was occurring at the time. For example, in the EFC analysis of North Carolina utilities, results indicated that a water provider experienced a significant decrease in residential water use in one year, even though the utility had not actively promoted or aggressively encouraged water use conservation because it had excess supply. In discussions with utility staff, the authors learned that a larger neighboring utility serving a community within the same TV and newspaper media-shed was experiencing a water shortage at the time and had strongly promoted water conservation. The customers of the smaller utility received the same message through the traditional media outlets and reduced their own use significantly, even though their own water provider was not actively promoting conservation. Step 6: testing the effect of program initiatives. Once customer profiles and patterns have been developed, utilities can use the outcomes to identify trends over time (in the context of operations during the time period) and develop hypotheses to explain these trends. For example, using customer profile analysis results, a water supplier might hy - pothesize that for its customers conservation ordinances did not influence residential water use behavior significantly, nonresidential customers were more likely to respond to rate hikes than residential customers, and rate increases for irrigation water use significantly curbed irrigation-metered water use among high users. With two or more years of customer data spanning a time period before and after a strategy or rate change has taken place, the utility can use simple statistics to test these hypotheses and quantify the effects of program initiatives on water use and revenues. This type of month-tomonth panel data analysis can be used to help conservation coordinators evaluate the relative success of various conservation programs among different groups of customers and determine how best to market the different initiatives to the specific 50 NOVEMBER 2011 JOURNAL AWWA 103:11 PEER-REVIEWED BOYLE ET AL

7 TABLE 2 Examples of customer profile analyses Analysis How to Perform the Analysis Application and Potential Use of the Analysis Variation in residential Determine each residential customer s Determining proportion of low users to high users in customers average average monthly use. Compute the customer base. Affects basis of rate structure design use cumulative distribution of all customers averages across a range from 0 gal per month Calculating how rate changes will affect the largest group of to > 20,000 gal per month. customers, based on use levels of that group rather than systemwide average usage level Determining how many customers average water use falls within the consumption allowance included with monthly base charge or within the lifeline block, if applicable Variation in residential Determine each residential customer s Determining nondiscretionary water demand in the customers baseline average monthly use during the three customer base for which utility must ensure adequate (indoor-only) use or four months of lowest systemwide water supply nearly 100% of the time. Determines the use (usually winter). Compute the base revenue levels the utility can rely on cumulative distribution of all customers baseline usage across a range from 0 gal per month to > 10,000 gal per month. Determining proportion of customers with high nondiscretionary demand, probably because of large family size or other unobservable factors. Customers may need protection from increasing block rate structures that penalize high use year-round or assistance in retrofitting with low-flow fixtures Identifying the usage point that covers almost all residential customers indoor base demand; use over this point is predominantly discretionary. Analysis can be used to select an appropriate block level over which water use is charged a much higher rate in an increasing block rate structure Variation in residential Determine each residential customer s Determining the appropriate block sizes in a block rate customers maximum maximum monthly use in a 12-month structure. Identifies how many customers will always, monthly use in a period. Compute the cumulative never, or only sometimes reach a specific block of use 12-month period distribution of all customers maximum usage across a range from 0 gal per month to > 30,000 gal per month. Assessing fairness of system development charges (often called impact fees or capacity fees) by revealing range of water system capacity required by groups of customers Provides information useful to capacity planning Customer usage Divide each customer s average highest Determining the proportion of customers with high peaking patterns four or five months of use by his average discretionary water use, as identified by their high-peaking nonzero usage during his lowest four ratios. Provides information on the amount of discretionary or five months. Calculate the number water use that can be curtailed through conservation of customers with peaking ratios in programs and the effect on revenue various ranges (e.g., up to 1.5 times, 1.5 to two times, two to three times, more Determining the proportion of customers with low discrethan triple). tionary use. Informs whether mass communication on water conservation and watering restrictions may fall on deaf ears Provides information to assess the importance and relevance of using an increasing block rate structure or other strategies aimed at limiting discretionary water use Identifying Identify customers with irrigation meters Determining amount of irrigation water used each month. residential irrigators and calculate use through the meters. Assists utility in designing seasonal programs, such as imple- Identify other customers (or all customers) menting seasonal rates. Provides information on potentially whose total use pattern varies seasonally discretionary use that can be curtailed through conservation proto a similar extent as that of irrigation- grams or irrigation meter cutoffs during water shortage periods metered customers. Calculate difference between summer and winter use to Identifying customers with irrigation use patterns despite the determine amount of water likely used absence of irrigation meters. Utilities can interact with these for irrigation. customers as with irrigation-metered customers Determining ratio of irrigation water usage to indoor (standard) water use for customers who irrigate. Provides information on system capacity needs (to build distribution systems sufficient to accommodate irrigation use). Provides information to help assess fairness of system development (impact) charges Changes in customers Determine each customer s average monthly Evaluating the effect of program changes on customers use, espeuse following water use before and after a specific month cially on specific groups of customers (e.g., did watering restricprogrammatic when a programmatic change (rate increase, tions result in seasonal users or high users reducing their use?) changes conservation program) was implemented. Calculate each customer s change in average Determining the number of customers and which groups of cususe. Compute the cumulative distribution tomers significantly altered their usage patterns, whether by design of the changes (both positive and negative). or unintentionally; determining revenue effects of their change Determine distribution for different groups of customers, e.g., low versus high users, Providing information to help utilities predict expected usage stable versus seasonal users. response to implementation of similar programs in the future Monitoring changes in demand within the customer base, particularly those affecting the peak demand capacity needs of the system. Assists with capacity planning and water system design BOYLE ET AL PEER-REVIEWED 103:11 JOURNAL AWWA NOVEMBER

8 groups that are most likely to respond positively the next time the utility implements such initiatives. Step 7: personalizing and creating customer lists. The phrase data mining implies that riches lie within the information that utilities have and that utilities only need to locate these treasures and extract them. This is generally true. However, the real value of data mining is demonstrated by how a utility uses information gleaned from these data to inform its management decisions. The information obtained from data mining using the current methodology can guide utility managers and customer service professionals in designing more appropriate rate structures, customizing programs and policies, and continuously monitoring customer-use patterns in response to changes in strategy. In business the use of data-mining techniques to inform and improve decision-making is a business intelligence approach or knowledge management ; for water utilities, it can best be described as personalization. Data can be personalized in various ways. Households % One use of the customer-use profile data is matchmaking (Adomavicius & Tuzhilin, 2001), in which utility planners match and/or develop appropriate services targeted to individual customers based on their use profiles. Instead of conducting specific operations and communicating with all customers, utilities use the segmentation approach to identify individual customers who should be targeted with the relevant operational information. For example, after creating customer profiles and segmenting the customer base according to the methods described here, EFC provided the five utilities with lists of residential customers whose average monthly use was either zero gallons or exceeded 20,000 gal per month for 12 consecutive months. The utilities then checked the accuracy of those water meters and whether there were residents in those premises. One water supplier determined that a few customers had been misclassified in its billing system as residential when they were, in fact, commercial customers. Without the list of high users to hone in on, the utility would have FIGURE 2 Average household monthly water use between July 2007 and June 2008 Households averaging > 10,000 gal per month Households averaging 6,001 10,000 gal per month Households averaging 3,001 6,000 gal per month Households averaging 1 3,000 gal per month Households averaging 0 gal per month A B C D E Utility faced a more difficult challenge in checking and verifying the classification of all of its customers. The EFC also created lists of residential customers who were not connected to irrigation meters but demonstrated seasonal water use patterns that suggested significant irrigation during the summer. The individual utilities can contact these customers and suggest the installation of irrigation meters. Furthermore, during the next water shortage period, the water providers can contact these customers first whether or not they are connected to irrigation meters and educate them about utility water conservation programs. In contrast to a mass communication campaign aimed at all customers, this type of communication targeted at a subset of customers who are more likely to respond positively to the message is more cost-effective for the utility a prime example of the value of personalization achieved through data mining. Personalization can also influence policy decisions on rates and rate structures, drought surcharges, conservation programs, peak shaving, and recurrent cutoffs and irrigation. In addition, the utility can use cutoff data profiles to target customer hardship programs that improve water provision while providing affordability or payment plans. Another personalization tool is personalized presentation characterized in terms of three different directions: pull, push, and passive (adapted from Schafer et al, 2001). A utility can pull information from customers, from data, and from surveys and focus groups. For information on specific types of customers, billing records can be used to segment the customer base into groups in order to isolate the types of customers to survey or invite to join focus groups. For example, if a water supplier is interested in developing an affordability program, it makes sense to survey or interview the customers who are likely to use this program, such as customers who have been delinquent in payments or 52 NOVEMBER 2011 JOURNAL AWWA 103:11 PEER-REVIEWED BOYLE ET AL

9 experienced recurrent cutoffs because of nonpayment of bills. This type of pull communication is an example of feedback a utility can use to ultimately improve both its public image and its communication strategies. Alternatively, a utility can use push personalization tools to identify groups and develop communication strategies tailored to the group. This type of personalization is performed directly by the water provider without interacting with customers or seeking specialized communication. The previous example of water utilities using data mining and segmentation to identify high-use irrigators during the summer and communicating with them directly during water shortage periods is an example of a push personalization technique. Finally, a utility can make use of passive personalization tools in which personalized information is shared with the customers through two-way interactions. Web-based services such as Amazon and Netflix use this approach, using a customer s browsing and purchasing records to create personalized lists of recommended products that might interest the customer, based on patterns identified from similar customers. Other examples include the interactive Positive Energy Together website (OGE Energy, 2011) that tailors energy conservation tips and programming based on assumptions about the customer s energy use. If the user s profile is not accurate, the user can modify the information and receive a more relevant list of recommendations. Water providers could design similar websites, prepopulate them with customers water use history or customer profiles, and provide an interactive interface for customers to edit their profiles and receive customized tips on water conservation, thus creating a two-way communication between the utility and its customer base. Because it relies solely on basic billing data, the methodology described here can be applied to any utility that charges its customers based on water use. Supplemented by additional data tied to the billing data, customer-level analysis can provide even more detailed understanding of customers over time by revealing trends in use and bill-paying practices such as late payments and cutoffs resulting from nonpayments. Billing data that are already linked to parcel data with such in - formation as house and lot size and house value can yield rich analyses for water providers concerned about the effect of their rate structures and policies on low-income households. These analyses can also be conducted if the utility currently maintains data on customer household characteristics such as household size and income. Hourly or daily water use data obtainable through the use of smart meters or time-of-use meters can provide even more refined analyses than monthly or bimonthly data. Producing profiles based on customer-level use history can constitute a powerful tool for evaluating utilitywide policy decisions that affect customers and allows utilities to better target specific marketing campaigns. FIGURE 3 Households % EXAMPLES FROM THE FIELD DEMONSTRATE METHODOLOGY IMPLEMENTATION A treasure trove of information is available from the more than 10 million billing records obtained from five of North Carolina s largest utilities: Charlotte-Mecklenburg Utilities, Fayetteville Public Works Commission, City of High Point Public Services, Orange Water and Sewer Authority, and Greenville Utilities Commission. The EFC analyzed the billing records for every residential customer (totaling more than 378,000 households) at these five utilities for a 30-month study period between July 2006 and December The water billing data-mining analysis and personalization was carried out with the goal of applying results to aid decision-making around specific policy questions. The following cases were developed through an investigation of the data according to the instructions provided in Table 2 and enhanced by the perspective of utility staff members. Although real examples from existing water providers are used, when specific results are dis- Cumulative distribution of households' maximum use in all months between July 2007 and June 2008 Utility A Utility B Utility C Utility D Utility E Maximum Monthly Use 1,000 gal BOYLE ET AL PEER-REVIEWED 103:11 JOURNAL AWWA NOVEMBER

10 cussed, the utility identity has been protected because the authors are not authorized spokespersons for these utilities. Residential customers can be categorized according to variations in average use. Residential customers, or house holds, can be grouped in various ways. One approach is to categorize the customers by their average water use over a 12-month period (Figure 2). The ability to determine these groups or create a comprehensive cumulative distribution of all residential customers average use offers utility managers a vital information resource to assess the effect of their rate structures on the full range of residential customers. Utilities often calculate and present customer charges at a single, specific consumption amount near their systemwide average use to demonstrate Utility A Utility B Utility C Utility D Utility E 33% the effect that their rate structures will have on the average residential customer. In North Carolina, for example, most utilities calculate the monthly residential bill for a consumption point between 4,000 and 6,000 gal per month, depending on the utility. As shown in Figure 2, however, only a small percentage of households actually have average use in that range, making the utility s rate structure assessment irrelevant to the majority of customers. In fact, as Figure 2 shows, average use for approximately one third of customers at four of the five utilities was < 3,000 gal per month. At the other end of the spectrum, 4 to 14% of households averaged > 10,000 gal per month. With the complete distribution of customers 12-month average use, the utility manager now has the information in hand to determine FIGURE 4 Profiles of household water use patterns between July 2007 and June 2008 High Peaking: High Monthly Use More Than Twice Low Monthly Use Low Peaking: High Monthly Use Less Than Twice Low Monthly Use Group 1 Low User: Average Use < 5,000 gal per month 24% 28% 36% 31% 34% 23% 23% 27% 13% 15% 40% 36% High User: Average Use > 5,000 gal per month 15% 17% 22% 23% 31% 19% 10% Group 2 Group 3 Group 4 how the rate structure will affect different groups of residential customers on the basis of their average consumption and can even calculate a systemwide weighted-average residential bill. Variation in residential customers maximum monthly use over the year is helpful information. In addition to determining each household s average water use, data mining and personalization allow the utility to track each household s maximum monthly use (including irrigation water use) over a 12-month period. Charting the cumulative distribution of households maximum monthly use (Figure 3) provides important information for designing block rate structures; such information cannot be obtained through contemporary cost-of-service billing analyses, which can determine the cumulative distribution of bills at different consumption amounts but not the households that exceed certain amounts. Consider the example of a manager at utility B who is interested in implementing an increasing block rate structure to encourage water use efficiency among residential customers. Using the data from Figure 3, the manager can estimate the percentage of households (not simply the number of bills) that would have received at least one bill at a higher block s rate during the year, depending on the size of the blocks. For instance, if the first block ended at 10,000 gal per month, then only 26% of households would have received a bill at the second block s rate, and 74% would have always stayed within the first block and never receive the increased price signal. This information is also helpful to utilities when considering system development charges or impact fees. As shown in Figure 3, households have a wide range of water system capacity needs. At utility C, only 5% of households exceeded 20,000 gal per month, whereas 23% never exceeded 3,000 gal per month (including irrigation use). By track- 54 NOVEMBER 2011 JOURNAL AWWA 103:11 PEER-REVIEWED BOYLE ET AL

11 ing customers use and combining indoor and outdoor water use, a water provider can assess system development charges on a sliding scale based on actual capacity needs. Customer usage peaking patterns help delineate discretionary use. Average and maximum residential water uses alone do not reflect changes in household use behavior over time. To provide an accurate assessment of water use fluctuations in a given year, residential customers are profiled on the basis of their peaking behavior, i.e., the ratio of their highest months use to their lowest months use. A high-peaking household is defined in this article as one whose average of three months of highest use divided by its average of three months of lowest nonzero use was greater than two. In other words, for one quarter of the year, high-peaking households used more than double what they used during another quarter of the year. This behavior suggests that these households have some discretionary use, i.e., a significant portion of their demand that they can live without for at least three months out of the year. This usage pattern is important to discern because utilities often use pricing and conservation marketing to reduce use and discourage heavy water use. In order to maintain revenues while encouraging conservation, waters suppliers may focus their highest price increases on discretionary users, on the assumption that these users can cut back on watering lawns, filling pools, and washing cars. For example, most increasing block rate schemes are structured to charge lower levels of use (sometimes referred to as lifeline consumption) at inexpensive rates and to charge higher levels of use at much higher rates. Furthermore, utilities design many of their facilities to meet customers maximum use during the year. Households that use 20,000 gal per month for three months and 5,000 gal per month for the rest of the year have a different effect on a utility s facilities than households that consistently use 9,000 gal per month all year long, even though both households use nearly the same amount in 12 months. In Figure 4, residential customers are categorized into four groups based on their average use and peaking pattern (similar to the groups listed previously in step 3 of the methodology section but using average use rather than baseline use). Group 1 consists of households that have a low average water use but also use significantly more water for a few months out of the year. An example might be a small family that uses a moderate amount of water in the winter but waters the lawn in the summer. Group 1 households are an ideal target for increasing block rate structures and other strategies de - signed to encourage peak shaving and conservation. During periods of water shortage, utilities can reach these households with pamphlets on water conservation tips and information on watering restrictions. Group FIGURE 5 Households % FY fiscal year Increased average use > 50% in FY 2008 Increased average use 5 50% in FY 2008 Maintained average use within 5% in FY 2008 Decreased average use 5 50% in FY 2008 Decreased average use > 50% in FY includes households that use more water on average and are also highpeaking. Like group 1, these households are targets for conservation strategies, but they may additionally be receptive to programs to promote water use efficiency over the long term to reduce their baseline de - mand. Utilities would benefit most by targeting the more expensive efficiency programs such as toilet rebates, showerhead exchanges, and rain sensors to these customers first. Group 3 households are low users and low-peaking. Utilities can use data mining to avoid the unnecessary expense of communicating conservation messages to these customers, who currently demonstrate efficient water use behavior and are unlikely to substantially decrease their already low use. Some utility managers are concerned about the unintended consequences of increasing block rate structures on group 4, i.e., households that have average high use year-round yet little to none of this Changes in average water use by all households in one utility from FY 2007 to FY < 5,000 gal 5,001 10,000 gal 10,001 20,000 gal > 20,000 gal Monthly Household Average in FY 2007 BOYLE ET AL PEER-REVIEWED 103:11 JOURNAL AWWA NOVEMBER

12 TABLE 3 Comparison of known and estimated irrigators use is discretionary. For example, a household with a large family may use high amounts of water all year and have little discretionary use but would still be charged the higher unit price every month. When a utility with many group 4 customers implements an increasing block rate structure, it likely will see less reduction in water use than a utility with greater numbers of high-use, highpeaking customers. The variation among the five water suppliers shown in Figure 4 illustrates how much utility customer bases can differ within a state, let alone state to state and across the country. This is why rates, policies, and strategies can and should be customized to address individual utilities situations, rather than broadly promoted and applied. Data mining and personalization are required to describe the customer base in sufficient depth to make appropriate decisions at the utility level. Data mining helps identify residential irrigators. The power of data mining and personalization to identify specific customers is even more evident in this example. Even though many water providers offer residential customers the option of installing a separate irrigation meter to measure their outdoor and irrigation water use and avoid being charged wastewater rates for this water, many households that irrigate significantly do not install such meters. Data mining and personalization provide the utility with a method of identifying households that may irrigate at high levels during the summer but do not currently have an irrigation meter. The water provider can then target this group with specific messages encouraging efficient irrigation habits or promoting installation of irrigation meters, which may be monitored and even cut off during water shortage periods. For the five utilities in this study, the authors examined the summertime-to-wintertime peaking ratios and total irrigation-metered water use of households with irrigation meters and then identified households that had similar water use patterns but lacked irrigation meters. A list of these customers was presented to each utility. Table 3 shows the estimated percentages of residential irrigators, with and without irrigation meters. Tracking customer use after programmatic changes helps pinpoint programs effects. One of North Carolina s most severe droughts occurred between 2007 and During this period, many water providers adopted outdoor watering restrictions and other campaigns to aggressively promote water conservation. By tracking each household s Households With In-ground Irrigation Households With Systems, With and Without Irrigation Utility Irrigation Meters % Meters % (estimated) A 8 12 B 3 4 C 3 4 D 2 12 E Without reliable data, it is difficult for water providers to customize their demand management and communication strategies to their customer base. water use across multiple years, however, a utility is able to go beyond assessing the net effects of its strategies and to determine their effects on various groups of residential customers. For example, one of the five utilities studied implemented mandatory outdoor watering restrictions, increased its rates for high water use, and actively promoted water conservation across its entire residential customer base starting at the end of the summer of Despite these strategies, however, average household water use increased by 9% from fiscal year (FY) 2007 (July 2006 through June 2007) to FY 2008 (July 2007 through June 2008). This statistic alone, obtainable from a simple billing analysis, would suggest that the conservation strategies used by the utility were ineffective. However, data mining and personalization add context to this assessment by drawing a more detailed picture of exactly which households reduced or in creased water use. As shown in Figure 5, the majority of households that were high users in FY 2007 (before implementation of the conservation strategies) significantly reduced their water use in FY 2008, which was the intended result of the conservation strategies. In fact, 27% of users who averaged more than 20,000 gal per month in FY 2007 more than halved their water use in FY Meanwhile, 21% of FY 2007 s low users (those who averaged under 5,000 gal per month) increased their water use by more than 50%. For most of these customers, their increased use was not significant enough to suggest 56 NOVEMBER 2011 JOURNAL AWWA 103:11 PEER-REVIEWED BOYLE ET AL

13 they were irrigating but that they increased their low usage slightly (as usually occurs during warmer, drier years). However, because the low users who increased their use (even slightly) outnumbered the high users who reduced use, the net effect was an increase of average household use systemwide, even though the outdoor watering restrictions and increased rates for high water use had been effective at curtailing discretionary use in FY As a result of the changes in water use and rate structure in FY 2008, the utility increased the total billed amount for all households by more than $2.3 million in FY By combining the water-use data with billing charges, data mining and personalization allowed the utility to determine how different groups of customers contributed to the change of revenue and assess its revenue vulnerability to reductions in water use. Figure 6 shows how the total billed amount changed from FY 2007 to FY 2008, based on customers change in water use. For example, those residential customers who decreased their average water use by more than 50% paid a net total of $1 million less than they did the previous year. DATA MINING HAS SOME LIMITATIONS AND CONCERNS Successful data mining may present software challenges and require time and expertise. A key element of the methodology described here is its ability to track each customer s use over time. However, this capability also produces some of the most difficult challenges to this approach. Customer billing software and databases are designed primarily for accounting purposes and not as a source of marketing information. Each transaction (including payments and billing adjustments) is recorded separately and for each meter individually. As a result, collecting and converting raw billing data into data that can be analyzed across time and comparatively for different utilities, FIGURE 6 Increased average use > 50% in FY 2008 Increased average use 25 50% in FY 2008 Increased average use 5 25% in FY 2008 Maintained average use 5% in FY 2008 Decreased average use 5 25% in FY 2008 Decreased average use 25 50% in FY 2008 Decreased average use 50% in FY 2008 as outlined in steps 1 and 2 of the methodology, require significant time and expertise in management and analysis of large data sets. The level of time commitment and expertise may spur utilities to outsource the process. Doing so, however, creates other challenges. In many states, utility billing data are not considered public record, even for publically owned utilities. Therefore, utilities and outside data analysts must take care to ensure data security when handling billing records. The utility and third-party data analysts must come to a mutual agreement on how to collect, store, transfer, analyze, and, on completion, destroy the data without compromising the security of any sensitive information. This process is similar to the one researchers use on institutional review board-approved projects that handle sensitive information. Security is an essential concern for both utilities and their customers. The data analysis undertaken by water providers should avoid the use of customer names, social security numbers, tax records, dates of birth, and addresses. The authors found that all of these fields could (and should) be excluded from the utility s initial data query of its billing database when creating the raw data set; the only requisite is that there is a unique location number that identifies the meter or premises, rather than the individuals living at that property. SUMMARY Customer bases even across those with similar systemwide usage patterns vary from utility to utility. For this reason, strategies that might work for one water provider may prove ineffective at another utility where the customers behave differently. Furthermore, the average customer does not exist; the individual customers within a utility s customer base are all different. Therefore, rather than view and treat the customer base as a single, large homogeneous mass, utilities would be better served by implementing data mining and personalization techniques to distinguish among groups of customers with distinct usage patterns. Data that are readily available from utility finance departments can be used to create customer profiles and provide answers to many questions needed to tailor utility policies, rate structures, and programs to best fit the various customers served. Analyzing and under standing customer profiles (1,500) (500) 500 1,500 2,500 Changes From FY 2007 to FY 2008 in Total Billed Amounts Charged $1,000 FY fiscal year Financial effect of changes in one utility s rate structure and households average water use from FY 2007 to FY 2008 BOYLE ET AL PEER-REVIEWED 103:11 JOURNAL AWWA NOVEMBER

14 may be difficult but are essential to utilities management decision-making framework and underpin the process required to create a continuous feedback loop between utility policies and customers behavior. The authors do not mean to imply that the use of such data can solve all of the challenges currently facing water utility managers. The lesson offered is that relatively accessible data can be a conduit to improving customer communication and more effectively reaching performance objectives. Through personalization of marketing and policy messages, water providers can implement communication campaigns that are better targeted around important rate, conservation, and public health updates. However, as suggested by other studies (Katz, 2002), marketing should not be confused with public participation as a means to integrate stakeholder views into important policy decisions. Community engagement remains critical to fostering cooperation around conservation and water quality. ACKNOWLEDGMENT The authors thank the North Carolina Urban Water Consortium and the Water Resources Research Institute in Raleigh, N.C., for funding these studies. Special appreciation goes to the staff members of the five participating utilities who supplied the authors with billing data and provided feedback throughout the process. In addition, the authors thank Jeannine O Brian and Laura Adams for editing this article. ABOUT THE AUTHORS Christine E. Boyle (to whom correspondence should be addressed) is a finance analyst at the Environmental Finance Center (EFC), University of North Carolina (UNC) at Chapel Hill, Campus Box 3330, Knapp-Sanders Bldg., Chapel Hill, NC ; cboyle@sog.unc.edu. She leads the NC Urban Water Consortium Customer Water Sales Data-mining and Analysis Project, as well as leading the EFC series on Market Assessments of Energy Efficiency Retrofit Financing Programs. She holds a bachelor s degree from Columbia University in New York and master s and doctoral degrees from the UNC at Chapel Hill. Her expertise focuses on rural irrigation governance and the fiscal policy of water distribution in both urban and rural sectors, as well as strategies to mitigate the effects of municipal and industrial development on local water quality. Shadi Eskaf is a senior project director at EFC and a doctoral student in the UNC Department of Environmental Sciences and Engineering. Mary Wyatt Tiger is a senior project director at EFC. Jeffrey A. Hughes is the director of the EFC at UNC, Chapel Hill, and a faculty member at the UNC School of Government. Date of submission: 10/11/10 Date of acceptance: 08/10/11 FOOTNOTES 1 Microsoft Excel, Redmond, Wash. 2 Stata, StataCorp, College Station, Texas 3 SAS and JMP, SAS Institute, Cary, N.C. 4 R, RFoundation for Statistical Computing, Vienna, Austria JOURNAL AWWA welcomes comments and feedback at journal@awwa.org. REFERENCES Adomavicius, G. & Tuzhilin, A., Using Data Mining Methods to Build Customer Profiles. Computer, 34:2:74. Campbell, M.C., Why Did You Do That? The Important Role of Inferred Motive in Perceptions of Price Fairness. Jour. Product & Brand Mgmt., 8:2:145. Chestnutt, T.W. & Beecher, J.A., Conservation Rates in the Real World. Jour. AWWA, 90:2:60. Gupta, S. & Zeithaml, V., Customer Metrics and Their Impact on Financial Performance. Marketing Sci., 25:6:718. Harvey, B. & Schaefer, A., Managing Relationships With Environmental Stakeholders: A Study of U.K. Water Electricity Utilities. Jour. Business Ethics, 30:3:243. Humm Keen, A.; Keen, D.; Francis, E.G.; & Wolff, A., High-Contact, Hands- On Outreach Program Changes Customers Water Use Behavior. Jour. AWWA, 102:2:38. Katz, S.M., As We See It: Don t Confuse Marketing With Public Participation. Jour. AWWA, 94:7:38. Mayer, P.W.; DeOreo, W.B.; Opitz, E.M.; Kiefer, J.C.; Davis, W.Y.; Dziegielewski, B.; & Nelson, J.O., Residential End Uses of Water. AwwaRF, Denver. Meyer-Emerick, N., Are We Asking the Right Questions? Improving CCR Communication. Jour. AWWA, 96:8:104. OGE Energy, Positive Energy Together. energyefficiency/pages/home.aspx (accessed July 18, 2011). Schafer, J.B.; Konstan, J.A.; & Riedl, J., E-Commerce Recommendation Applications. Data Mining & Knowledge Discovery, 5:1:115. Shridhar, P., The Natural Lawn Program: A New Approach to Outdoor Water Conservation. Proc. Conserv 99, Monterey, Calif. Silva, T.; Pape, D.; & Szoc, R., Water Conservation: Customer Behavior and Effective Communication. Water Res. Fdn., Denver. USCB (US Census Bureau), American Community Survey Washington. USEPA (US Environmental Protection Agency), Safe Drinking Water Infrastructure System Federal Database, Washington. 58 NOVEMBER 2011 JOURNAL AWWA 103:11 PEER-REVIEWED BOYLE ET AL