EMPOWERING customers to save energy by informative billing

Transcription

1 EMPOWERING customers to save energy by informative billing Deliverable 6.3 Publishable report on the evaluation of EMPOWERING outcomes Project No. IEE/12/697/S Project co-funded by the European Commission within the IEE Programme Dissemination Level PU Public x CO Consortium and EACI

2 Authors Name Organisation Michele Pastorelli, Mariapia Martino Giovanni Pede POLITO SINERGIE Revision history Revision Date Author Organisation Description /12/2015 Giovanni Pede SINERGIE /01/2016 Giovanni Pede SINERGIE Summary of customer satisfaction surveys Initial template and customer satisfaction chapters /01/2016 Marina Pirazzi, Vittorio Gimigliano REGGIO EMILIA Insertion of results from the two meetings with households held in Reggio Emilia /01/2016 Giovanni Pede SINERGIE Insertion of customer satisfaction infographics /02/2016 Mariapia Martino POLITO Introduction to the documents, comments to the graphs and conclusions /03/2016 Mariapia Martino POLITO Executive summary inclusion 2

3 Table of contents Executive Summary Introduction Procedure for evaluation of energy saving in EMPOWERING Evaluation of energy savings in Empowering Evaluation of the impacts on customer satisfaction with EMPOWERING Evaluation of results in each pilot site Austria France Italy Spain Conclusions Bibliography

4 Executive summary Energy saving and effective consumer empowerment constitutes two of the dimensions of the European Commission s Energy Union Package, designed to improve sustainability, energy security and competitiveness. In fact, the incorporation of new information and communication technologies in the electricity network will turn the consumers - traditionally passive end-users - into active players. A common ICT framework for monitoring of energy savings, with harmonized data could improve the amount of energy saved and could allow a very interesting comparison between similar consumers. In the report, the outcomes from the evaluation of EMPOWERING outcomes are reported, with detailed analysis of each of the pilot sites involved in the projects. The quantification of the service impact over the energy consumption and the changes in user behaviour and satisfaction with the utilities is provided. The obtained results are evaluated adopting the Evaluation framework methodology for EMPOWERING (see D6.1). In the table is reported the evaluation of electric energy consumption, gas consumption and district heating consumption for each site involved in the program: electric energy gas district heating Linz (Austria) X Grenoble (France) X X Torino (Italy) X X Reggio Emilia (Italy) X Soller (Spain) X The consumption data are collected by the utilities and sent to CIMNE for the elaboration and processing. The preliminary data cleaning procedure is based on: - removal of lower and upper values that are ties around the core values - removal of consumers that show high differences in energy consumption for the same month in different years The experimental design, adopted in each site, for the selection of the experimental and control groups is reported. The ADC (average daily consumption), computed on monthly time base, is evaluated and energy savings are computed from the start of EMPOWERING program up to December The most important results about information effects on energy savings are: - National laws concerning personal data protection heavily affect the process of selection of the users involved in the program. Opt-in option or registration to a free service allows the selection of users more interested in the in the service that are prepared to change their habits to a more responsible behaviour concerning energy, but limits the number of users. - Billing reports and data available on a web site are more effective than billing reports sent by e- mail. The final user has to be active part in looking for information on the web site, while with e- mail is a passive receiver. - The reduction of time delay between the use of energy and the measurement of energy can help to obtain direct and immediate control on the behavioural actions, giving in this way immediate acknowledgment. Higher values of energy savings can be achieved with higher values of ADC. In 4

5 particular higher savings are computed with users that employ electric energy for sanitary hot water and space heating. - The combination of the on line tool and the billing tool (availability of real time information on the web portal and paper invoice combined with information on energy consumption) constitutes a good solution for an active consumer engagement. This because combines two different engagement modalities: one is based on the active willingness of the user to check its personal energy consumption and related cost, the second is based on the systematic recall (based on a specific time frame) from the utilities to have a look on the bill and on the energy consumption. Even if some program-external factors can influence the impact of the services (e.g. building-type, family composition, billing system), it was demonstrated that saving could be achieved across all pilot site without correlation with those factors. For that reason the end-user information services are effective for various kinds of users. The evaluation of the customer satisfaction in the EMPOWERING project was limited to investigating the satisfaction with the billing information provided by the utilities, which was directly influenced by the services developed in the project. The evaluation didn t have as objective to assess the global satisfaction of the customers with their utility company, as it is affected by a number of external factors such as energy prices, governmental policies, taxes, etc., which are out of the project scope. The evaluation, similarly to the SERVQUAL methodology, surveyed the perceptions and expectations of users regarding the services, establishing as an indicator of satisfaction the gap score between them (Gap score=perception-expectation). The methodology allowed to quantitatively evaluate the quality of the EMPOWERING services and the changes of satisfaction by analysing the user perceptions and the gap score before and after the receiving the services. The results can be summarised in: - The EMPOWERING program obtained good acceptance among the users (rank over 100). - In all the sites the utilities achieved improvements in the information services and 67% of the whole amount of users involved in the program are satisfied with the services. 5

6 1. Introduction This report is compiled in the frame of EMPOWERING project, work package WP6, task T6.3 Evaluation and performance monitoring. The overall objective of the evaluation work is to develop approaches that can be used for an impact analysis of the ICT tools and solution adopted in the five pilot sites: Grenoble-France, Linz-Austria, Soller- Spain, Torino and Reggio Emilia-Italy. WP6 is based on the previous work packages where requirements have been defined and services have been developed. Fig. 1 Simplified flow chart of major activities in EMPOWERING project. The D6.3 represents the publishable report on the evaluation of EMPOWERING outcomes performed at each pilot site. It will provide quantification of their impact over energy consumption and the changes in user behaviour and satisfaction with the utilities. The activities of widespread service application and continuous evaluation of the service perception, preference and satisfaction of customers, are performed taking into account the methodology defined in previous deliverables. The evaluation procedure is carried out in compliance to the requirements defined in the annex I to the Grant Agreement. In the deliverable are analysed the results up to the month of December 2015, with the aim to have an evaluation of the project results, based on the comparative behaviour of experimental and control groups. After December 2015 four pilots (Linz, Soller, Torino and Reggio Emilia) have continued the services, with the aim to extend EMPOWERING services to all the customers, this taking into account the specific regulatory frameworks in each site. The comparative analysis of different energy savings in the pilot site is carried out and general recommendation and comments are reported in the conclusion. 6

7 2. Procedure for evaluation of energy saving in EMPOWERING Energy consumption in residential environment depends on both social and weather condition that are not easy to model and predict. The general approach to solve this problem is the definition of a control and experimental group, located in the same site, where the treatment is provided to the experimental group only. The different behavior of the two groups in the same environment is then compared, this in order to isolate the effect of the treatment (EMPOWERING services) among the influence of several disturbances that are observable with difficulties (weather conditions, social constraints). 2.1 Evaluation of energy savings in Empowering As reported in D6.1, the method for the evaluation of energy saving is based on the difference in difference algorithm. This approach is based on comparing changes in energy consumptions, at given time samples, for the experimental and control group of users involved in EMPOWERING project. The experimental group is composed by a group of users that accept (by opt-in choice or non-exercising an opt-out choice) to be engaged in EMPOWERING services, while the control group consists of user with similar characteristics but not receiving the services. The energy consumption of the both groups are analyzed: - prior of the start of services in order to define the baseline values; - during the project to allow a preliminary and ongoing evaluation; - at the end of the project for final evaluation. Fig. 2 General structure for the creation of the control and experimental groups. 7

8 The consumption data are collected by the utilities and transferred to CIMNE for processing. The values of the daily consumption for both electrical and gas measurements are stored and processed. Due to the large amount of data processed it is not possible to avoid outlier values and measurement errors, thus a data cleaning algorithm is mandatory. The data cleaning procedure is essentially based on: - identification and removal of the quantiles related to the lower and upper values that are ties around the core values, that really affect the mean value. This algorithm allows the cancellation of all the measurements close to zero (i.e. users that interrupted the consumption during the project following to transfers, relocation and/or termination of activities) and measurement related to a reduced number of users with consumption much larger than the average. This removal process is carried out by choosing the values of the lower and upper quantiles to be erased from the process. - Identification of consumers that show high differences for energy consumption acquired in the same month and different year, this in order to take into account the different weather conditions measurement. Energy consumption related to a specific user are processed if all the consumptions related to a specific month are related to the minim consumption by the formula: ADC max ADC min / ADC min <= Factor Factor = -8,5*Restrictor + 10 where: ADC ism the average daily consumption computed on monthly base restrictor is an input parameter, chosen by the evaluator during data processing, among a list of available values. factor is the maximum allowed p.u. variation The values of the available restrictor values and the correspondent computed factor values are reported in Table 1 Table 1 Available restrictor parameters and correspondent factor values. Input Parameter: Restrictor Output parameter: Factor 1 1,5 0,95 1,925 0,9 2,35 0,8 3,2 0,75 3,625 0,5 5,75 0 infinite 8

9 Fig. 3 - Design based on experimental and control groups. Fig 4 - Data flow and computation. 2.2 Procedure for the evaluation of the customer satisfaction with EMPOWERING EMPOWERING services have been provided to customers in the five sites, with different characteristics and offering different information according to the specific 'desiderata'. In order to know the information requested by the customers in every site, surveys were carried out in the initial phases of the project to collect the users' requirements in term of needed services, information, data and data visualisation. Those data have been used to define a preliminary version of the EMPOWERING billing/online tools that were 'fine-tuned' after usability and intelligibility tests. 9

10 The preliminary versions of EMPOWERING services were used in the testing phase asking to end users and utilities' internal staff to assess them in term of usability and intelligibility of information. An additional testing activity has been implemented organising in March 2014 a Focus Group with experts on energy management, infographics, sociology, graphics assessed all the services developed by all pilots. Usability and Intelligibility questionnaires with the results from the Focus Group provided useful feedbacks to develop the final version of the EMPOWERING services to be provided to the experimental group. In the following chapters the results related to the impacts of EMPOWERING on utilities' customers are inserted for each pilot. The report is based on information provided by 1715 questionnaires collected during the whole project in different phases, and addressed to different respondents such as customers, technicians and stakeholders. 22 key informants participate to 3 Focus Group, which provided a deep views on issues related to the diffusion of EMPOWERING. The table below summarize the collected questionnaires and data used in the present report. Purposes Table 2 Collected questionnaires and used data Analysis of customers expectations for the design of the services 964 Stakeholders and external organisations involved 31 Usability assessment of services 96 Sociological analysis on end users behaviours in term of energy consumptions 27 Evaluation of the EMPOWERING services 542 Additional questionnaires on the evaluation of EMPOWERING collected from new utilities 55 The information provided by the respondents is organised around the four customer service development and improvement principles: Develop customer insight (investigate customer expectations). Design services around customers (incorporate the customer results in the services development). Respond to individual user needs within available resources. Support employees to provide excellent customer service. Tot 10

11 3. Evaluation of results in each pilot site 3.1 Austria In Austria the services are provided in the city of Linz (Upper Austria) by the utility LINZ AG. Through its subsidiary companies Linz Strom and Linz Strom Netz, it is responsible for energy production, trading, energy related services, telecommunications and electricity distribution. The utility supplies electricity to approximately customers. LINZ AG developed the first version of its web portal and on line customer service for consumption visualization and evaluation of CO 2 emission in 2009 (projects esesh and INTELIKON). EMPOWERING services are offered as additional services to the already existing Energie Informations System (EIS) through on line tools. Fig. 5 - Definition of control and experimental groups in Linz. Customers that want to access to EMPOWERING services have to voluntary subscribe to the services by a two steps registration: first registration to EIS and additional registration to EMPOWERING Fig. 5 shows the procedure for the definition of control and experimental group in Linz, the pilot customers in Linz are customers already registered for EIS service, to access to EMPOWERING services they have to additional opt-it by a voluntary choice. The availability of EMPOWERING services is shown in the EIS site and visitors are encouraged to apply for EMPOWERING. The open access to the services of all the customers by a voluntary registration make impossible to randomly pre-define the control and experimental groups. The procedure adopted carries to the formation of an experimental group composed by always taker and compliance users. The services are on line services: the accesses in the site can be monitored and a sub-group of user, in the experimental group, that really makes use of the service, can be defined. The active user is the user that, after registration, voluntary access to EMPOWERING service by entering in the portal to acquire information. For all the user registered in the experimental group the date of registration is stored, this in order to allow the evaluation of energy savings starting from the effective, specific user dependant, registration date. 11

12 Another subgroup of the experimental group is the test group, that is a group with a preferred contact channel with the utility. This group was selected in order to have fast and effective feedback on the supplied services. The control group is composed by users that decided to not opt-in in EMPOWERING services that are mainly never taker users, since they refused to opt-in the additional service even if informed and asked by the portal. All the users involved in EMPOWERING project (experimental and control group) are equipped with smart meters. Table 3 Site Linz. Total number of users in the experimental and control group Number of users in the experimental and control group experimental group 60 control group The services developed in the Linz site are: - Comparison of specific user monthly consumption with average consumption of the most efficient neighbours and with average consumption of all neighbours. - Yearly representation of comparisons defined at the previous point. - Comparison of specific user monthly consumption with the consumption in the same specific month in the previous year. - Representation of monthly consumption over an energy certification scale. - Display of specific user consumption per flow rate zone as a stack diagram graph. Computation of statistic with average and sum of consumption for every flow-rate zone with display of the costs. This diagram and the statistic are available for all the days, months and years in the past since the start of the data storing service. - Recommendations for good practices in summer, winter or on holidays. - Plot of consumption and average outdoor temperature vs. time. - Alert on suddenly variation of consumption. The definition of similar users is decided by LINZ AG, based on the customer data already stored in their data base, giving particular relevance to the dwelling surface an location identified by postal code. 12

13 Table 4 Energy Saving Valuation Evaluation of Energy Savings Service provided OT on line tools Energy Electrical energy measured values Total period analysed NOV 2013 DEC 2015 Active date January 2015 Saving values Fixed active date - Result for the whole period Group All contracts Total savings results computed from Jan 15 to Dec 15 for all the users involved in the project Estimated average daily consumption of experimental group in absence of service [kwh/day] Actual average daily consumption of experimental group [kwh/day] 7.71 Average daily energy savings [kwh/day] 0.44 Average daily energy savings [%] 5.39 Standard deviation [%] 3.31 Confidence 95% [%] Cleaning parameters restrictor 0.9 Lower quantile 1% Upper quantile 99% 8.15 Total measures experimental group 1178 Total measures control group Customers experimental group 49 Customers control group

14 Fig. 6 - Site: Linz. Average daily consumption computed on monthly base vs time for the control group (blue) and the experimental group (red). Period of evaluation of energy savings January 15 December 15. Fig. 7 Graphical representation of daily energy savings (kwh, %). Scenario for evaluation of customer satisfaction A first survey on users' requirements and expectations on the EMPOWERING services in Austria was launched in January customers answered to an online survey that aimed to investigate what kind of information and data users already receive from their utility and what additional information they need in order to adopt behaviours addressed to save energy. Previously to EMPOWERING, LINZ AG customers had an additional web-based information on energy consumption named EIS (Energy Information System); this is reflected on the answers to the first surveys in which the majority of the respondents declared that they can easily obtain information on their consumption in order to, for example, recognize the differences among their periodic bills or the connection between their actual consumption and the investments they realised to save energy. The 14

15 respondents of the Linz pilot found their energy bill more easy to be read and understood respect the bills of other pilots, the only dissatisfaction point was related to the cost of energy, which has been judged by the end users as too high and not clear regarding its composition. Only the 29% of the respondents declared that they are satisfied with their utility. In those questionnaires, users expressed their preference according to the kind of services to be added to EMPOWERING, the desired data visualisation tools and the request for additional services (e.g. the possibility to download the *.csv files of their consumption). After the processing of the data of the first survey, first versions of the Empowering services has been realised. The first version has been sent to non-partners organisations/stakeholders with a specific questionnaire, in order to collect from them feedbacks and possible suggestions on how to improve the services. In Austria, the following organisations provided feedbacks on EMPOWERING, assessing the developed services as useful for customers and for utilities in order to increase the satisfaction of their customers: - Econgas Gmbh; - OÖ Energiesparverband; - OÖ Gas-Wärme Gmbh; - Energie AG; - E7; - Energie AG Vertrieb Gmbh; - Energie AG OÖ Vertrieb Gmbh & Co KG; - Energie AG After the feedbacks from the above mentioned stakeholders, the developed services have been validated by 15 employees (external to EMPOWERING) of LINZ AG and 18 end users who considered the implemented services usable and understandable. Perceived quality of the EMPOWERING services In the widespread application phase of EMPOWERING, customers who received the services have been asked to fill in a questionnaire in which they can evaluate the services received by their utility. 161 Customers filled in a questionnaire addressed to investigate the expectations and the perceptions of users among different service dimensions. The obtained results (expressed as a indicator for a better readability) and the investigated dimensions are reported in the table below. Table 5 Obtained results EMPOWERING service dimensions Users quality perception score Presence of additional information on energy consumption 73,12 Provision of helps and suggestions on how to save energy 75,16 Provision of handy tools to understand energy consumptions 70,44 Information available at any time 75,16 Globally, Linz customers have very good perceptions on the quality of the EMPOWERING services 15

16 Did EMPOWERING meet users expectations? After the first implementation of the services and before the widespread application phase of Empowering to the whole experimental group, a questionnaire addressed to identify the expectations of the end users in term of the effects/impacts that smart billing services should have has been launched. 25 users expressed an evaluation on several dimensions that constitutes a general smart billing service, providing a quantitative rating (on a 1-5 scale) on their importance. This survey was aimed to evaluate a starting point, a baseline for the final evaluation. From April to November 2015, the same questions have been asked to the EMPOWERING users in the widespread application phase in order to assess changes in the perceived quality and if the services met the initial expectations of end users. 161 end users answered to this survey, using the online form available on the utility s website or the paper-based versions at utility s customer care office. A gap score (Q) have been calculated as the difference between users Perceptions and Expectations for each investigated dimension. For the purpose of the present report, Q-scores have been transformed in a scale. A score equal or major than 50 means that the services meet the expectations and the utility create a good service. The last column shows the improvements on the Quality of EMPOWERING as developed in Linz. The table below shows the assessed dimensions and the Q-scores for each of them: Table 6 - Assessed dimensions and the Q-scores General smart billing service dimensions Q-score (baseline) Q-score (final) Improvements on the Quality of services Presence of additional information on energy consumption Provision of helps and suggestions on how to save energy Provision of handy tools to understand energy consumptions 45,00 51,87 +15,28% 41,25 51,25 +24,24% 36,50 49,87 +36,64% Information available at any time 41,00 50,00 +21,95% 16

17 For all investigated service dimension, an increase on the quality of the EMPOWERING services is recognisable. Changes in customer satisfaction 1 A first question on the satisfaction of customers with their energy provider has been inserted in the first survey carried out in the initial phase of the project, when the requirements of the users to develop the EMPOWERING services were collected. The same question was inserted in the baseline and evaluation questionnaires developed after the implementation of EMPOWERING. The table below shows the changes on the percentage of satisfied customers. Table 7 - % of satisfied customers changes Question User expectations survey (2014) Baseline and Evaluation surveys (2015) Are you satisfied with your energy provider? 29% answered 'YES' 70% answered 'YES' 1 Results on customer satisfaction are related only to services connected to informative services on energy billing. They do not represent the satisfaction of users with their utility company. 17

18 3.2 France The services in France are provided by Gas Electricity of Grenoble (GEG), one of the few independent municipality-owned utilities in France, acting as grid operator, electricity and gas supplier, and energy producer. Prior to EMPOWERING project, GEG was involved in several smart grid projects focused on informative services and demand side management, many GEG s customers in the Grenoble are used to interact with the on line services supplied by GEG. GEG employ traditional energy meters and the readings are performed every 6 moth, the readings are nonsynchronous for all the customers, around 1/6 of the customers are measured per month. Monthly consumptions are computed from the readings by proprietary algorithm developed in GEG. The algorithm was tested and refined in years of field experience, the average daily consumptions are directly obtained from the estimated monthly consumptions. All the customers already registered in the GEG web portal were contacted by and were asked to connect to a separate portal where EMPOWERING services were available ( The open access to the services of all the customers, by a voluntary registration, make impossible to randomly pre-define the control and experimental groups. The procedure adopted carries to the formation of an experimental group composed by always taker and compliance users. The services are on line services, thus the accesses in the site can be monitored and an additional grouping, among user in the experimental group, defined active user group can be identified. For all the users registered in the experimental group the date of registration is stored, this in order to allow the evaluation of energy savings, starting from the registration date, when EMPOWERING services are assumed to be employed. A subgroup of the experimental group is the test group, that is a group with a preferred contact channel with the utility, it was selected in order to have fast and effective feedback on the supplied services. The control group is composed by users that decided to not opt-in in EMPOWERING services that are mainly never taker users, since they refused to opt-in even if informed and asked by the portal. Using the customer data, already available in the GEG data base, the user in the experimental croup were divided in three classes bases on the usage of electric energy and contract power (Table III). The members of the control group were matched to the members of the experimental group following a criterion based on the three groups identified in Table III where are defined: group 1: low contract power and electric energy used for home appliances only, group 2: high contract power and indoor space heating by electric energy, group 3: low contract power but occasional use of electric energy for sanitary hot water and space heating. Table 8 - Site: Grenoble. Grouping of the user involved in EMPOWERING program. Contract power electric appliances Energy use sanitary hot water production electric space heating Number of users control group experimental group group 1 6 kva yes group 2 > 9kVA yes yes yes group 3 6 kva yes occasional occasional

19 The procedure of admission in EMPOWERING services, for groups 1, 2 and 3, is shown in Fig. 351: it is the usual procedure with opt-in choice for the experimental group. After the definition of the experimental group the control group is composed by selecting the members in order to match it on the ground of Table III classification. Fig. 8 - Definition of control and experimental groups in Grenoble. The information available in the portal are: - Actual three months consumption and comparison with the same period of the former year (computed correction due to different weather condition available). - Comparison with consumption of similar dwellings. - Display of monthly consumption during the last year within the limits of maximum and minimum consumptions of similar customers. - Consumption forecast for the following three months. - Display of energy scale for the ranges of consumption per unity dwelling surface and position of the specific customer with respect to the scale. 19

20 Table 9 Energy saving valuation Evaluation of Energy Savings Service provided OT on line tools Energy Electrical energy estimated values Total period analysed NOV 2013 DEC 2015 Active date Date of activation (after Oct. 14) Saving values Date of activation Dec 15 Group Group 1 Total savings results computed from Jan 15 to Dec 15 for group 1 Estimated average daily consumption of experimental group in absence of service [kwh/day] Computed average daily consumption of experimental group [kwh/day] Average daily energy savings [kwh/day] 0.22 Average daily energy savings [%] 3.25 Standard deviation [%] 3.86 Confidence 95% [%] Cleaning parameters restrictor 0.9 Lower quantile 1% Upper quantile 99% Total measures experimental group 875 Total measures control group Customers experimental group 36 Customers control group

21 Fig. 9 - Site: Grenoble Group 1. Average daily consumption computed on monthly base vs time for the control group (blue) and the experimental group (red). Period of plot of ADC (average energy consumption) November 13 October 15. Period of evaluation of energy savings from date of activation (after October 14) to December 15. Fig Graphical representation of daily energy savings (kwh, %). 21

22 Table 10 Energy saving valuation Evaluation of Energy Savings Service provided OT on line tools Energy Electrical energy estimated values Total period analysed NOV 2013 DEC 2015 Active date Date of activation (after Oct. 14) Saving values Date of activation Dec 15 Group Group 2 Total savings results computed from Jan 15 to Dec 15 for group 2 Estimated average daily consumption of experimental group in absence of service [kwh/day] Computed average daily consumption of experimental group [kwh/day] Average daily energy savings [kwh/day] 1.07 Average daily energy savings [%] 6.23 Standard deviation [%] Confidence 95% [%] Cleaning parameters restrictor 0.9 Lower quantile 5% Upper quantile 95% Total measures experimental group 186 Total measures control group Customers experimental group 8 Customers control group

23 Fig Site: Grenoble Group 2. Average daily consumption computed on monthly base vs time for the control group (blue) and the experimental group (red). Period of plot of ADC (average energy consumption) November 13 October 15. Period of evaluation of energy savings from date of activation (after October 14) to December 15. Fig Graphical representation of daily energy savings (kwh, %). 23

24 Table 11 Energy Saving valuation Evaluation of Energy Savings Service provided OT on line tools Energy Electrical energy estimated values Total period analysed NOV 2013 DEC 2015 Active date Date of activation (after Oct. 14) Saving values Date of activation Dec 15 Group Group 3 Total savings results computed from Jan 15 to Dec 15 for group 3 Estimated average daily consumption of experimental group in absence of service [kwh/day] Computed average daily consumption of experimental group [kwh/day] Average daily energy savings [kwh/day] 2.12 Average daily energy savings [%] Standard deviation [%] 8.31 Confidence 95% [%] Cleaning parameters restrictor 0.9 Lower quantile 1% Upper quantile 99% Total measures experimental group 244 Total measures control group Customers experimental group 10 Customers control group

25 Fig Site: Grenoble Group 3. Average daily consumption computed on monthly base vs time for the control group (blue) and the experimental group (red). Period of plot of ADC (average energy consumption) November 13 October 15. Period of evaluation of energy savings from date of activation (after October 14) to December 15. Fig Graphical representation of daily energy savings (kwh, %). 25

26 Gas consumption In Grenoble EMPOWERING services operate also on gas consumption: 500 gas user were selected for the experimental group with the procedure adopted for the electric energy, while the control group is composed by 5000 users. Gas consumption are read every six months and monthly consumption are computed by proprietary algorithm. EMPOWERING services similar to the on line tools available for electric energy have been developed for the gas customers. In particular the following tool are available: - Actual three months consumption and comparison with the same period of the former year. Bothe measured consumption and corrected consumption, to take into account different weather conditions are available. - Consumption forecast for the following three months Gas consumption have been evaluated in the same way of electrical consumption, the results of the treatment are shown in the following tables and diagrams. 26

27 Table 12 Energy Saving valuation Evaluation of Energy Savings Service provided OT on line tools Energy Gas estimated values Total period analysed NOV 2013 DEC 2015 Active date Date of activation (after Oct. 14) Saving values Date of activation Dec 15 Group All (gas) users Total savings results computed from Jan 15 to Dec 15 for all gas users Estimated average daily consumption of experimental group in absence of service [kwh/day] Computed average daily consumption of experimental group [kwh/day] Average daily energy savings [kwh/day] 0.79 Average daily energy savings [%] 4.32 Standard deviation [%] 8.97 Confidence 95% [%] Cleaning parameters restrictor 0.9 Lower quantile 5% Upper quantile 95% Total measures experimental group 275 Total measures control group Customers experimental group 12 Customers control group

28 Fig Site: Grenoble Gas consumption. Average daily consumption computed on monthly base vs time for the control group (blue) and the experimental group (red). Period of plot of ADC (average energy consumption) November 13 October 15. Period of evaluation of energy savings from date of activation (after October 14) to December 15. Fig Graphical representation of daily energy savings (kwh, %). Scenario for evaluation of customer satisfaction A first survey on users' requirements and expectations on the EMPOWERING services in France was launched in July customers answered to an online survey that aimed to investigate what kind of information and data users already receive from their utility and what additional information they need in order to adopt behaviours addressed to save energy. The majority of GEG's customers (60%) declared from the beginning that they can understand their bill, but is not so easy to obtain information on how to compare past and present consumption. Cost composition 28

29 appears clear, and a great part of the respondents declared that they are available to read the meter once a month and to communicate the values to the utility in exchange of more detailed information on their consumptions. A relevant part of respondents, found difficult to understand the amount of energy over the period using the bill and to compare current power consumption with the past consumptions. Analysis of the French scenario was deepen organising a focus group (June 2013) with relevant stakeholders, such as customer representatives, trade unions, end users. The participant to focus groups highlighted that the French energy bill lack on information on the composition of the price of energy per consumed kwh. In this first phase of survey, users expressed their preference according to the kind of services to be added to EMPOWERING, the desired data visualisation tools and the request for additional services, expressing in particular (in accordance with their comments on the capacity to understand their energy bills) the need to receive information to compare present and past consumptions and advices and recommendations to save energy. After the focus group with stakeholders and the processing of the results from the survey, the developed services have been validated by 13 employees (external to EMPOWERING) of GEG and 5 end users who considered the implemented services usable and understandable. Perceived quality of the EMPOWERING services In the widespread application phase of EMPOWERING, customers who received the services have been asked to fill in a questionnaire in which they can evaluate the services received by their utility. 121 Customers filled in a questionnaire addressed to investigate the expectations and the perceptions of users among different service dimensions. The obtained results (expressed as a indicator for a better readability) and the investigated dimensions are reported in the table below. Table 13 Obtained Results EMPOWERING service dimensions Users quality perception score Presence of additional information on energy consumption 89,09 Provision of helps and suggestions on how to save energy 93,22 Provision of handy tools to understand energy consumptions 83,14 Information available at any time 88,92 Globally, Grenoble customers have very good perceptions on the quality of the EMPOWERING services 29

30 Did EMPOWERING meet users expectations? After the first implementation of the services and before the widespread application phase of Empowering to the whole experimental group, a questionnaire addressed to identify the expectations of the end users in term of the effects/impacts that smart billing services should have has been launched. 25 users expressed an evaluation on several dimensions that constitutes a general smart billing service, providing a quantitative rating (on a 1-5 scale) on their importance. This survey was aimed to evaluate a starting point, a baseline for the final evaluation. From April to November 2015, the same questions have been asked to the EMPOWERING users in the widespread application phase in order to assess changes in the perceived quality and if the services met the initial expectations of end users. 121 end users answered to this survey available in the websites of GEG and ALEC. A gap score (Q) have been calculated as the difference between users Perceptions and Expectations for each investigated dimension. For the purpose of the present report, Q-scores have been transformed in a scale. A score equal or major than 50 means that the services meet the expectations and the utility create a good service. The last column shows the improvements on the Quality of EMPOWERING as developed in Grenoble. The table below shows the assessed dimensions and the Q-scores for each of them. Table 14 - Assessed dimensions and the Q-scores General smart billing service dimensions Q-score (baseline) Q-score (final) Improvements on the Quality of services Presence of additional information on energy consumption Provision of helps and suggestions on how to save energy Provision of handy tools to understand energy consumptions 57,00 63,87 +12,06% 67,00 76,62 +14,37% 63,00 70,62 +12,10% Information available at any time 59,00 64,00 +8,47% For all investigated service dimension, an increase on the quality of the EMPOWERING services is recognisable. 30

31 Changes in customer satisfaction 2 A first question on the satisfaction of customers with their energy provider has been inserted in the first survey carried out in the initial phase of the project, when the requirements of the users to develop the EMPOWERING services were collected. The same question was inserted in the baseline and evaluation questionnaires developed after the implementation of EMPOWERING. The table below shows the changes on the percentage of satisfied customers. Table 15 - % of satisfied customers changes. Question User expectations survey (2013) Baseline and Evaluation surveys (2015) Are you satisfied with your energy provider? Not assessed in a quantitative way but there were indications on dissatisfaction related to lack on transparency of the bill 60,95% answered YES 2 Results on customer satisfaction are related only to services connected to informative services on energy billing. They do not represent the satisfaction of users with their utility company 31

32 3.3 Italy The services in Italy are provided by IREN Energia, which is supplier of the services for electricity and district heating distribution in the city of Torino and for district heating distribution in the city of Reggio Emilia. IREN Energia operates as DSO in the city of Torino. The whole city is equipped with smart meters, the company AEM Torino Distribuzione S.p.A. (subsidiary of IREN Energia) is charged of various services related to smart meters, in particular maintenance service and continuous reading. Electric energy measures are available with the granularity of 15 min, thus starting from the raw data acquired by smart meters, daily energy consumptions can be computed. Following the Italian regulations IREN Energia is separated from the various retailer companies that operates as final sellers on the market. The final users do not have direct contact with IREN Energia because every final user refers to his retailer for all the problems related to electric energy supply. The readings acquired by the smart meters are collected by IREN Energia and then send to the retailers that compute, following the specific contract of each final user, the amount of the invoices. IREN Energia has the knowledge of the energy consumption but it do not have access to the final price of energy and billing amounts. In order to select the population for EMPOWERING project a recruitment campaign, based on newspaper advertising, was started. In addition to the newspaper campaign IREN Energia selected 3000 around valid s from his data base, related to electric users in Torino, and send to them a message to encourage to joint to EMPOWERING program. Both the s and newspaper advertisements were asking to final users to volunteer register themselves in the IREN Energia site, to give consent for EMPOWERING program and fill a questionnaire. The questionnaire contents few simple questions to allow a class definition of the user involved in EMPOWERING: - number and age (adults/children) of people living in the dwelling - dwelling surface - information about the presence and age home appliances in the dwelling (microwaves oven, refrigerator, dishwasher, washing machine ) At the end of the informative campaign less than 200 customers registered themselves on the site, this users were included in the experimental group. In order to reach the scheduled population for the program (2000 unities), 1800 customers, selected in the IREN Energia data base, were included in the program, without asking for previous consent, but offering them an opt-out option. The experimental design adopted in Torino is shown in Fig. 401, that depict the main selection channel for the customers to be included in the project. The selected pilot customers are random assigned to the experimental and control groups and opt-out choice is available to whose that are assigned to the experimental group. Due to the selection process adopted both experimental and control groups are surely composed by always taker, compliers in addition some never taker users were included in the experimental group and opt-out option is offered. Since EMPOWERING services are free of charge only active never taker user exercised the opt-out option. 32

33 Fig Definition of control and experimental groups in Torino. The informative service available in the site of Torino is a monthly report about electric consumption. The first report was related to January 2015 and the service is scheduled to operate up to the end of the project. The information reported in the monthly report are: - comparison of specific user monthly consumption with the same month in the previous year. - Comparison of specific user monthly consumption with the best users in the same class and with average user in the class. - Total consumption of the year (from 1 st January): value and percentage referred to the previous year consumption. - Plot of daily consumption vs. time for the analysed month and comparison with the same class average user daily consumption. The informative reports are available at the end of the month with 30 days delay. In additional to the numeric values, an emoticon is added to give a concise and effective information. The service is available to the customers is a billing tool, the monthly reports are send to the customers by . Table 16 Number of users Number of users in the experimental and control group experimental group control group 200 opt-in + registration 1600 included with opt-out option available For a total opt-out option available 33

34 The evaluation of energy saving is carried out taking into account two separate groups defined as: group 1: consumers with contract power of 3 kw A contract power of 3 kw is the usual contract for family use in Italy. For this contract power a particular contract defined enhanced protection contract is available; this kind of contract is a contract that comes from the electric market structured as a monopole, before the opening to competitive operators. group 2: consumers with contract power greater than 3 kw Both group use electric energy for home appliances and probably some users have electric boiler for hot sanitary water production, it can be excluded the use of space electric heating except for short term emergency use. Air cooling systems are assumed to be included in electric appliances, both group1 and group 2 can have air cooling systems. 34

35 Table 17 Energy Saving valuation Evaluation of Energy Savings Service provided BT Billing tools Energy Electrical energy measured values Total period analysed NOV 2013 DEC 2015 Active date January 2015 Saving values Fixed active date - Result for the whole period Group Group 1 Total savings results computed from Jan 15 to Dec 15 for group 1 Estimated average daily consumption of experimental group in absence of service [kwh/day] Actual average daily consumption of experimental group [kwh/day] 3.95 Average daily energy savings [kwh/day] 0.23 Average daily energy savings [%] 5.46 Standard deviation [%] Confidence 95% [%] Cleaning parameters restrictor 0.9 Lower quantile 1% Upper quantile 99% 4.18 Total measures experimental group 297 Total measures control group 26 Customers experimental group 9 Customers control group 1 35

36 Fig Site: Torino Group 1. Average daily consumption computed on monthly base vs time for the control group (blue) and the experimental group (red). Period of evaluation of energy savings January 15 December 15. Fig Graphical representation of daily energy savings (kwh, %). 36

37 Table 18 Energy Savings valuation Evaluation of Energy Savings Service provided BT Billing tools Energy Electrical energy measured values Total period analysed NOV 2013 DEC 2015 Active date January 2015 Saving values Fixed active date - Result for the whole period Group Group 2 Total savings results computed from Jan 15 to Dec 15 for group 2 Estimated average daily consumption of experimental group in absence of service [kwh/day] Actual average daily consumption of experimental group [kwh/day] Average daily energy savings [kwh/day] Average daily energy savings [%] Standard deviation [%] 5.25 Confidence 95% [%] Cleaning parameters restrictor 0.9 Lower quantile 2% Upper quantile 98% Total measures experimental group 5910 Total measures control group 950 Customers experimental group 299 Customers control group 39 37

38 Fig. 20 Site: Torino Group 2. Average daily consumption computed on monthly base vs time for the control group (blue) and the experimental group (red). Period of evaluation of energy savings January 15 December 15. Fig Graphical representation of daily energy savings (kwh, %). District heating services EMPOWERING project acts on district heating services in the town of Torino and Reggio Emilia. The standard heating season is from 15 October up to 15 April, the city managers, following particular cold weather conditions, can authorise to switch on the district heating outside the standard period. From the heating season up to October 2015 there was no necessity to change the standard heating periods. In Torino the selected buildings to be involved in EMPOWERING program are condominiums. There is only one connection point between the district heating network and the building, where the heat exchanger and 38

39 the measurements units are located. The whole condominium is assumed as a single unit and only one fiscal bill is sent to the building manager that take care of all the contacts with IREN Energia. Following the specific rules of every condominium the invoice of IREN Energia is spitted on the flats. At least one meeting per year has to be organised by the building manager to discuss the budget, during the meeting the cost and consumption of district heating are analysed. Information exchange during the year from the building manager to the users depends on the specific rules and habits of the condominium, the contact between the building manger and the dwellers are usually carried out by printed letters or s. In Torino the experimental group is composed by 23 condominiums for a total volume of m 3, while the control is composed by 4 condominiums for a total volume of m 3. In Reggio Emilia the users involved in EMPOWERING project are both condominiums and single family houses. The condominiums are controlled by the Social Housing Agency. The Municipality of Reggio Emilia asked to involve these buildings in EMPOWERING with the goal to improve the education and awareness about the use of energy of the tenants. These condominiums receive only one invoice, concerning to the consumption of the whole building, then the building manager computed the amounts related every flat. The family houses are independent houses, distributed all over the city, the invoice is sent directly to the owners that live in the house. The total number of user involved in EMPOWERING project is 539 user for the experimental group, while 60 users are in the control group. The condominiums are one third around of the total users. Table 19 Number of users Number of users in the experimental and control group for the DH Torino experimental group Torino control group Reggio Emilia experimental group Reggio Emilia control group 23 (opt-out option available) 4 (opt-out option available) 539 (opt-out option available) 60 ((opt-out option available) Scenario for evaluation of customer satisfaction A first survey on users' requirements and expectations on the EMPOWERING services in Italy was launched in July customers answered to an online and paper based survey that aimed to investigate what kind of information and data users already receive from their utility and what additional information they need in order to adopt behaviours addressed to save energy. The majority of customers, both in Turin and Reggio Emilia, showed dissatisfaction on the services of their utility and considered their bill as difficult to be read and not transparent. In particular the dissatisfaction is higher among the district heating users. Analysis of the Italian scenario was deepen organising a focus group (June 2013) with relevant stakeholders, such as customer representatives, trade unions, end users and with 2 meetings with customers in 2 buildings initially involved in the test group and one-to-one interviews to almost all the involved families in Reggio Emilia. Users engagement activities and the confrontation with stakeholders In this first phase of survey, users expressed their preference according to the kind of services to be added to EMPOWERING, the desired data visualisation tools and the request for additional services, expressing in particular (in accordance with their comments on the capacity to understand their energy bills) the need to receive information to compare present and past consumptions and advices and recommendations to save energy, in addition to the necessity to insert information on cost composition. 39

40 After the implementation of the prototypes of services, they have been assessed by stakeholders and nonpartner organisation, who provided feedback on the usefulness of the smart billing tools developed by IREN. The involved non partners organisations were: - Social services of Reggio Emilia; - Representatives of households; - Social housing organisations; - Building managers; - The CGIL trade union After the focus group with stakeholders and the processing of the results from the survey, the developed services have been validated by 11 employees (external to EMPOWERING) of IREN and 15 end users who considered the implemented services usable and understandable. Perceived quality of the EMPOWERING services In the widespread application phase of EMPOWERING, customers who received the services have been asked to fill in a questionnaire in which they can evaluate the services received by their utility. 128 Customers filled in a questionnaire addressed to investigate the expectations and the perceptions of users among different service dimensions. The obtained results (expressed as a indicator for a better readability) and the investigated dimensions are reported in the table below. Table 20 Obtained Results EMPOWERING service dimensions Users quality perception score Presence of additional information on energy consumption 50,15 Provision of helps and suggestions on how to save energy 42,83 Provision of handy tools to understand energy consumptions 44,06 Information available at any time 45,16 Globally, Turin and Reggio Emilia customers have very good perceptions on the quality of the EMPOWERING services. 40

41 Did EMPOWERING meet users expectations? After the first implementation of the services and before the widespread application phase of Empowering to the whole experimental group, a questionnaire addressed to identify the expectations of the end users in term of the effects/impacts that smart billing services should have has been launched. 25 users expressed an evaluation on several dimensions that constitutes a general smart billing service, providing a quantitative rating (on a 1-5 scale) on their importance. This survey was aimed to evaluate a starting point, a baseline for the final evaluation. From April to November 2015, the same questions have been asked to the EMPOWERING users in the widespread application phase in order to assess changes in the perceived quality and if the services met the initial expectations of end users. 128 end users answered to this survey via or filling in a paper-based questionnaire available at IREN and municipality premises. In pilot of Reggio Emilia, part of the questionnaires have been collected during specific meetings organised with households involved in EMPOWERING. These meetings with households were held in November 18 th and 19 th, with the participation of Vittorio Gimigliano (RE Municipality), Ivan Arduini (ACER- Reggio Emilia), Marina Pirazzi (Extrafondente) and Federico Boni Castagnetti (IREN). Combining the questionnaires that they filled-in and their opinion expressed in the conversation, a few interesting comments came out. The first and most stressed comment was that the information conveyed by the additional Empowering info sheet was of limited utility because it just states that a household is consuming more than a comparable household, yet it does not tell the cause of it. In other words, it spreads the alarm without offering an explanation that could help the client to mitigate the damage. The second collected comment was about the (apparent) inconsistence of the high consumption red face alarm and the consumption of the past year, which, on the contrary, the consumers did not reckon higher than the present year. This comment offered IREN the opportunity to check the system and possible errors. Finally, the meetings confirmed the necessity to continue the distribution of printed info sheets (vs e-info) in not electronically literate environments that are still widespread in Italy. In processing the questionnaires data, a gap score (Q) have been calculated as the difference between users Perceptions and Expectations for each investigated dimension. For the purpose of the present report, Q-scores have been transformed in a scale. A score equal or major than 50 means that the services meet the expectations and the utility create a good service. The last column shows the improvements on the Quality of EMPOWERING as developed in Turin and Reggio Emilia. The table below shows the assessed dimensions and the Q-scores for each of them Table 21 - Assessed dimensions and the Q-scores General smart billing service dimensions Q-score (baseline) Q-score (final) Improvements on the Quality of services Presence of additional information on energy consumption Provision of helps and suggestions on how to save energy Provision of handy tools to understand energy consumptions 19,50 26,87 +37,82% 17,00 20,75 +22,06% 18,00 19,87 +10,42% Information available at any time 22,00 22,12 +0,57% For all investigated service dimension, an increase on the quality of the EMPOWERING services is recognisable. The low Q-scores are affected of high expectations scores. Analysing the questionnaires, 41

42 expectations scores exceeded the perceptions one. As showed in the paragraph before, perceptions about the services developed in Italy are good, and great efforts have been made in order to meet the users expectations as showed in the table above. During the project lot of improvements have been made and this is reflected on the improving Q-score continuously monitored during the project. Changes in customer satisfaction 3 A first question on the satisfaction of customers with their energy provider has been inserted in the first survey carried out in the initial phase of the project, when the requirements of the users to develop the EMPOWERING services were collected. The same question was inserted in the baseline and evaluation questionnaires developed after the implementation of EMPOWERING. The table below shows the changes on the percentage of satisfied customers. The low rate reflects the high expectations. Table 22 - % satisfied customers changes Question User expectations survey (2013) Baseline and Evaluation surveys (2015) Are you satisfied with your energy provider? 24,5% 23,37% 3 Results on customer satisfaction are related only to services connected to informative services on energy billing. They do not represent the satisfaction of users with their utility company 42

43 3.4 Spain The services in Spain are provided at the pilot ELGAS by Grup ElGas in the city of Sóller, Mallorca. ElGas is a pioneer energy in Mallorca which is operating as local energy distributor with the company Vall de Sóller Energia, operates as electricity retailer through the company Electrica Sollerense and participates in companies producing renewable energy from wind, solar or hydraulic power. ElGas has deployed smart meters to all of its around 9000 residential customers which are all included in the pilot. The daily consumption required for EMPOWERING project are computed from the higher frequency measurements carried out by the smart meters. The services at the pilot started on April 2015 by offering them on the online portal while the dispatch of the energy reports started in May 2015 and will operate up to March The customers are initially randomly assigned to the experimental group or to the control group, but only the member of the experimental group have access to the services. The services are provided to the customers in two lines: through an on line portal (on line tool) and by an informative report attached to the energy bill (billing tool). Due to the multi-national population of the customers in Sóller the services are offered in 3 languages (English, Spanish and Catalan). Both lines provided the following services: - Comparison of specific customer monthly consumption with the average consumption of the most efficient neighbours and with the consumption of the average neighbour. - Plot of specific customer monthly consumption, most efficient neighbours monthly consumption and average neighbours monthly consumption vs. time (last 12 months). Multiple graphic options available for the on line tool. - Plot of specific customer monthly consumption for the last 12 months and comparison with the consumption in the previous year. - Plot of specific use consumption and average outdoor temperature. - Alert for anomalous increment of electric consumption Billing tool The energy reports are sent attached to the energy bill every 2 months to all the user assigned to the experimental group, opt-out option is available. Since the additional information supplied by EMPOWERING are contemporary to the billing they can immediately help to evaluate the amount of the invoice. The informative services and graphs are generated each month, the report attached to the bill is related to the last month. The experimental design for the billing tool is shown in Fig

44 Fig Definition of control and experimental groups for billing tool in Soller. Due to the selection process adopted both experimental and control groups are surely composed by always taker, compliers. Never taker users are automatically included in the experimental group and opt-out option is offered. Since EMPOWERING services are free of charge only active never taker user exercised the opt-out option. On line tool The users are encouraged to register themselves in the on line portal in order to access to the online tools. The active registration in the portal is equivalent to an opt-in option to the EMPOWERING on line tools, the users assigned to the control group have no access to the on line services (the option is not active on the screen). All the users are informed about the availability of on line tools, if a user, assigned to the control group, asks to have access to the on line information, his access is authorized but he is contemporary dropped out the control group. The experimental design for the on line tool is shown in Fig

45 Fig Definition of control and experimental groups for on line tool in Soller. Since opt in is required to access the on line tool, the experimental group for the on line tool is composed by always taker and complier customers. Table 23 Number of users Number of users in the experimental and control group experimental group control group selected for BT 364 selected for BF and registered for OT 45

46 Table 24 Energy Savings valuation Evaluation of Energy Savings Service provided BT Billing tools Energy Electrical energy measured values Total period analysed NOV 2013 DEC 2015 Active date May 2015 Saving values Fixed active date - Result for the whole period Group All contracts Total savings results computed from Jan 15 to Dec 15 for all the users involved in the project Estimated average daily consumption of experimental group in absence of service [kwh/day] Actual average daily consumption of experimental group [kwh/day] Average daily energy savings [kwh/day] 0.70 Average daily energy savings [%] 6.05 Standard deviation [%] 1.34 Confidence 95% [%] Cleaning parameters restrictor 0.9 Lower quantile 2% Upper quantile 98% Total measures experimental group Total measures control group Customers experimental group 2978 Customers control group

47 Fig Site: Soller. Average daily consumption computed on monthly base vs time for the control group (blue) and the experimental group (red). Period of evaluation of energy savings May 15 December 15. Fig Graphical representation of daily energy savings (kwh, %). 47

48 Table 25 Energy Savings valuation Evaluation of Energy Savings Service provided BT Billing tools + OT on line tools Energy Electrical energy measured values Total period analysed NOV 2013 DEC 2015 Active date May 2015 Saving values Fixed active date - Result for the whole period Group All contracts Total savings results computed from Jan 15 to Dec 15 for the active users in the experimental group Estimated average daily consumption of experimental group in absence of service [kwh/day] Actual average daily consumption of experimental group [kwh/day] Average daily energy savings [kwh/day] 1.50 Average daily energy savings [%] Standard deviation [%] 3.11 Confidence 95% [%] Cleaning parameters restrictor 0.9 Lower quantile 5% Upper quantile 95% Total measures experimental group 3917 Total measures control group Customers experimental group 151 Customers control group

49 Fig Site: Soller. Average daily consumption computed on monthly base vs time for the control group (blue) and the experimental group (red). Active users (OT + BT). Period of evaluation of energy savings May 15 December 15. Fig Graphical representation of daily energy savings (kwh, %). Scenario for evaluation of customer satisfaction A first survey on users' requirements and expectations on the EMPOWERING services in Spain was launched in July customers answered to an online and paper based survey that aimed to investigate what kind of information and data users already receive from their utility and what additional information they need in order to adopt behaviours addressed to save energy. The majority (56%) of customers showed satisfaction on the services of their utility, but a relevant part did not answer. Among 49

50 the dissatisfied, the main comments are related to costs. In addition, more than an half of the respondents found it difficult to understand the consumed amount of energy over the period using the bill. In this first phase of survey, users expressed their preference according to the kind of services to be added to EMPOWERING, the desired data visualisation tools and the request for additional services, expressing in particular (in accordance with their comments on the capacity to understand their energy bills) the need to receive information to compare present and past consumptions and advices and recommendations to save energy,. After the implementation of the prototypes of services, they have been assessed by stakeholders and nonpartner organisation, who provided feedback on the usefulness of the smart billing tools developed by EL GAS. The involved non partners organisations were: - Government of Catalunya - Som Energia SCCL After the focus group with stakeholders and the processing of the results from the survey, the developed services have been validated by 8 employees (external to EMPOWERING) of EL GAS and 5 end users who considered the implemented services usable and understandable. Perceived quality of the EMPOWERING services In the widespread application phase of EMPOWERING, customers who received the services have been asked to fill in a questionnaire in which they can evaluate the services received by their utility. 32 Customers filled in a questionnaire addressed to investigate the expectations and the perceptions of users among different service dimensions. The obtained results (expressed as a indicator for a better readability) and the investigated dimensions are reported in the table below. Table 26 Obtained Results EMPOWERING service dimensions Users quality perception score Presence of additional information on energy consumption Provision of helps and suggestions on how to save energy Provision of handy tools to understand energy consumptions Information available at any time Globally, Sollér customers have very good perceptions on the quality of the EMPOWERING services. 50

51 Did EMPOWERING meet users expectations? After the first implementation of the services and before the widespread application phase of Empowering to the whole experimental group, a questionnaire addressed to identify the expectations of the end users in term of the effects/impacts that smart billing services should have has been launched. 25 users expressed an evaluation on several dimensions that constitutes a general smart billing service, providing a quantitative rating (on a 1-5 scale) on their importance. This survey was aimed to evaluate a starting point, a baseline for the final evaluation. From April to November 2015, the same questions have been asked to the EMPOWERING users in the widespread application phase in order to assess changes in the perceived quality and if the services met the initial expectations of end users. 32 end users answered to this survey via or filling in a paper-based questionnaire available at EL GAS premises. In processing the questionnaires data, a gap score (Q) have been calculated as the difference between users Perceptions and Expectations for each investigated dimension. For the purpose of the present report, Q-scores have been transformed in a scale. A score equal or major than 50 means that the services meet the expectations and the utility create a good service. The last column shows the improvements on the Quality of EMPOWERING as developed in Turin and Reggio Emilia. The table below shows the assessed dimensions and the Q-scores for each of them. Table 27 - Assessed dimensions and the Q-scores General smart billing service dimensions Q-score (baseline) Q-score (final) Improvements on the Quality of services Presence of additional information on energy consumption Provision of helps and suggestions on how to save energy Provision of handy tools to understand energy consumptions % % % Information available at any time % For all investigated service dimension, an increase on the quality of the EMPOWERING services is recognisable. The low Q-scores are affected of high expectations scores. Analysing the questionnaires, expectations scores exceeded the perceptions one. As showed in the paragraph before, perceptions about 51

52 the services developed in Spain are very good, and great efforts have been made in order to meet the users expectations as showed in the table above. During the project lot of improvements have been made and this is reflected on the improving Q-score continuously monitored during the project. This is reflected also in the results on customer satisfaction. Changes in customer satisfaction 4 A first question on the satisfaction of customers with their energy provider has been inserted in the first survey carried out in the initial phase of the project, when the requirements of the users to develop the EMPOWERING services were collected. The same question was inserted in the baseline and evaluation questionnaires developed after the implementation of EMPOWERING. The table below shows the changes on the percentage of satisfied customers. Table 28 - % of satisfied customers changes Question User expectations survey (2013) Baseline and Evaluation surveys (2015) Are you satisfied with your energy provider? 56% 71.92% 4 Results on customer satisfaction are related only to services connected to informative services on energy billing. They do not represent the satisfaction of users with their utility company 52

53 4. Conclusions The responsible use of energy is an activity that implies several aspects related to both the kind of energy involved and to use the energy itself. Electrical energy, that was analyzed in all the sites involved in EMPOWERING project, is a particularly difficult area within which to promote sustainable consumption, even in targeted groups. The electric energy usage in civil application, is involved in ICT processes, home appliances, sanitary hot water production, control and regulation of indoor spaces to get comfortable conditions of temperature and humidity. A responsible use of electric energy starts from the choosing electricity from renewable or other less environmentally detrimental sources, and then there is conscious choice of appliances, and related duration, and, at the end, a responsible use of electric apparatus. Final users are in general not necessary expert in the field of energy or have specific knowledge on good practices in energy usage; in order to obtain an improvement in the use of energy the first step is the information and the education of final users. To get effective results the information have to be transferred taking into account that both the aspect of: - specific information contents (i.e. daily consumption, monthly consumption...) - way and procedure adopted to transfer the information (i.e. mail, , web portal,...) EMPOWERING project operated in five different sites, producing different results in each pilot site. The comparative analysis of the results allows a qualitative information on possible best practices that could be adopted in the future, when informative billing services will be spread over a larger number of customers. The final goal of the information transfer is the improvement of final uses behavior, the specific information transmitted is not able to grant by itself a result, but also the transfer process i.e. adopted channel and timing is an essential component of final results. The general rule to transfer well explained and timing information has to adapted to the specific site condition. In EMPOWERING project similar information were transferred by different channels, the comparison of the results can give suggestion to best practice to adopted in the future. Final users have to involved as active parts in the process starting from the beginning. The solution to send an containing information about energy usage is a one way channel, it is difficult to control if the final user has really read the mail and understood the content, a request of receipt is generally discouraged due to the big number of customers. Moreover the user receive the mail when they are checking their mail account, but this do not means that it is the best instant to send the information. A more effective solution is to engage the final user by a specific web portal, where the final user can play an active role in looking for the information, in this way the contents are transferred when the final user is prepared to receive them, since he is looking for them. Information on a web site are continuously available and the delay between energy consumption and information can be forced to the minimum feasible value. In this way the link between behavior and consumed energy and related costs (billing cost, social cost, ecologic cost) can be emphasized. The reduction of time delay between the use of energy and the measurement of energy can help the direct and immediate control of the effectiveness of user s actions for energy saving, since nearly real time feedback is available. The inclusion of information by a billing tool received together with invoice looks to be an interesting solution because information are transferred in a key instant of the relationship between customer and utility, when the customer is prepared to receive tips about energy and money savings. 53

54 The availability of a web portal together with billing tool is the most recommended solution: information are always available in the portal and billing tools acts as periodic recommendation to register and keep being informed by regular access to the portal. A training period look to be necessary to the customer to really skilled to access to the portal and obtain information, better results are obtained when web portals are available from years and the customers are used to access them. The time required by the average user to be familiar with the service has to be taken inti account during the experimental design. If opt-out option is made available the service is supplied to larger amount of consumers. People not initially interested in the services have time to get in contact with it and start to evaluate it during the time. By the analysis of the different situations occurred in the different sites in EMPOWERING project the transfer of information is more effective on the energy consumption when higher energy consumption are present. This is due to the general greater interest in energy associated to the higher costs and to the fact that energy is use to heat and cooling production. The better use of heat and cooling affects directly the amount of primary energy employed. On the other hand high energy reduction in home appliance can be obtained by renewing themselves, but this implies the substitution of still working appliances with newer more efficient. On the basis of the 442 questionnaires collected in the 4 pilots, it is possible to conclude that EMPOWERING obtained a good acceptance among their users. This is detectable analysing globally all the collected questionnaires. The perceived quality of the implemented services is good in all pilots. Rating the quality perceptions of users on a scale 0 (low quality) to 100 (high quality), the evaluation of EMPOWERING given by 442 users is 70,81. Considering the expectations preliminary to the widespread application of EMPOWERING, the surveys show that in all the involved countries the utilities are made improvements on the implemented services in order to majorly meet the users expectations and therefore improve the quality of the provided services. The chart below show the average increase of the gap-score Q, identifying how much the developed services meet the users expectations, in the period comprised between the first implementation of empowering in 2014 and the widespread application phase of the final part of

55 The scores reported above are reflected on the average satisfaction of users with Empowering. The 67% of users addressed with the EMPOWERING services in the 4 pilots are satisfied with the services they received. The creation and transfer of information force all the actors in the energy chain: production, transmission, distribution, final use to redefine their relations in a more collaborative way. This is a not negligible effect that EMPOWERING project produced, since long term results are more important than actual energy savings. Sustainable energy saving, with reduction of CO 2 emission, can be achieved by a more collaborative relationship between users, utilities and DSOs that can lead to a more responsible energy usage. 55