Research Report. Customer Satisfaction Monitoring Prepared for: Ombudsman Services

Transcription

1 Research Report Customer Satisfaction Monitoring 2017 Prepared for: Ombudsman Services

2 Customer Satisfaction Monitoring 2017 Prepared for: Ombudsman Services Prepared by: Steve Handley, Research Director Date: February 2018 Produced by BMG Research BMG Research Ltd, Project: 1181 Registered in England No Registered office: BMG Research Beech House Greenfield Crescent Edgbaston Birmingham B15 3BE UK Tel: +44 (0) UK VAT Registration No Birmingham Chamber of Commerce Member No. B4626 Market Research Society Company Partner The provision of Market Research Services in accordance with ISO 20252:2012 The provision of Market Research Services in accordance with ISO 9001:2008 The International Standard for Information Security Management ISO 27001:2013 Investors in People Standard - Certificate No. WMQC 0614 Interviewer Quality Control Scheme (IQCS) Member Company Registered under the Data Protection Act - Registration No. Z A Fair Data organisation Cyber Essentials certification The BMG Research logo is a trade mark of BMG Research Ltd.

3 Introduction Table of Contents 1 Introduction Background Methodology The report Sample profile Key Findings Overall Satisfaction Customer focus throughout the customer journey Speed and the quality of customer service Signposting and awareness of Ombudsman Services Website usage and feedback Portal usage and feedback Delivering for customers Service awareness and website use Signposting and awareness of Ombudsman Services Website usage Website usefulness Initial contact with Ombudsman Services The Investigation The Outcome Delivering against expectations Remedy implementation Portal Usage and Feedback Overall satisfaction measures Case handling Effort perceptions Net promoter score Suggestions for improvement Appendix: Statement of Terms

4 Customer Satisfaction Monitoring Introduction 1.1 Background This report summarises the findings of the customer satisfaction research conducted among consumers whose case was investigated and closed by Ombudsman Services (OS) in In 2017 the delivery of the customer satisfaction research was transitioned from a telephone sample to an online approach where all individuals who have had a case closed are invited to provide feedback on their service experiences (assuming a valid address is provided). This switch in methodology was made on the basis of: The extremely high incidence of addresses among the consumers using Ombudsman Services for complaint resolution. The potential to produce monthly sample sizes that are significantly larger relative to the previous telephone survey, with no additional investment. The data in this report is based on over 5,000 completed surveys, with this larger sample size increasing both the statistical robustness of the data and the potential for it to be explored in greater depth. The ability to link the real time survey returns to an online reporting dashboard, thus providing Ombudsman Services with direct access to their latest customer feedback scores. Alongside the transition to the online methodology, the survey content itself was redesigned to ensure it was fully aligned with the customer journey that consumers experience and so that consistent metrics are recorded throughout that journey. The survey also now includes measures of customer effort and how this compares to the effort OS is perceived to have made on their behalf. 1.2 Methodology At the start of each calendar month, Ombudsman Services provide BMG Research the details of the cases closed from the previous calendar month. After checking these cases for duplicates, BMG Research sends a survey invite to each individual in the first week of each month, with the survey closing at the end of that month. The survey link sent to each consumer is unique, allowing the details already know about the case, (sector, decision and outcome) to be linked to the feedback provided by consumers. Please note however, that BMG Research does not provide feedback to Ombudsman Services that can be directly attributable to a particular case. Any individual responding to the survey invite with a query about their case e.g. progress regarding redress implementation, were signposted back to Ombudsman Services. 2

5 Introduction In total, 35,801 surveys invites were sent during 2017 and BMG received a total of 5,161 completed surveys giving an overall response rate of 14%. Table 1: Returns and response rates per month Date in which case closed Sent out Completed surveys Response rate Jan % Feb % March % April % May % June % July % August % Sept % Oct % Nov % Dec % Total 35, % The total survey sample size of 5161 is subject to a maximum standard error of +/- 1.26% at the 95% confidence level on an observed statistic of 50%.Therefore, we can be 95% confident that the responses are representative of those that would be given by all consumers, had each completed a questionnaire, to within +/-1.26% of the percentages reported. For example, if a satisfaction score of 50% is given for a particular question, we can be 95% confident that if we had actually gained a response from every staff member at the organisation, the score would lie between 48.74% and 51.26%. This makes the 2017 data a very robust dataset. The data presented in this report is also highly robust for the Energy and Communications sectors. The relatively low same base of consumers responding to Property cases means that greater caution is required when reviewing the feedback in relation to this sector. 1 Some overlap with December

6 Customer Satisfaction Monitoring 2017 Table 2: Survey returns and confidence intervals per sector Sector Sent out Returned Response rate Confidence interval (+/-) Energy % 1.64 Property % 9.04 Communications % 2.03 Total % 1.26 Survey responses accrue in real time to the bespoke online reporting dashboard. Please note that no data weighting has been applied to the data by variables such as sector or decision, so that the data in this report remains consistent with the data that is reported in real time on the dashboard. As shown by the table below, the distribution of responses per sector is in line with the volume of cases provided for this research, meaning in any case that data is not required at this level. Table 3: Distribution of survey responses per sector Sector Despatched % Returned % Energy 58% 59% Property 1% 2% Communications 41% 39% 1.3 The report The fact that this research produces a continuous data set for 2017 and that a number of key variables related to each case are incorporated into the survey responses, means that the data set is rich in potential insight. Decsion type ER MAS OSD Sector Energy Property Communications Touch points Portal use Website use 4

7 Introduction Wherever possible the analysis will identify variations by sector, by decision type and by outcome. The analysis will also examine how outcome satisfaction influences views on the wider customer journey. The data used in this report is rounded up or down to the nearest whole percentage point. It is for this reason that, on occasions, tables or charts may add up to 99% or 101%. Where tables and graphics do not match exactly to the text in the report this occurs due to the way in which figures are rounded up (or down) when responses are combined. Results that differ in this way should not have a variance that is any larger than 1%. Significance testing has been used to look for statistical differences in the responses between groups of consumers. In this case, the T-Test 2 has been used. Where there is said to be a statistically significant difference between two or more variables, this is based on a 95% confidence level. All differences noted in this report are statistically significant (unless stated). The following symbols will be found throughout this report: *% Denotes where the figure is less than 0.5% cf. X%* Denotes the term confer, which is to compare. This is used where two or more figures are compared against each other Denotes where slightly different question wording has been used 2 The T-Test assesses whether the means of two groups are statistically different from each other. 5

8 Customer Satisfaction Monitoring Sample profile The tables below summarise the demographic profile of the survey sample. Overall: Individuals aged (23%), (25%) and 66 and over (24%) are most commonly found within the sample. In contrast, just 1% of respondents are aged 19 to 25. More than seven in ten survey respondents (72%) are owner occupiers, with 12% renting privately. 94% of consumers state that English is their first language. 85% of consumers describe themselves as White British or Irish. Examining these demographic variables by sector shows that those aged 66 and over are significantly more likely to have had an Energy case (26%), rather than one relating to Property (17%) or Communications (21%). It is also notable that those aged were significantly more likely to have responded in relation to a Property case (27%). Variation by racial or ethnic background is marginal, although the proportion of consumers who are White English is significantly lower among Communications cases Table 4: Sample profile (All responses) Gender Total Energy Property Communications Male 57% 58% 64% 56% Female 43% 42% 36% 44% Age *% *% 0% 0% % 1% 0% 2% % 10% 27% 9% % 15% 17% 16% % 22% 21% 24% % 25% 14% 25% 66 and over 24% 26% 17% 21% Refused 2% 2% 4% 3% Tenure Owner occupier 72% 73% 87% 70% Tenant of a council 5% 4% 0% 5% Tenant of a housing association 5% 5% 0% 5% Tenant of a private landlord 12% 12% 6% 11% Other (type your response in the 3% 2% 4% 3% box) Refused 4% 3% 3% 6% 6

9 Sample profile Table 4 continued... Is English your first language? Total Energy Property Communications Yes 94% 95% 92% 93% No 6% 5% 8% 7% How would you describe your racial or ethnic background? English 66% 67% 74% 64% Welsh 4% 3% 1% 4% Scottish 10% 12% 1% 8% Irish 2% 1% 0% 2% Other British 3% 3% 2% 2% Any other white background 4% 4% 6% 5% Mixed background 1% 1% 2% 1% Indian 2% 2% 1% 2% Pakistani 1% *% 0% 1% Bangladeshi *% *% 0% *% Any other Asian background 1% 1% 1% 1% Caribbean 1% 1% 1% 1% African 2% 1% 2% 2% Any other black background *% *% 0% *% Chinese *% *% 2% *% Any other background *% *% 0% *% Refused 4% 4% 6% 4% 7

10 Customer Satisfaction Monitoring Key Findings 3.1 Overall Satisfaction Seven in ten (72%) consumers were satisfied with the overall handling of their case, including 56% who gave the most positive response on the scale of very satisfied. Only a small proportion (6%) gave a neutral response to this question, with 22% being dissatisfied. Overall satisfaction with case handling is consistent in the Energy and Communication sectors at 73% and 71% respectively. However, satisfaction is significantly lower for those who had a Property case at (53%). Among these consumers the proportion dissatisfied is nearly double the survey average (42% c.f. 22%). 3.2 Customer focus throughout the customer journey The survey design provides an indication of the perceived level of customer service and focus at each stage of the customer journey. Figure 1 overleaf shows the levels of agreement with specific service elements during initial contact, during the investigation and during the communication of the outcome. A clear and important message is that a majority of consumers give positive responses regarding their customer experiences at all stages of their customer journey. More specifically: Managing of expectations appears to be strong given that at initial contact 83% of consumers agree that the role of Ombudsman Services was clearly explained and that during the investigation the process was clearly explained (82% agree). Ombudsman Services are seen to understand consumer complaints, with 82% of consumers during initial contact stating that OS understood the nature of their complaint and that it understood what they wanted to achieve (80%). During the investigation the perceived level of understanding of the complaint drops marginally to 78%. Communication with consumers is largely seen as positive. During initial contact, 85% state that the person that consumers first had contact with was clear and easy to understand. At this stage 78% also say that the person they dealt with listened to them and cared about what they had to say. However, it should be noted that this proportion falls to 71% when the same question was asked at the investigation stage. Although only 56% of consumers agree that they were asked if they had any particular communication needs during their initial contact, 86% agreed that during their investigation that Ombudsman Services communicated with them in their preferred format of communication. Four in five (80%) also give positive views in relation to the way the outcome was communicated agreeing that the decision was clear and easy to understand. However, the agreement that decisions were fully reasoned and was supported by the available evidence (69%), were fair and unbiased (69%) and addressed the key elements of the complaint (69%) are notably below the higher scores recorded for earlier parts of the process. 8

11 Key Findings Figure 1: Customer Focus throughout the customer journey Initial Contact - % agree Clearly explained the role of Ombudsman Services - 83% Understood the nature of your complaint - 82% Understood the outcome that you wanted to achieve - 80% Was clear and easy to understand - 85% Listened to you and cared about what you had to say - 78% Asked you if you had any particular communication needs - 56% Explained what would happen next - 88% Index score per stage (average of agreement for all indicators) 79% The investigation - % agree Clearly explained the investigation process - 82% Understood the nature of your complaint - 78% Kept you informed and updated on progress (including following up on any promises they made) -78% Listened to you and cared about what you had to say - 71% Was clear and easy to understand - 82% Communicated with you in your preferred format of communication - 86% 79% The outcome - % agree Was clear and easy to understand 80% Was fully reasoned and was supported by the available evidence - 69% Was fair and unbiased - 69% Addressed the key elements of your complaint - 69% 72% 72% were satisfied overall with the handling of their case 9

12 Customer Satisfaction Monitoring 2017 The immediacy with which consumers receive the remedy that is recommended to them remains an issue given that 30% of consumers had to contact Ombudsman Services about the company in question not implementing the remedy that had been recommended by the investigation officer. This proportion is consistent by sector (Energy: 29%, Property: 31%, Communications: 32%). Among those who had to contact Ombudsman Services about remedy implementation a majority do provide positive feedback. However, the levels of agreement at this stage are lower than seen earlier in the process. Clarity of communication remains good, with 70% agreeing that the Remedy Implementation Team were clear and easy to understand and communicated in a preferred format (78%). However, only 62% agree that they were kept updated until the remedy was implemented, with 24% disagreeing this was the case. Perceived understanding and empathy from this team is weaker than earlier in the process with two thirds (66%) agreeing that the detail of their case was understood and that the team listened and cared to what the consumer had to say (62%). 3.3 Speed and the quality of customer service Throughout each stage of case handling, consumers were also asked to rate the speed/timeliness of responses and the overall quality of the customer service received. During initial contact, four in five (81%) gave a positive response to both aspects. However, ratings of both indicators drop to the mid 70s during the investigation. For those consumers who needed to re-contact Ombudsman Services about the company in question not implementing a remedy that had been recommended, the proportion who were satisfied with the speed and timeliness of responses drops to 64% and 63% of consumers at this point are satisfied with the overall quality of service received. Figure 2: Ratings of speed and customer service quality throughout the customer journey % Satisfied Initial contact Investigation Remedy implementaion Speed/ timeliness of their response 81% 77% 64% Quality of the customer service 81% 76% 63% 10

13 Key Findings 3.4 Signposting and awareness of Ombudsman Services All were asked to say how they first found out that Ombudsman Services might be able to help with their complaint. A quarter of respondents indicate that they were told about Ombudsman Services by the company they were complaining to (25%), with another 7% indicating that they found information on the company s website. A quarter of consumers said that that they went to the Ombudsman Services website directly (24%). This latter percentage highlights the need to ensure that the organisation s website continues to be an easy to navigate resource for consumers. 3.5 Website usage and feedback Half of consumers had visited the Ombudsman Services website before they first made contact (50%), a proportion that increases significantly to 77% amongst respondents from the Property sector. A third of respondents state they visited the Ombudsman Services website during their case (33%), while 28% of consumers had no interaction with the website at all throughout their customer journey. Among the 3,759 consumers within this sample who visited the Ombudsman Services website, more than nine in ten (92%) indicated that they were able to find the information that they were looking for. There is no significant variation by sector on this measure (Energy: 91%, Property: 88%, Communications: 92%), with these high proportions suggesting that the website content is well aligned with consumer s needs. 3.6 Portal usage and feedback As the portal is a key customer interface, feedback was requested from users on how easy it is to use. Among users who uploaded documents or downloaded documents for review, around three quarters described using the portal as easy (74% during initial contact and 78% during the investigation). One in nine (11%) gave a neutral response to this question, answering neither easy nor difficult When uploading documents 15% of portal users said that this was difficult, with 11% giving the same response in relation to downloading documents for review. Among those who suggested that uploading or reviewing documents via the portal was difficult, the most common explanation given was that the system was too confusing/complicated (55/58%). Around a quarter (26%/22%) also mentioned technical/compatibility issues with their computer. Example responses from the other explanations given mention a lack of smartphone compatibility, password issues along with further comments on clarity of the user experience. Frustrations with the portal are also communicated in some of the other open questions within the survey. 3.7 Delivering for customers When asked to state if the remedy they received was better or worse than expected, 67% suggest that it was either in line with expectations (43%) or indeed better than expected (24%). The fact that a quarter of consumers are achieving an outcome that exceeds their expectations does suggest that Ombudsman Services is making a difference on behalf of consumers. 11

14 Customer Satisfaction Monitoring 2017 In terms of the effort consumers have to expend using Ombudsman Services, the proportion of individuals who perceive this to be low (scores of 1 or 2 on a 5-point scale) is 46%. The proportion of consumers that perceived that Ombudsman Services made high effort on their behalf (scores of 4 or 5 on a 5-point scale) is considerably higher than the proportion who indicate that they themselves had to make high levels of effort in the process (55% cf. 33%). These perceptions, where the organisation is seen to deliver high effort relative to the input the consumer provides is an encouraging finding, given that the inevitable fact that Ombudsman Services will not always be able to deliver a desired outcome to the consumer. For Ombudsman Services users, there appears to be a strong level of service advocacy. When asked how likely they are to recommend the organisation to a friend or relative, the highest proportion of customers (62%) are classed as Promoters (giving a response of 9 or 10 on a 10 point scale). Indeed, more than half of the whole sample (57%) gave a 10 rating on the scale suggesting that they are extremely likely to recommend the organisation to others. Less than one in ten (9%) of consumers are Passive on the scale (7 or 8 rating out of 10), with the remaining 28% classed as Detractors. Subtracting the Detractors from the Promoters produces a positive Net Promoter Score (NPS) score of 34. This level of advocacy is equal for Energy (NPS of +36) and Communications (+33), but is lower among those how had a Property case. Indeed, for these individuals a negative NPS of -13 is found, with the proportion who are Detractors (53%), being greater than the proportion who are Promoters (40%). 12

15 Service awareness and website use 4 Service awareness and website use 4.1 Signposting and awareness of Ombudsman Services Before consumers were asked to provide feedback on their service experiences, all were asked to say how they first found out that Ombudsman Services might be able to help with their complaint. A quarter of respondents indicate that they were told about Ombudsman Services by the company they were complaining to (25%), with another 7% indicating that they found information on the company s website. A quarter of consumers said that that they went to the Ombudsman Services website directly (24%). This latter percentage highlights the need to ensure that the organisation s website continues to be an easy to navigate resource for consumers. Signposting from the company being complained to is the most commonly mentioned channel to Ombudsman Services in the Energy (26%) and Property (28%) sectors. Among those with a Communications complaint, equal proportions found out about Ombudsman Service via the company in question (22%) and the OS website (23%). Less than a fifth of respondents claim to have found out about Ombudsman Services via an internet search (17%), although this increases to 24% amongst respondents from the Property sector. Figure 3: How did you first find out that Ombudsman Services might be able to help you with your complaint? (All responses) I was told about Ombudsman Services by the company I was complaining to I went to the Ombudsman Services website directly 25% 24% An internet search 17% Word of mouth e.g. friend/relative 12% I found information on the website of the company I was complaining to From an advice service such as Citizens Advice 5% 7% Other 11% 0% 5% 10% 15% 20% 25% 30% Sample base: 5161 Respondents in the Property sector are less likely to have found out about Ombudsman Services via word of mouth; 4% compared to 12% across all areas. Word of mouth is a more dominant way of finding about Ombudsman Services among the year old age group. The 31% of this age group who said that they found out about Ombudsman Services in this way is significantly higher than in any other age 13

16 Customer Satisfaction Monitoring 2017 group. Indeed, this is the primary means by which this age group found about the service. With word of mouth being the fourth most important means overall of signposting to Ombudsman Services, levels of service advocacy will be examined later in the report using the Net Promoter Score. 4.2 Website usage Half of consumers had visited the Ombudsman Services website before they first made contact (50%), a proportion that increases significantly to 77% amongst respondents from the Property sector. A third of respondents state they visited the Ombudsman Services website during their case (33%), while 28% of consumers had no interaction with the website at all throughout their customer journey Figure 4: Before we ask you for feedback, can we ask if you visited the Ombudsman Services website, either before you first made contact or at anytime during your case? (All responses) 50% Yes beforehand: Energy: 50% Property: 77% Communications: 47% 33% 28% No: Energy: 27% Property: 11% Communications: 29% Yes during my case Yes beforehand No Sample base:

17 Service awareness and website use Non use of the website varies increases slightly with age (25% among those aged to 31% among those aged 66 and above). However, a majority of all age groups are having an interaction with the Ombudsman Services website as part of their customer experience. Looking over time, the proportion of respondents that visited the Ombudsman Services website before they first made contact rises from 44% in Quarter 1 up to 55% in Quarter 4. The opposite is the case for respondents who have not visited the website at all, which has fallen by 7% points between Quarter 1 and Quarter 4. Figure 5: Before we ask you for feedback, can we ask if you visited the Ombudsman Services website, either before you first made contact or at anytime during your case? (All responses) 70% 60% 50% 44% 47% 50% 55% 40% 30% 20% 32% 32% 35% 34% 28% 27% 30% 25% 10% 0% Q1 Q2 Q3 Q4 Yes beforehand Yes during my case No Sample base: Q1: 688, Q2: 1038, Q3: 1297, Q4:

18 Customer Satisfaction Monitoring Website usefulness Among the 3759 consumers within this sample who visited the Ombudsman Services website, more than nine in ten (92%) indicated that they were able to find the information that they were looking for. There is no significant variation by sector on this measure as shown by the figure below, with these high proportions suggesting that the website content is well aligned with consumer s needs. Figure 6: Did you find the information you were looking for on the website? (All website visitors) % yes Sample base: Energy: 2231, Property: 80, Communications: 1448 Among those who said that they could not find the information they were looking for on the website, the main themes in the responses related to a desire for information on the complaints process and a wish to track progress on individual cases. Mentions of portal issues are also found within the responses to this question. Service scope and process What areas of customer service were or were not covered in my complaint to SSE regarding overcharging for electricity direct debits. Step-by-step procedure regarding the complaint and time frame of procedure. Clarity on the potential outcomes of the cases that are reviewed. How the information was to be gathered how I could look at what was being looked at by the services to see if this was relevant to my issues. I could not find the various stages of complaint before the ombudsman can be approached and timescales. The limits of the powers of the OS. The process and some specifics. 16

19 Service awareness and website use Basic information about the process and who to complain to if I was unhappy with the Ombudsman's response. 2I was looking for anonymised decisions reached by Ombudsman Services. For example the Local Government Ombudsman publishes the outcome of every case it investigates, the entire decision is published, but any references to names are removed. I was looking on the Ombudsman Services website for a similar feature. Details about the consumers own case. More detailed info on the progress of my case. I couldn't find my case when I tried to track it. Portal issues/support I couldn't upload anything relating to my case, and I couldn't see any update information on my case. There was no upfront written explanation of how the process would work. All I could do was make a login for myself then see a page with basically nothing on it. I had problems accessing the portal. How to send information and uploads. The website was not always stable, and ultimately locked me out entirely even though all my details (login etc.) had not change. 17

20 Customer Satisfaction Monitoring Initial contact with Ombudsman Services In general, respondents are very positive about their first contact with Ombudsman Services, with around eight in ten agreeing to all but one of the statements about the first stage of their customer journey. Managing of expectations appears to be strong given that at initial contact 83% of consumers agree that the role of Ombudsman Services was clearly explained. Ombudsman Services are seen to understand consumer complaints, with 82% of consumers during initial contact stating that OS understood the nature of their complaint and that it understood what they wanted to achieve (80%). Fewer than six in ten respondents agree that the person they spoke to asked if they had any particular communication needs (56%). However, this is not necessarily due to a high proportion that disagree (11%), but due to more respondents that say they can t recall/ don t remember (18%). This figure increases to 27% amongst respondents from the Property sector. Figure 7: Thinking specifically about the person that you spoke to when you first contacted Ombudsman Services about your complaint, do you agree or disagree that the person? (All responses) Explained what would happen next 88% 6% 4% 2% Was clear and easy to understand 85% 8% 5% 2% Clearly explained the role of Ombudsman Services 83% 9% 4% 4% Understood the nature of your complaint 82% 8% 8% 2% Understood the outcome that you wanted to achieve 80% 9% 9% 2% Listened to you and cared about what you had to say 78% 11% 9% 2% Asked you if you had any particular communication needs 56% 15% 11% 18% Sample base: % 20% 40% 60% 80% 100% Agree Neither agree nor disagree Disagree Can't recall/ don't remember 18

21 Initial contact with Ombudsman Services Further analysis into perceptions of the person they spoke to asking if they had any particular communication needs shows that agreement levels peak at 59% in Quarter 4, compared to just 54% in both Quarter 1 and 2. Figure 8: Thinking specifically about the person that you spoke to when you first contacted Ombudsman Services about your complaint, do you agree or disagree that the person...? (Valid responses) Asked you if you had any particular communication needs 70% 60% 50% 54% 54% 55% 59% Agree Neither 40% Disagree 30% 20% 10% 0% 19% 17% 18% 18% 15% 16% 15% 13% 11% 13% 11% 9% Q1 Q2 Q3 Q4 Can't recall/ don't remember Sample base: Q1: 1619, Q2: 1038, Q3: 1297, Q4: 1207 For each stage of the customer journey, a summary index score has been calculated using the questions included to explore the specifics of the customer experience. In the initial contact stage this index score has been calculated using the seven statements shown above in Figure 7. In this calculation, the average agreement across all seven statements has been calculated. The resulting index score of the customer experience at initial contact is 79%. This summary measure is consistent in each quarter. There is also consistency in this summary measure between those who had an Energy Case and those who had a Communications case (both 79%). However, this index score of customer experience at initial contact for those with a Property case is significantly lower at 60%. 19

22 Customer Satisfaction Monitoring 2017 Overall, four fifths of respondents were satisfied with the speed/timeliness (81%) and the quality of the customer service (81%) during their initial contact with Ombudsman Services. The level of satisfaction falls to just 62% in terms of speed/ timeliness and 63% in terms of the quality of the customer service amongst respondents who had a Property case. Further investigation shows that satisfaction levels differ by the case decision even at this early stage of the process. Respondents who had an early resolution (ER) are more likely to be satisfied with the speed/ timeliness (95%) and the quality of the customer service (93%). In comparison, among those who had an Ombudsman Services Decision (OSD), just short of seven in ten were satisfied with the speed/ timeliness (69%) and the quality of the customer service (67%) at initial contact. Figure 9: Overall, thinking about your initial contact with Ombudsman Services how satisfied were you with... (All responses) The speed/timeliness of their response 81% 7% 11% *% The quality of the customer service they provided 81% 7% 11% *% Sample base: % 20% 40% 60% 80% 100% Satisfied Neither Dissatisfied Can't recall/ don't remember 20

23 Initial contact with Ombudsman Services The figure below highlights the change over time in regards to respondent s satisfaction with the speed timeliness and the quality of customer service of their initial contact with Ombudsman Services. It is clear that the levels of satisfaction have significantly improved between Quarter 1 and Quarter 4 (+8% points and +6% points respectively). Additionally, dissatisfaction has fallen between the two quarters (-5% points and -3% points respectively). Figure 10: Overall, thinking about your initial contact with Ombudsman Services how satisfied were you with... (All responses) 100% 95% 90% 85% 80% 75% 79% 78% 81% 81% 80% 81% 86% 85% Speed/ timeliness Quality of the customer service 70% 65% 60% Q1 Q2 Q3 Q4 Sample base: Q1: 1619, Q2: 1038, Q3: 1297, Q4:

24 Customer Satisfaction Monitoring The Investigation In the next set of questions respondents were asked to provide feedback on the dealings they had with their investigation officer. Over seven in ten respondents answered in a positive manner about their investigation officer. It is interesting to note that when respondents were initially contacted, only 56% agree they were asked if they had any particular communication needs, however, over eight in respondents agree that their investigation officer communicated with them in their preferred format of communication (86%). Investigating officers appear to providing clarity of communications given that 82% of consumers agree that their officer clearly explained the investigation process and that the officer was clear and easy to understand (also 82%). In terms of keeping the consumer updated on process, 78% of consumers agree that their officer did this. This included the officer following up on any promises that they made. The lowest agreement in this bank of statements is found in regards to the investigation officer listening and caring about what the consumer had to say. While seven in ten (71%) consumers agree that this was the case 17% actively disagree their investigation offer listened to them and cared about what they had to say. Disagreement on this indicator is particular high amongst respondents that had subsequently reached an OSD (32%). Figure 11: Based on all the dealings you had with your investigation officer do you agree or disagree that the person? (All responses) Communicated with you in your preferred format of communication 86% 7% 6% 1% Clearly explained the investigation process 82% 8% 9% 1% Was clear and easy to understand 82% 9% 9% 1% Understood the nature of your complaint 78% 7% 14% *% Kept you informed and updated on progress (including following up on any promises they made) 78% 9% 13% *% Listened to you and cared about what you had to say 71% 11% 17% 1% Sample base: % 20% 40% 60% 80% 100% Agree Neither agree nor disagree Disagree Can't recall/ don't remember 22

25 The Investigation Looking at the perceptions of the investigation officer by quarter throughout 2017, all agreement levels significantly increase between Quarter 3 and Quarter 4. In particular, there is a 6-percentage point increase in agreement that the investigation officer listens and cares about what respondents had to say. Table 5: Based on all the dealings you had with your investigation officer do you agree or disagree that the person? (All responses) % Agree Communicated with you in your preferred format of communication Clearly explained the investigation process Q1 Q2 Q3 Q4 Diff Q3 vs. Q4 86% 84% 86% 89% +3% 81% 82% 81% 86% +5% Was clear and easy to understand 81% 81% 80% 85% +5% Understood the nature of your complaint 77% 78% 77% 82% +5% Kept you informed and updated on progress (including following up on any promises they made) Listened to you and cared about what you had to say 75% 75% 78% 83% +5% 71% 69% 69% 75% +6% The index score for the customer experience during the investigation (based on the 6 questions shown in Figure 11) is 79%. This is the same average satisfaction level as seen in the initial contact stage (also 79%). Although the two index scores are calculated using different question statements, this does in broad terms suggest that customer satisfaction levels during the initial contact are being sustained as the customer journey progresses. Across 2017 Quarter 4 shows a significantly higher index score for the investigation stage (83%) relative to the preceding three quarters. Analysis by sector again shows a consistency in the index score of customer experience between Energy (80%) and Communications (79%) cases, with this score lower for Property (64%). 23

26 Customer Satisfaction Monitoring 2017 Overall, nearly four fifths of respondents were satisfied with the speed/ timeliness (77%) and the quality of the customer service (76%) provided by the investigation officer during their investigation. Almost one fifth of respondents were dissatisfied with the latter (18%). Furthermore, this dissatisfaction rises to 31% amongst respondents from the Property sector and 35% amongst respondents who case required an OSD. Figure 12: Throughout the investigation of your complaint how satisfied were you with the investigation officer in terms of? (All responses) The speed/timeliness of their response 77% 8% 15% The quality of the customer service they provided 76% 6% 18% Sample base: % 20% 40% 60% 80% 100% Satisfied Neither Dissatisfied Can't recall/ don't remember Again, when reviewing responses overtime it is clear that the levels of satisfaction have significantly improved between Quarter 1 and Quarter 4 (+10% points and +4% points respectively). Additionally, dissatisfaction has fallen between the two quarters (- 8% points and -3% points respectively). Figure 13: Throughout the investigation of your complaint how satisfied were you with the investigation officer in terms of? (All responses) 100% 95% 90% 85% 83% Speed/ timeliness 80% 75% 70% 75% 73% 76% 74% 78% 75% 79% Quality of the customer service 65% 60% Q1 Q2 Q3 Q4 Sample base: Q1: 1619, Q2: 1038, Q3: 1297, Q4:

27 The Outcome 7 The Outcome When respondents were asked to think about the decision that was made about their complaint following the investigation, four fifths (80%) agree that it was clear and easy to understand. This perceived clarity is significantly higher amongst respondents who had an Early Resolution (93%) or reached a Mutually Accepted Settlement (87%). Among those whose case resulted in an OSD, the proportion who said that the final decision was clear and easy to understand drops significantly to 65%. Among all consumers, seven in ten agree that the decision was fully reasoned and supported by the available evidence (69%), fair and unbiased (69%) and addressed the key elements of the complaint (69%). However, nearly a quarter actively disagree across these three measures. Figure 14: To what extent do you agree that the decision? (All responses) Was clear and easy to understand 80% 7% 13% *% Was fully reasoned and was supported by the available evidence 69% 8% 23% *% Was fair and unbiased 69% 9% 22% *% Addressed the key elements of your complaint 69% 7% 24% *% 0% 20% 40% 60% 80% 100% Agree Neither agree nor disagree Disagree Can't recall/ don't remember Sample base: 5161 * denotes less than 0.5% 25

28 Customer Satisfaction Monitoring 2017 When looking at the perceptions of the decision by sector, it is clear that respondents who had an Energy case are much more positive, with agreement significantly above the survey average for all elements. Table 6: To what extent do you agree that the decision? (All responses) % Agree Total Energy Property Communicati ons Was clear and easy to understand 80% 81% 62% 80% Was fully reasoned and was supported by the available evidence 69% 70% 42% 68% Was fair and unbiased 69% 71% 44% 67% Addressed the key elements of your complaint 69% 71% 42% 67% Base Combining the average satisfaction for these four statements gives an index score of 72%. This score is significantly higher among those with an Energy case (74%) compared to both those with a Communications case (71%) and those with a Property case (48%). In interpreting these findings, it should be noted that the Energy and Communications cases for which feedback was provided, there is an equal split between Early Resolution/Mutually Accepted Settlement and Ombudsman Services Decisions. Among Energy cases this split is 69%/31% while among Communications case it is 67%/33%. By comparison, all Property cases involved an OSD. Importantly, there is no notable difference in the distribution of the outcomes achieved between the Energy and Communications sectors. Those whose case was either settled or upheld are most likely to give positive responses in relation to the specific elements of their decision. The proportion of those achieving each of these outcomes who agree with the content of the decision are below: Decision was fully reasoned and was supported by the available evidence (Settled: 87%, Upheld: 72%); Decision was fair and unbiased (Settled: 91%, Upheld: 72%); Decision addressed the key elements of your complaint (Settled: 89%, Upheld: 72%); 26

29 The Outcome Among all consumers participating in this research, two thirds (67%) were satisfied with the remedy suggested for their complaint. This includes 47% who were very satisfied, the highest proportion on the five point scale. Very few respondents (5%) express a neutral view on the remedy achieved, while 28% were dissatisfied with what Ombudsman Services recommended. Figure 15: And overall, how satisfied were you with the remedy to your complaint? (All responses) Very satisfied: 47% Fairly satisfied: 20% Very Dissatisfied: 19% Fairly Dissatisfied: 9% Sample base: 5161 Analysis by sector show that remedy satisfaction for Energy complaints at 69% is significantly higher than the satisfaction seen for Communication (65%) and Property (46%) cases Figure 16: And overall, how satisfied were you with the remedy to your complaint? (All responses) % Satisfied Sample base: Energy: 3060, Property: 90, Communications:

30 Customer Satisfaction Monitoring 2017 There is also significant variation in remedy satisfaction by decision type. For those who had an Early Resolution, 90% were satisfied with their remedy, while for those who had a Mutually Accepted Settlement 76% were satisfied with their remedy. Both these proportions are significantly higher than the 44% of consumers who had an OSD who are satisfied with the remedy they received for their complaint. Examining this further, as might be anticipated, where a case was Maintained or Not Upheld remedy satisfaction is significantly lower (45% and 33% respectively), than among those whose case was Settled (90%) or Upheld (70%). 7.1 Delivering against expectations When asked to state if the remedy they received was better or worse than expected, 67% suggest that it was either in line with expectations (43%) or indeed better than expected (24%). As already noted above, managing of expectations appears to be strong given that at initial contact 83% of consumer agree that the role of Ombudsman Services was clearly explained and that during the investigation the process was clearly explained (82% agree). Even so, the fact that a quarter (24%) of consumers are achieving an outcome that exceeds their expectations does suggest that Ombudsman Services is making a difference on behalf of consumers. Figure 17: Would you say the remedy you received was? (All responses) 43% Worse than you expected: Energy: 32% Property: 53% Communications: 35% 24% 33% Sample base: 5161 Better than you expected About what you expected Worse than you expected At a sector level the proportion of consumers who state that the remedy they received was worse than they expected is lowest for Energy cases at 32%. This proportion is significantly lower than the 35% of Communications consumers who felt their remedy was worse than expected and the 53% of Property consumers who gave the same response. 28

31 The Outcome Among those who indicated that that the remedy achieved was better than expected 99% express satisfaction with their case remedy overall. For those whose remedy was in line with their expectations, 90% were satisfied with their remedy. However, where the remedy achieved was worse than expected only 14% express overall satisfaction with their remedy, with a clear majority (77%) dissatisfied. Comments given regarding outcomes being worse than expected tended to be very specific in nature, referencing the details of their case. In interpreting these responses it can be seen that for some consumers there is a perceived mismatch in their eyes of the evidence presented and the outcome delivered. Criticisms are also made about the depth of the investigations. Others appear to suggest the remedies achieved are not commensurate with the time and effort involved to produce them. A small selection of such comments are provided below by means of illustration. I totally believed Ombudsman Services would sort out what to me was a blatantly unfair additional charge of 250 imposed by Cooperative Energy on top of my regular direct debits. The outcome for me was zero compensation as, in their supposed wisdom, Ombudsman Services considered Cooperative Energy had done nothing wrong, and so I wasted a serious amount of time and energy pursuing this for absolutely nothing. Failed to understand my complaint - failed to provide relevant information to support the amazing conclusion that SSE had not done anything wrong - quoted the incorrect company in s and at this point I have still not received the financial settlement detailed. Many of the salient facts in the case were not disputed by the company. Because the Ombudsman refused to listen to me that the resolution was not appropriate in my case even when the call was escalated to a higher level because they did not understand the warm home energy discount scheme, specifically the broader group and seemed to have not interest at taking me at my word. None of the impartial evidence we provided was deemed valid. It was provided by a party not involved with the dispute, one of the key points in our case was about access to the property, we got a statement from a bank which said that access was always available as this wasn't he banks fire escape. The ombudsman chose not to accept this statement and instead relied solely on the energy companies statement that access wasn't available. It clearly wasn't an unbiased decision. I felt that the investigator just repeated what they had been told without actually looking into the reasons behind the complaint. Nor did they explain why they found what they did. The damages offered were ludicrously small for the trouble I had been put to. 100 compensation for all my time and effort is unreasonable. 29

32 Customer Satisfaction Monitoring Remedy implementation The immediacy with which consumers receive the remedy that is recommended to them remains an issue given that 30% of consumers had to contact Ombudsman Services about the company in question not implementing the remedy that had been recommended by the investigation officer. This proportion is consistent by sector (Energy: 29%, Property: 31%, Communications: 32%). It is also notable that the proportion of customers having to have this re-contact with Ombudsman Services is significantly higher among those who had an OSD (33%) compared to where a MAS was reached (28%). Figure 18: Did you contact Ombudsman Services about the company in question not implementing the remedy that had been recommended by the investigation officer? (All responses) Yes: 70% Energy: 29% Property: 31% Communications: 32% 30% Early resolution: 31% MAS: 28% OSD: 33% Yes No Sample base: 5161 Among those who had to contact Ombudsman Services about remedy implementation a majority provide positive feedback. However, the levels of agreement are lower than earlier in the process. Clarity of communication remains good with 70% agreeing that the Remedy Implementation team were clear and easy to understand and communicated in a preferred format (78%). However, only 62% agree that they were kept updated until the remedy was implemented, with 24% disagreeing this was the case. Perceived understanding and empathy from this team is weaker than earlier in the customer journey, with two thirds (66%) agreeing that the detail of their case was understood and that the team listened and cared to what the consumer had to say (62%). 30

33 Remedy implementation Figure 19: Would you say that in response to this particular stage of your case the staff at Ombudsman Service have... (All responses - where contacted Ombudsman Services about remedy not being implemented) Communicated with you in your preferred format of communication 78% 9% 12% 1% Were clear and easy to understand 70% 13% 16% 1% Clearly explained the action they could take 67% 11% 21% 1% Understood the detail of your case 66% 10% 24% 1% Listened to you and cared about what you had to say 62% 12% 24% 1% Kept you updated until the remedy was implemented 62% 13% 24% 1% Sample base: % 20% 40% 60% 80% 100% Agree Neither agree nor disagree Disagree Can't recall/ don't remember For those consumers who needed to re-contact Ombudsman Services about the company in question not implementing a remedy that had been recommended the proportion who were satisfied with the speed and timeliness of responses drops to 64% and 63% of consumers at this point were satisfied with the quality of service received. As shown by the figure below, one quarter of consumers were dissatisfied with speed and customer service at this late stage in the customer journey. Figure 20: How satisfied were you with the Remedy Implementation Team in terms of? (All responses - where contacted Ombudsman Services about remedy not being implemented) The speed/timeliness of their response 64% 12% 23% 1% The quality of the customer service they provided 63% 10% 26% 1% Sample base: % 20% 40% 60% 80% 100% Satisfied Neither Dissatisfied Can't recall/ don't remember 31

34 Customer Satisfaction Monitoring 2017 The suggestions for improving service from the Remedy Implementation Team are summarised below. Premature closure of cases, an inability to enforce remedies, plus service weakness in terms of accuracy, and communication can all be found among those dissatisfied with the Remedy Implementation Team. Premature case closure The case was closed on the basis that the company had told the Ombudsman that it had actioned all the remedies as agreed, no contact was made with me to confirm that this was the case and I had to contact them several times to get the case re-instated and the remedies completed. They closed my case prematurely without checking with me that the remedy had been implemented by Vodafone. I believe this was because they hadn't properly checked the documentation they'd received from Vodafone which was full of errors. The problem still isn't solved and yet the Remedy Implementation Team inform me the case is closed. I am still being pursued by the utility provider for over 1,000 which I don't owe. Enforcement powers What is the point of having your service if you can't enforce your decisions? It would appear that the Ombudsman service is toothless. They could have insisted that the company concerned completed the remedies, I called the company and they told me right from the start that they would not implement them because they did not agree with them! British Gas have not done anything that the ombudsman said they should do. Could have addressed the fact that their original recommendations were not implemented. Recommendations were ambiguous and caused problems. Well, as I expected, npower have completely ignored the resolution. Communication and support The remedy implementation team have not got back to me. They did not respond to my /letter. They kept calling me when I was at work instead of and didn't really listen to me. I was not kept informed of anything that was going on. They could have made me feel more supported in terms of ensuring implementation of the remedy. No elements of the remedy have been implemented and at this stage I have no real confidence that they will be. 32

35 Remedy implementation Accuracy Failed to understand complaint, sent wrong information to energy company Read the facts of the case thoroughly and act on behalf of the customer. Took the wrong action Sent back to the investigation officer - then took too long when he sent thro revised statement. The staff give out wrong and conflicting information. They do not return phone calls or give written explanation of the process. This has delayed the legal action I must now take to enforce the award. 33

36 Customer Satisfaction Monitoring Portal Usage and Feedback At two stages of the survey (initial contact and investigation) consumers were asked whether they used the organisation s portal system. At the initial contact stage just over half of consumers (53%) used the portal to upload documents about their case. At the investigation stage 46% used this platform to review documents sent to them. Figure 21: As part of the process of explaining your issue did you use the Ombudsman Services portal system to upload documents/to review documents sent to you? (All responses) 47% 54% 53% 46% No Yes No Yes Sample base: Initial contact: 5071, Investigation: 5161 As the portal is a key customer interface, feedback was requested on how easy it is to use. Among users at both stages around three quarters described using the portal as easy (74% during initial contact and 78% during the investigation). One in nine (11%) gave a neutral response to this question, answering neither easy nor difficult When uploading documents 15% of portal users said this was difficult, with 11% giving the same response in relation to downloading documents for review. 34

37 Portal Usage and Feedback Figure 22: How easy was it to upload documents/review documents sent to you? (All responses where used portal system) Very easy Fairly easy Neither easy nor difficult Fairly difficult Very difficult 11% 11% 11% 7% 5% 4% 37% 44% 37% 34% Initial contact Investigation Summary: Easy 74% 78% Summary: Difficult 15% 11% 0% 20% 40% 60% 80% 100% Sample base: Initial contact: 2670, Investigation: 2368 There is no significant variation by age in those stating uploading or reviewing documents was difficult both at the initial contact stage/ or during the investigation. Among those who suggested that uploading or reviewing documents via the portal was difficult the most common explanation was that the system was too confusing/complicated (55/58%). Around a quarter (26%/22%) also mentioned technical/compatibility issues with their computer. Example responses from the other explanation given mention a lack of smartphone compatibility, password issues, plus further comments on clarity of the user experience. Figure 23: Why was it difficult to upload/review documents? (All responses those who said it was difficult to upload/review documents) System was too confusing/complicated Technical/compatibility issues with my computer Took too long Other 26% 22% 20% 15% 33% 32% 55% 58% Initial contact Investigation 0% 20% 40% 60% 80% Sample base: Initial contact: 405, Investigation:

38 Customer Satisfaction Monitoring 2017 An illustrative selection of the comments made in relation to the portal can be found below: Unclear whether documents had uploaded or not. A problem caused me to upload files a few times without realising at first. There was no obvious link showing where to upload documents. It continually locked me out and I was unable to log back in. I am fairly IT literate so I would think this would be even worse for others who aren't. Unclear progress. Dialogue boxes confusing. Not user friendly - antiquated website. I couldn't find the documents and there is no obvious instructions as to how to find them. The portal is not at all intuitive. I wasn't actually told about the portal to start with I didn't receive the , then I didn't receive a password to access it. Not smart phone compatible. Not smart phone friendly. Very difficult to use on a mobile device. Do not have a p.c. only my Android Smart phone. Had to search all over to find what had transpired over time. Found this quite stressful due to my age. 36

39 Overall satisfaction measures 10 Overall satisfaction measures 10.1 Case handling Having given their feedback on the key stages of their case, consumers were asked how satisfied they were overall with the handling of their case. In response, seven in ten (72%) were satisfied with overall case handling, including 56% who gave the most positive response on the scale of very satisfied. Only a small proportion (6%) gave a neutral response to this question, with 22% being dissatisfied. Figure 24: Irrespective of the final decision that was made for your complaint, overall, how satisfied were you with Ombudsman Services' handling of your case? (All responses) Very satisfied: 56% Fairly satisfied: 16% Sample base: 5161 Overall satisfaction with case handling is consistent in the Energy and Communication sectors at 73% and 71% respectively. However, satisfaction is significantly lower for those who had a Property case at (53%). Among these consumers, the proportion dissatisfied is nearly double the survey average (42% c.f. 22%). 37

40 Customer Satisfaction Monitoring 2017 Figure 25: Irrespective of the final decision that was made for your complaint, overall, how satisfied were you with Ombudsman Services' handling of your case? (All responses) % Satisfied Sample base: Energy: 3060, Property: 90, Communications: 2011 The level of satisfaction with case handling is significantly higher among those who received an Early Resolution (92%) and a Mutually Accepted Settlement (81%). In comparison only 51% of those whose case involved an Ombudsman Service Decision were satisfied with their case handling. Although the question wording asked the consumer to discount the outcome of their case in response to this question, the data does suggest that outcome has an influence. The highest satisfaction with case management is among those whose complaint was Settled (92%). The next highest satisfaction is seen among those whose case was Upheld (75%), with satisfaction significantly lower for those where the outcome was Maintained (54%) or Not Upheld (42%). Furthermore, where individuals express dissatisfaction with the remedy achieved only 15% are satisfied with how their case was handled overall. Among those satisfied with the remedy achieved the proportion satisfied is 82-percentage points higher at 97%. 38

41 Overall satisfaction measures Examples of the positivity around case handling are shown by the comments below. In these comments mentions are made of timelines and the ability to sort our long standing problems, understanding and strong communication. The process was simple and achieved in days what I had failed to achieve in months. They understood the problem, acted very promptly, when the same problem reoccurred they resolved it immediately. I had been struggling with the energy co for over a year, the Ombudsman resolved it in days. I was very impressed. The service was quicker than I thought and they got a result in weeks after I had been fighting it for a year with no luck. The Ombudsman acted promptly and professionally, addressed my concerns and communicated clearly. From the start of our telephone conversation detailing my issues the officer was attentive and responded to my comments. Very good. Understood complaint and listened to me. They forced the energy supplier to take action which previously they were very sluggish to provide. Agent was friendly and professional and achieved a positive outcome from a tricky case. Resolved in a timely manner. Having tried to sort out the problem myself to no avail it was brilliant that they sorted it out so quickly. I was very satisfied as they listened to what I said as the company I complained about did not listen to me. Achieved the outcome I wanted in a timely manner. Very nice people, thorough and professional. Easy to deal with and very quick. The professional way I was kept informed. Kept up to date with s throughout. The process was so easy! I was missing a document that was relevant to the case but it wasn't a problem. I gave a full description of the reason for my complaint and was never asked to repeat it. It was confirmed back to me in a very clear and easy format. I cannot think of a single thing that I was not impressed with or completely satisfied. Although I did not get the resolution I wanted I understood why from the explanation given and it was clear that the Ombudsman had found a reasonable and fair solution on my behalf. I almost wish I had more to complain about!! Thank you - top class service!! From the moment I discussed my complaint with your representative. He explained to me exactly what the procedure to be adopted. It s my first time I have used the services of an Ombudsman. I was very confident with what was explained to me and in the end, I am very satisfied how everything progressed and very satisfied with the end result. Well done 39

42 Customer Satisfaction Monitoring 2017 Among those dissatisfied with case handling overall a number of themes emerge which are show below along with illustrative comments. Overall dissatisfaction witch achieved outcome He was all in the favour of the people I was making the complaint about. It was not the outcome I wanted. Because you did nothing other than get a apology and compensation which was already offered to me (nothing more than I had got myself!) completely a waste of 2-3 months and doc managing. I expected a better outcome instead I felt like I was told this is it take it or leave it. Poor quality of adjudication by Ombudsman. Lack of legal knowledge. The investigating officer completely ignored my evidence and therefore came to a wrong decision. They didn't listen to me and my issues they backed British gas in lowering my compensation from 150 to just 75. The initial contact gave the impression I had a good case. The final outcome did not support this. I just wasted my time and achieved the same outcome I already had. Quality of investigation/interpretation I don t feel my case was properly understood due to the amount of documentation I had to send and the complex nature. The outcome I received wasn't worth the time and effort I put into collating all the evidence etc. Lack of care, effort and diligence by the case officer. It felt like he couldn't be bothered to understand the complexities of my claim and analyse the large amount of evidence. They did not listen to me. Information is inaccurate Information is inconsistent Officers are unable to provide the information asked for. The investigating officer did not understand the detail of the complaint including my disability. 1. Not independent. 2 Did not investigate fully 3. Process followed by investigator left me disadvantage. Mainly because all three people I spoke to, didn't listen to what I was saying and try to understand my issues. My evidence was ignored. It was not looked into fairly, the company change things and never told me I could put in details to show the company lied. As before - they fundamentally misunderstood the nature of my complaint. 40

43 Overall satisfaction measures Sky failed to provide the promised transcript of the relevant conversation, which would have proved that I was telling the truth. The Ombudsman disregarded this. As my case hinged on the conversation, I feel that it was extremely unfair to only use Sky s notes and completely disregard my notes. Unfortunately, no pressure was put on Sky to provide the transcript. Totally unfair! I don t think they understood the issues properly and seemed to lack empathy. Again because the wrong decision was made and that s because no one really asked the correct questions. The case handler didn t use all the information at hand to make her decision and I had to on 2 occasions to look over the s provided which lead to her changing her mind. This meant I had lost faith in the service. Service felt unprofessional, involved little empathy and needs bringing rapidly into the 21st century! Perceived bias/ lack of impartiality The process was very slow and difficult and weighted in favour of the company I was complaining about. Because they always back the large companies that are able to walk over the working man. And let's face it the service is funded by these companies. I don't believe on the basis of my experience that the energy services ombudsman is genuinely impartial. I feel there was too much bias to a very large company who are able to produce better evidence due to their systems than me, all I had to rely on was my account and this was not taken properly. As far as I am aware no recordings of any of the conversation that I had were ever listened to just lots of weight put on what was written by the person/persons I was complaining about. I found that my ''investigation officer'' was biased in favour of the energy companies involved. Speed Slow had to chase took ages for a poor outcome! I feel you support bigger companies. The whole process took too long and I had to call the ombudsmen service to find out what was happening and also check the website portal. Technical/portal issues Wasn't able to use portal system to view or send documents, Because I could never log in. reset password, password never sent. poor back office. 41

44 Customer Satisfaction Monitoring 2017 Very poor website layout and lacks information. Very poorly constructed web tools (portal). Lack of communication to make me aware that it was being looked at rather than waiting to be looked at. The portal is awful Outcome was pretty alright but I couldn't review any of the documents or respond because your website's awful on mobile. It felt like I was being hounded by faceless texts and s, but it wasn't clear how I could respond (I was on holiday at the time and only had my phone). Remedy implementation You have a single purpose - to provide remedies to customers that have been let down by customer service functions. Once a remedy has been agreed you need a process which ensures compliance. This appears to be missing which means that you are not fit for purpose. The Ombudsman appears to have little power to deal with these companies effectively - adding to customer frustration. Remedy has not been fully implemented more than 6 months after our complaint with supplier was initially raised. The company has still not actioned any of the Ombudsman s resolution points. 42

45 Overall satisfaction measures 10.2 Effort perceptions As well as recording satisfaction, questions were also included to provide an indication of how much effort consumers judge is required by them when using Ombudsman Services and how that compares to the effort that Ombudsman Services expends. The effort scale used originally was a 0-10 scale, but this was adapted from wave 8 onwards to a 1 to 5 scale and it is data using the latter scale that is reported upon below. The key message from the figure below is that the proportion of individuals who perceived high levels of effort were made by Ombudsman Services (scores of 4 or 5 on the scale) is considerably higher than the proportion who indicate they themselves had to make high levels of effort in the process (55% cf. 33%). In total, 46% of consumers indicate that their dealings with Ombudsman Services required low levels of effort (scores of 1 or 2 on the scale). Figure 26: Overall on a scale of 1-5, how much effort did YOU have to go to with Ombudsman Services/ do you perceive Ombudsman Services went to resolve your complaint? (All responses) 100% 90% 80% 70% 20% 13% 30% 5 - Much effort 60% 50% 40% 30% 21% 19% 25% 21% % 10% 0% 26% You 9% 14% Ombudsman 1 Little effort Sample base: 2054 (Note only based on waves 8-12 due to question wording change Analysis by sector shows that 48% of those with an Energy complaint and 44% of those with a Communications case felt that little effort was required on their part. However, this proportion drop significantly to just 12% among consumers with a Property case. As might be anticipated, in term of the consumer s own efforts this is most commonly described as low (scores of 1 or 2) among those who had an Early Resolution (65%). Among those who case resulted in a Mutually Accepted Settlement 53% felt low effort was required from them. However, where a case required an OSD the balance of opinion reversed with 29% indicating low effort was required from them and 52% suggesting that high levels of effort (scores of 4 and 5) were required. 43

46 Customer Satisfaction Monitoring 2017 Figure 27: Effort perceptions for ER cases 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 6% 6% 23% 21% 43% You 32% 29% 25% 5% 9% Ombudsman 5 - Much effort Little effort Sample base: 205 Figure 28: Effort perceptions for MAS cases 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 13% 11% 23% 23% 30% You 34% 28% 19% 8% 10% Ombudsman 5 - Much effort Little effort Sample base: 1111 Figure 29: Effort perceptions for OSD cases 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 34% 18% 19% 13% 16% You 23% 20% 23% 12% 22% Ombudsman 5 - Much effort Little effort Sample base:

47 Overall satisfaction measures In terms of the effort that Ombudsman Services is perceived to expend, this is highest for Early Resolution Cases (60%) and those whose case result in MAS (63%). For those individuals who had an Ombudsman Service Decision the proportion who feel that a high effort was made by Ombudsman Services drops to 43%. This drop in perceived effort is notable given that to reach an OSD requires more involvement and resources from Ombudsman Services Net promoter score To further extend the analysis beyond satisfaction, a question was also included to assess the level of customer advocacy among service users. This question asked individuals how likely they would be to recommend Ombudsman Services to family or friends if they had a similar issue to their own. Responses were sought on a scale of 0 to 10, where 0 is not at all likely and 10 is extremely likely. This question structure allows the calculation of the Net Promoter Score (NPS). In this calculation individuals are classed as a Detractor if they give a score of 0-6 on the scale. Those who give a score of 7-8 are classed as Passive while those who respond with a score of 9 or 10 are classed as Promoters. The Net Promoter Score is calculated by subtracting the number of Detractors from the number of Promoters. For Ombudsman Services users, there appears to be a strong level of service advocacy. As shown by the figure overleaf the highest proportion of customers (62%) are classed as Promoters. Indeed, more than half of the whole sample (57%) gave a 10 rating on the scale suggesting that they are extremely likely to recommend the organisation to others. Less than one in ten (9%) of consumers are Passive on the NPS scale with the remaining 28% classed as Detractors. This distribution produces a positive NPS score of 34. The fact that service uses provide a positive Net Promoter Score is encouraging given that as reported earlier in Chapter 4, word of mouth is the fourth most common means of becoming aware of Ombudsman Services (12%). Breaking responses down by Sector shows that this level of advocacy is equal for Energy (NPS of +36) and Communication (+33), but is lower among those who had a Property case. Indeed, for these individuals a negative NPS of -13 is found, with the proportion who are Detractors (53%), being greater than the proportion who are Promoters (40%). 45

48 Customer Satisfaction Monitoring 2017 Figure 30: And how likely would you be to recommend Ombudsman Services to family or friends if they had a similar issue to yours. Please answer on a scale of 0 to 10, where 0 is not at all likely and 10 is extremely likely? (All responses) 0 - Not at all likely (0) Extremely likely (10) 2% 2% 2% 2% 4% 2% 3% 6% 5% 15% 57% 0% 10% 20% 30% 40% 50% 60% Sample base: 5161 Again a clear interaction between the NPS and the outcome an individual achieved can be found. Among those who had their case Settled a NPS score of +76 is observed, with strongly positive score found among those whose case was Upheld (39%). However where the outcome was either Not Upheld or Maintained the NPS drops to become negative (-22 and -4 respectively). 46

49 Overall satisfaction measures At a total sample level, the Net Promoter Score is positive in all four quarters of In line with some of the other indicators in the dataset this indicator has had a positive trajectory throughout the year. The NPS for Quarter 4 at (+40) is 10-percentage points higher than in Quarter 1 as shown by the figure below. Figure 31: And how likely would you be to recommend Ombudsman Services to family or friends if they had a similar issue to yours. Please answer on a scale of 0 to 10, where 0 is not at all likely and 10 is extremely likely? (All responses) Net Promoter Score Sample base: Q1: 1619, Q2: 1038, Q3: 1297, Q4:

50 Customer Satisfaction Monitoring Suggestions for improvement At the end of the survey all respondents were given the opportunity to make suggestions regarding how the service they received could be improved. Within these responses, comments were made that re-enforce the generally high level of positivity seen in the quantitative data: Great job continue the high standards. I thought the service was fantastic. No, they are the best people I've had dealings with for years!... they always got back to me if I needed help with a query, and I was kept informed. In my opinion no, the service was excellent. Keep doing what your (sic) doing as its a great help when dealing with the big companies Superb service. Lovely people to deal with. The remaining comments pick on the themes identified within the responses to earlier questions, namely the perceived quality of the investigation including potential bias; speed, communication and technology issues. In some of these responses there is a perception that Ombudsman Services role as an advocate for/defender of the customer, is not recognised as strongly as might be hoped. Quality of investigation/interpretation Take the time to look at the evidence supplied properly. Not lecture victims about how the Ombudsman service is free for consumers. This isn't some kind of privilege for consumers, it's a right. Just ensure that they get the full facts and 'be fair and reasonable' with their outcomes. Check the actual information received from the company, before accepting they have complied to the Ombudsman instruction. Listen to the complaint and all of the implications. Actually listen to the customer and be prepared to tackle companies when they are lying. Perceived bias/ lack of impartiality Be more unbiased. Try and be less biased, and put themselves in the shoes of the customer rather than the supplier. Listen to consumers. Be a little less arrogant, a lot less dismissive and maybe just take the time out to really consider the facts. Behave independently. Apply logical reasoning to cases. 48

51 Overall satisfaction measures They should be impartial and value the complaints of individuals, no matter how small they are compared to big organisations. Support the consumer Defend the consumer. Once a final decision is made more should be done to help the customer if they are not happy with the overall outcome. Stick up for the customers not the companies. Read the evidence on both sides and listen to the complainant. Take a common sense approach to resolving the problem and don't just assume that the company is right. Actually help the consumers that need these services come to fair and equitable resolutions with these massive organisations, that waste consumer time! Speed My only observation was that after you taking approximately 6 weeks to investigate the complaint, I was only given 14 days to respond 11 days of which I was away on holiday. I would have thought that during the holiday period, an allowance would have been made for people being unavailable due to holidays. Overall I was very satisfied but the case may have been closed too quickly. Within days of the closure I received another incorrect bill from the company. Hopefully they have put this right but I will not know until I receive the next bill. I would rather the Ombudsman's file was still pending until that is resolved. Try and work a result quicker if possible having tried to come to a result with our old supplier over several months. Communication A little more communication about timescales to expect. Better communication about case progress. More independent in terms of the assessment, should take into account where suppliers give appalling customer service. The rudeness I experienced was not considered at all. Communication with the person the way they would like to and support them thought out Technical/portal issues I did not find the portal useful as found it difficult to upload documents so I used to communicate. I think that if any improvement it would be in this area. Sort out the portal as it was confusing to upload documents and couldn't do anything with an ipad. 49

52 Customer Satisfaction Monitoring 2017 Completely redesign the website portal area that is supposed to store all the documents relating to a case. It did not appear to me to serve any useful purpose and was slow and cumbersome when I tried to use it. Clearer online application process. Phone is good though. Portal could be easier to use if compatible with all systems. The service as a whole was excellent. The only issue we had difficulty with was using the portal as there was a lack (or we couldn't find) and help or information on how to actually use it. On contacting them about it though, they advised of different ways we could send/access information which were better for us. Improve the IT system so that documents can be uploaded more easily. 50

53 Appendix: Statement of Terms 11 Appendix: Statement of Terms Compliance with International Standards BMG complies with the International Standard for Quality Management Systems requirements (ISO 9001:2008) and the International Standard for Market, opinion and social research service requirements (ISO 20252:2012) and The International Standard for Information Security Management (ISO 27001:2013). Interpretation and publication of results The interpretation of the results as reported in this document pertain to the research problem and are supported by the empirical findings of this research project and, where applicable, by other data. These interpretations and recommendations are based on empirical findings and are distinguishable from personal views and opinions. BMG will not publish any part of these results without the written and informed consent of the client. Ethical practice BMG promotes ethical practice in research: We conduct our work responsibly and in light of the legal and moral codes of society. We have a responsibility to maintain high scientific standards in the methods employed in the collection and dissemination of data, in the impartial assessment and dissemination of findings and in the maintenance of standards commensurate with professional integrity. We recognise we have a duty of care to all those undertaking and participating in research and strive to protect subjects from undue harm arising as a consequence of their participation in research. This requires that subjects participation should be as fully informed as possible and no group should be disadvantaged by routinely being excluded from consideration. All adequate steps shall be taken by both agency and client to ensure that the identity of each respondent participating in the research is protected. 51

54 With more than 25 years experience, BMG Research has established a strong reputation for delivering high quality research and consultancy. BMG serves both the public and the private sector, providing market and customer insight which is vital in the development of plans, the support of campaigns and the evaluation of performance. Innovation and development is very much at the heart of our business, and considerable attention is paid to the utilisation of the most up to date technologies and information systems to ensure that market and customer intelligence is widely shared.