White paper. QoE-driven network & business optimization

Transcription

1 White paper QoE-driven network & business optimization

2 QoE-Driven Network & Business Optimization CONTENTS INTRODUCTION QoE IMPACTING FACTORS LEVERAGING QoE INDICATORS Global Satisfaction Index (GSI) per service or per application Differentiated QoE per customer segment, per subscriber, per service Correlation TCPDR and radio measurements Moving from NPS-only to predictive satisfaction indicators QoE, everyone s business MEETING THE NEED FOR MEANINGFUL METRICS Voice Video Internet browsing Gaming CONCLUSION GLOSSARY 2

3 It works! I have network coverage at last! How are you? Introduction Communication Service Providers (CSPs) are faced with rising operating costs, flattening ARPU and the need to address an increasing demand for data. On top of that, according to research 1, 50% of subscribers are at risk of churning in the next 12 months. The survey shows that the quality of mobile broadband experience is the leading driver for mobile operator churn, with 37% of consumers citing slow connection speeds as the main reason for churning. CSPs recognize these figures and know that churn corrodes their business. Quality of Experience (QoE) measures the quality perception from an end user s point of view of the delivered services and is therefore a good indicator of a subscriber s satisfaction level. So, in order to prevent churn and differentiate themselves from competitors, understanding and assuring the quality of subscriber s experience is paramount. Delivering superior customer experience has therefore become a central concern and a constant battle for CSPs. The implementation of a Service Operation Center (SOC) puts QoE at the center of the organization and breaks the silo-based organization. A SOC provides the means to monitor services in an end-to-end context and to leverage actionable data in order to deliver the best possible QoE. This paper explores the different factors that impact QoE, proposes different methods for measuring QoE and leveraging this information intelligently to different teams. We shall introduce meaningful QoS metrics to monitor, through customer case studies, we shall see how QoE-oriented network optimization contributes to an intensified consumption of mobile data services, enhanced customer loyalty and increased revenues. 1 Ovum 2014 Telecoms Customer Insights 3

4 QoE-Driven Network & Business Optimization QoE IMPACTING FACTORS QoE is all about how satisfied a subscriber is with a service in terms of accessibility, responsiveness, quality and retainability. Subscriber QoE is based on factors such as: The ability to make a phone call when on the move The time required to download a webpage The responsiveness of a mobile application The amount of stalling in the video being viewed The performance of a particular mobile device A multitude of elements have an impact on QoE: device and OS type, latency, congestion levels, application type, network topology, location, time etc. In order to decide how to optimize their networks CSPs require a holistic view of the subscriber, network, device and application usage. Leveraging QoE indicators Global Satisfaction Index (GSI) per service or per application Based on its thorough experience with more than 200 CSPs world-wide, Astellia has developed a real-time global satisfaction index which gives detailed information on QoE per service for a group of subscribers or an individual. This GSI takes into account QoE of voice, messaging and selected data services, as well as access & mobility. Alarms can be triggered when a metric is out of range and hence impacting the subscriber experience. Case study : QoE of Top10 apps At a Middle East CSP, Astellia monitors the QoE of the top 10 most used applications and evaluates its evolution over time. This allows the CSP to create a partnership with the right application provider and to optimize the QoE of these most used applications. Through alarms the CSP is informed of any quality degradation and is then able to locate the issue (RAN / Core / content server). If the quality regression comes from the application provider, the CSP can send them a trouble ticket. 4

5 Case study : Customer-centric network performance indicators A European CSP wanted to use a customer-centric indicator to measure network performance and customer experience while benchmarking service perception across the different regions in the country. Astellia and the CSP created together a GSI based on the correlation of 3 main KPIs: Call set-up efficiency Call drop Throughput data for data sessions to identify the worst performing areas Optimization campaigns were prioritized on these areas, followed-up by regional marketing campaigns to reduce churn. The vendor independent indicator was also used to assess operations efficiency and allowed to follow-up SLAs with Managed Services Providers. This GSI was used afterwards to assess customer experience of different MVNOs and inbound roamers. Differentiated QoE per customer segment, per subscriber, per service Data explosion and the increased number of applications create usage diversity that should be better leveraged to service the end user and is creating the need for adapted customer experiences. It is of no use to provide high throughputs to subscribers using only Twitter, while a subscriber watching a video on YouTube requires a seamless data service. The issue is all the more important if it concerns high value subscribers rather than occasional service users. CSPs therefore need to monitor a differentiated QoE based on subscribers main centers of interest. They have to define and weigh application and services KPIs differently for each customer segment and adapt network optimization campaigns accordingly. Case study : Differentiated QoE per customer segment 12% 39 USD Media lovers Other 14% OS Apple Android 27% 57% 8,8 GB Subscribers belonging to the Media Lovers customer segment at a Middle East CSP are consuming on average 8,8GB per month by watching videos and streaming music. Video delays and frozen video are having a higher impact on these subscribers than on those surfing only social media sites like Facebook and WhatsApp. Astellia therefore puts more weight on selected KPIs like throughput, packet retransmission, jitter, buffering delay to evaluate the customer experience of this segment. Astellia provides per segment a list of subscribers with bad QoE and hence with a potential to churn. By taking into account the ARPU of the impacted subscribers, optimization activities are being prioritized on high revenue generating customers prone to churn. 5

6 QoE-Driven Network & Business Optimization Case study : Optimizing CX on high speed trains A high speed train is a very complex environment due to the high speed at which customers are travelling. So special challenges require special solutions. One of Juan Serrano Sanchez, Quality Manager at Orange Spain, objectives is preventing any quality degradation in the customer experience during traveling on a high speed train. For this, central and regional teams are focused on monitoring KPIs, analyzing root causes, detecting gaps and proposing improvement plans. Before, Orange Spain mainly used drive tests to optimize network conditions along the railway. Now, with Astellia, they adopt a new and very innovative approach which manages to filter out those people that are actually on the train. The optimization of high speed train routes is particularly important for business travelers who need to stay connected at any time. This allowed Orange Spain to become the best service provider on high speed trains in Spain and created a real competitive differentiator. 6

7 Correlation TCPDR and radio measurements When a subscriber experiences bad throughput, the cause can be manifold: device, content server, radio, core, etc. Correlating application usage and radio measurements allows root cause problem elimination: having good radio measurements doesn t necessarily mean providing good QoE. However, if radio measurements are bad (coverage, congestion, etc) it will result in poor QoE. Methodology: Correlating probe-based data and call traces Different data services and applications require adapted levels of throughput and latency. So with exactly the same radio conditions, customer experience might be completely different. Therefore, it is very important to be able to correlate user plane information (including application usage info) provided by probes with radio measurements coming from BSC, RNC and enodeb call traces. PROBE + CALL TRACES Because of bad radio conditions, these customers stopped their web browsing session, leading to immediate loss of revenue for the operator. The poor radio conditions didn t have an immediate impact on youtube usage since part of the videos had been buffered. After that, customers stopped using the service. 7

8 QoE-Driven Network & Business Optimization Moving from NPS-only to predictive satisfaction indicators Marketing departments often use customer surveys including the Net Promoter Score (NPS), a customer loyalty metric, to measure customer satisfaction based on a simple question Would you recommend our company to a friend? Subscribers are then categorized as promoters, passives or detractors. The Net Promoter Score translates customer satisfaction at a given point in time for a sample of customers but it doesn t allow operators to identify precisely why the customer is not likely to recommend their brand nor does it allow them to identify the reason of their dissatisfaction (bill shock, bad QoE, better competitive offer, etc.). Therefore it is impossible for operators to troubleshoot and solve issues and therefore retain these customers. Astellia has developed Satix, a powerful customer-centric indicator, that correlates radio network metrics and end-user perception information collected through customer surveys. Customer experience feedback such as blocked calls, dropped calls, coverage issues, voice and data quality are correlated with technical radio measurements such as RSCP, Ec/N0, throughput, CQI, ne market in Spain, we nswer the following questions : BLER, time spent on 4G, 3G vs time spent on 2G. The result is an advanced customer satisfaction index that not only provides objective customer perception indicators but also precise location information where radio optimization has to be reinforced. In case of a bad customer experience radio optimization teams will know exactly which parameters to fine-tune to improve QoE. This customer-centric optimization allows prioritizing of network investments based on high value customers perception. S How likely would you recommend OPERATOR to a friend, a relative or a COVER.IN How likely would you rate OPERATOR s coverage indoor? URBAN COVER And the coverage in urban areas? RURAL COVER And the coverage in rural areas? E X A M P L E Customer survey BLOCKING In areas with a good coverage, how would you rate your level of satisfaction r the capacity to establish a call on the first attempt? ROPS And the capacity to keep a call without lo EECH ALITY And the quality 8

9 Case study : Correlating network performance and NPS A European Tier 1 operator that relentlessly pursues network performance excellence to provide the best customer experience called upon Astellia for customer-centric optimization. Satix radio network metrics at this Tier 1 operator are based on traffic of 12 million subscribers covered by G/3G/4G cells These metrics are then compared with user perception information based on weekly automatic customer surveys amongst a panel of 3000 subscribers. The correlation between Satix and Survey scores enabled the operator to discern whether a customer was dissatisfied due to poor network quality (low Satix score) or due to nonnetwork related reasons i.e. commercial reasons (high Satix score). CX Survey Correlation Network performance indicators SatiX CX index QoE scored by subscribers Coverage Blocked calls Dropped calls Voice quality Data quality Calibration Technical metrics RSCP Throughput CQI / BLER Ec/No,... CX index & triggering optimization actions To improve QoE & network performance The survey score identifies detractors, i.e. customers on the verge of churning. The Satix score however shows that these customers have not experienced any network quality degradation. The survey scores detractors. The Satix score is very low and indicates that the customer has experienced several network quality issues. IMSI SATIX ATTRIBUTES Customer Promoter Reason for Attribute Voice Attribute Voice Attribute Speech Attribute Attribute Data SATIX score survey Neutral disatisfaction Blocks Drops Quality Coverage Quality score Detractor IMSI 1 4,98 5,00 2,58 1,63 1, Detractor Network QoS related IMSI 237 4,97 5,00 2,21 1,74 1, Detractor Network QoS related IMSI ,95 4,8 2,38 1,79 1, Detractor Network QoS related IMSI 412 4,84 4,83 2,29 1,94 1, Detractor Network QoS related IMSI ,00 5,00 1,98 1,35 1, Detractor Network QoS related IMSI ,99 4,96 3,42 3,03 3, Detractor Non-technical reason IMSI ,99 4,96 3,51 3,16 4, Detractor Non-technical reason IMSI 486 5,00 5,00 3,65 2,27 3, Detractor Non-technical reason IMSI 9 4,98 4,86 3,50 2,95 3, Detractor Non-technical reason IMSI ,96 5,00 2,82 2,66 3, Detractor Non-technical reason Correlation between Satix and NPS Survey This information was then used by marketing teams to launch win-back campaigns and by network operations teams to better target network optimization campaigns based on customer location and value. 9

10 QoE-Driven Network & Business Optimization QoE, everyone s business Understanding, managing and ensuring a good QoE is everyone s concern within a CSP. Network performance and radio engineers can perform proactive QoE monitoring by diagnosing QoE & service degradation and drilling down to the root cause of the problem. For optimal ROI, they can prioritize their investigations on high value customers, corporate subscribers and key areas. Marketers can improve campaign effectiveness by including QoE as a dimension for understanding their target audience. It will also allow them to improve their ability to predict which subscribers are likely to churn. Moreover, they will need to adapt the timing of their upsell and cross-sell campaigns to send the right offer at the right time. Device managers need to understand device behavior to select the best performing ones for their new data plans and inform device manufacturers or network equipment vendors in case of a device issue. Customer Care needs information to ascertain in real-time the level of satisfaction for every subscriber and for each type of service (voice, messaging, data). They need to know if the customer is the only one impacted by the issue or if it is the issue that impacts all subscribers in that area. Service Operation Center s (SOC) manages E2E service quality and is at the heart of a CSPs customer focused strategy. A SOC breaks down organizational barriers and has the role of an intermediary between the operations, engineering, performance, marketing and customer care teams. 10

11 Meeting the need for meaningful metrics There are numerous elements which can degrade QoS and might affect QoE. As mentioned before, it is often a combination of network availability, capacity, throughput, location, time-of-day, usage profile, and device. To deliver consistent quality, there must be consistency in the metrics used to determine that quality. CSPs therefore need QoS (KPIs) & QoE (KQIs) metrics that take into consideration all of the impairments that affect the quality of each service. Voice Call Setup Time Call Setup Delay Call Setup Success Call Drop Call Block Connection Gap SRVCC Success SRVCC Speech CSFB Success CSFB Setup Time RTP Packet Loss RTP Jitter RTP Delay Bearer Establishment Success MOS Video Jitter Latency Throughput Stalling Rate Stalling Duration Packet loss Transaction Delay Frame Rate Buffering Delay Session Drop Session Start Time Cut-off Rate Establishment Success Establishment time Used codecs Internet browsing Http Efficiency Uplink Efficiency Downlink Efficiency Response Time Packet Loss Throughput Transfer Transfer Duration Bitrate duration Page Refresh Delay Page Download Delay Packet Loss Session Setup Success Send/Receive Delay Abort Rate Gaming Jitter Latency Ping Packet Loss Delay 11

12 QoE-Driven Network & Business Optimization Voice Voice still remains a vital service CSPs run over their networks. The destabilizing impact of OTT providers is putting pressure on CSPs to deliver quality voice services. Room for error round voice services is therefore slight to non-existent. Delivering a quality voice service in an LTE environment is a major goal. For VoLTE to succeed, end-to-end QoS is essential. CSPs will need to provide seamless fallback (SRVCC) to 2G/3G networks where LTE is unavailable. Case study : Launching VoLTE service Astellia helped a European operator launch its VoLTE service focusing optimization activities on 3 areas: Astellia analyzed the SRVCC efficiency and delay per handset and indicated the best performing devices for VoLTE Fast Voice call establishment Voice call continuity with 3G & 2G Voice & data quality Astellia could show that voice call setup with VoLTE was clearly faster than with CSFB: ms vs 8 sec Astellia provides the MOS score of each VoLTE call. Our solution analyses a call every 5s, from start to end, to depict the real quality of experience of the voice service. This provides a much more reliable vision of customer experience than competitors who calculate the average MOS. 12

13 Video Cisco s report estimates that video will account for 70% of traffic by 2018, driven by VOD and P2P streaming and even mobile games containing video. So, there is no application that is more important to the consumer experience than mobile video, especially for video addicts. Ensuring a consistently great video experience by preventing video blackouts, frozen video, silence, buffering delays etc. will have an immediate impact on customer satisfaction and loyalty. Methodology: Analyzing video quality When measuring, for instance, the user experience of on-demand Internet streaming service Netflix, it is important to make the distinction between > Browsing Netflix navigation menu > Watching a movie The KPI server response time reflects the user experience for menu browsing but throughput is a better KPI for evaluating the movie watching experience. Browsing Watching KPI Server response time Throughput Internet browsing Subscriber waiting time is the key determinant of web browsing QoE: the longer users have to wait for the web page to arrive, the more dissatisfied they tend to become with the service. Gaming QoE of so called First Person Shooter games poses strict requirements with respect to network quality. Having a low ping is desirable because lower latency provides smoother gameplay by allowing faster updates of game data. 13

14 QoE-Driven Network & Business Optimization satisfy analyze improve Conclusion Operating mobile networks today is expensive for CSPs, competition from OTTs is fierce and revenues are flattening. Subscribers increasing reliance on mobile networks and the exponentially rising demand for mobile data services are putting pressure on the network. However, CSPs cannot afford to continue simply adding more bandwidth, especially in cases where bandwidth alone isn t the cause of the customer s poor experience. Optimizing QoE has therefore become a prime concern for everyone. Measuring QoE is difficult and CSPs rely on different QoE indicators to do so. Focus on QoE is worth the effort and holds a great opportunity to improve loyalty, reduce churn, raise ARPU and margins. In other words, QoE is the key to drive customer lifetime value and emerges as the new battleground for CSPs. 14

15 Glossary ARPU BLER BSC CQI CSFB CSP GSI KPI KQI MOS NPS P2P QoE RAN RNC ROI RSCP RTP SOC SRVCC TCPDR VOD VoLTE Average Revenue Per User Block Error Rate Base Station Controller Channel Quality Indicator Circuit Switched Fall Back Communication Service Provider Global Satisfaction Index Key Performance Indicator Key Quality Indicator Mean Opinion Score Net Promoter Score Peer to Peer Quality of Experience Radio Access Network Radio Network Controller Return on Investment Received Signal Code Power Real-time Transport Protocol Service Operation Center Single Radio Voice Call Continuity Transmission Control Protocol Data Record Video on Demand Voice over LTE 15

16 ZA Airlande 2 rue Jacqueline Auriol CS Rennes Cedex 9 France Tel : Fax : infos@astellia.com BRAZIL CANADA INDIA LEBANON RUSSIA SOUTH AFRICA SPAIN USA