Applying AI the Right Way to Fully Leverage Social/VoC Data for CX

Transcription

1 Applying AI the Right Way to Fully Leverage Social/VoC Data for CX Forrester CX Exchange June 18, 2018 Rob Key, CEO, Converseon

2 A Leader in Social Intelligence for Over 15 Years Mission: provide the world s best social data quality and insights through AI-powered technology (Conversus), ecosystem partners and deepest industry experience. We have been developing machine learning for social/voc data for over a decade. Drive highest precision/recall (exceed human level performance) Transform use of data from reactive/descriptive to predictive/prescriptive Enable clients to build their own custom models using machine learning on top of our core technologies through democratized AI Provide transparency of model performance and enable adaptation to client environments and taxonomies Power the social/voc ecosystem through API integrations (Brandwatch, Sprinklr, Crimson Hexagon, etc)

3 3 Language serves not only to express thought but to make possible thoughts which could not exist without it. - Bertrand Russell

4 4 Our Obligation In an era where the collective voice of customers and citizens, empowered through social channels, have become a primary agent-of-change and transforming governments, societies, industries, brands and products, there is arguably no greater obligation for our industry than to effectively, thoroughly and accurately capture, analyze report and act on these needs, wants, experiences, hopes, opinions of these voices.without inadvertent discrimination or bias. Converseon Obligation Statement

5 But human language is complicated which is why companies, on average, are processing only 21% of their unstructured data (Forrester Research). And data that is processed In almost all cases falls short of our precision obligations

6 We re now more than 60 years into trying to understand language at scale Georgetown Experiment Ontologies Statistical Models Deep Learning AI as a Service Micro World Grammars

7 Key Challenges Challenges of Disambiguating Words, Meanings and Context Overfitting of machine learning classifiers: Reviews of Anne Hathaway are biased to be positive

8 Sarcasm and slang abound. Context is challenging. Lady Gaga killed it at halftime Most automated sentiment approaches achieve only 60% precision versus human gold standard.

9 Word-Spotting approaches miss essential and implicit meaning. I spent my entire lunch hour waiting In line for my prescription...

10 Language Differs by Domain and Context Unpredictable Movie is Positive Unpredictable Breaking, Not so Much

11 Human Bias is Often Reflected in Machine Learning Algorithm Bias The concern around discriminatory outcomes in machine learning is not just about upholding human rights, but also about maintaining trust and protecting the social contract founded on the idea that a person s best interests are being served by the technology they are using or that is being used on them. Absent that trust, the opportunity to use machine learning to advance our humanity will be set back. World Economic Forum

12 1 Not Surprisingly Far from being unfixable, however, miscalculations in social-media analyses can already be fixed using methods developed to fix similar problems in studies in epidemiology, statistics and machine learning. - ComputerWorld

13 1 AI is a Game-Changer (But Only if Done Right )

14 ML approaches generally outperform (individual) human performance (on average) Precision: Accuracy 0% 20% 40% 60% 80% 100% Standard Approach Conversus Baseline Recall 0% 20% 40% 60% 80% 100% Relevancy 0% 20% 40% 60% 80% 100% Source: Converseon 2018

15 And Can Classify Language Like Humans Do 1 Advocacy Motives Emotion Purchase Intent Customer Journey Brand Attributes Unmet Needs Intensity Sentiment

16 Paving the Way for Broad Application to CX Examples for purposes illustration Social Care CX / VoC Insights Social NPS 10 Net Promoter Score: 61 This value is the average of the combined scores multiplied by 10. The only thing we like better than a sweet Chevy is seeing one being driven hard... REALLY can you can you please shed some light on how to file a warranty claim? My husband's '16 Chevy Colorado has been at the dealer service for a week 16 - Speed-to-insight - Reduced costs - Greater efficiency - Root cause analysis - Real 40k miles my Sonic has had an engine replaced and NOW my transmission. Dealer is doing their best but HOLY CRAP what the heck?

17 But Doing it Effectively Requires Critically Important Approaches 1. Active Inclusion The development and design of ML applications must actively seek a diversity of input, especially of the norms and values of specific populations affected by the output of AI systems.. World Economic Forum Three diverse analysts code same sample data independently. Instances where they agree (consensus) in their coding form the basis of an accuracy gold standard. Intercoder reliability and adjudication essential.

18 2. Don t Prejudge Organic Topic Discovery Provides a bottoms up approach to organically discover emerging topics/issues. Solves the problem of only finding what one looks for. Avoid potential top down inadvertent bias

19 3. Require Accuracy Scoring and Performance Understanding Clear Performance Measurement and Validation of ALL Classifiers Before Deployment and Use F1 score shows specific classifier performance based on precision and recall, including when generalized.

20 4. Process Even the Hard Stuff Although mixed sentiment, sarcasm, slang and implicit meaning challenging, it still requires effective processing and cannot just be ignored. In fact, it often is the data with the most rich insight once unlocked. ALL expressions of opinion must be captured and analyzed

21 6. Document Level Won t Do: Target or Facet Level Essential Negative for Overall Brand You post about togetherness and family, yet you disrespect your customers and don t support them when they get a defective vehicle. My 2018 Colorado is still leaking after multiple trips to the dealer and you guys won t do anything for me. Negative for Dealership Experience Negative for Nameplate: Leaking Oil

22 5. Keep Humans-in-the-Loop and Leverage Domain Expertise Semi-supervised methods provide confidence scores associated with each record processed Allowing domain experts to evaluate threshold confidence records for further scrutiny to help avoid bias and improve data quality through additional analysis. This human-in-loop approach essential for effective machine learning model development

23 Summary: The Transformation Underway Rules-based Unsupervised Document-level analysis One-size-fits-all models Black box performance Significant Bias Platform Provided Machine learning Semi-supervised Entity-level analysis Custom models for Industry and organization Transparent measurement Unbiased Brand Control

24 Thank You Conversus.AI Key Features Built on training corpus and continually evolved with active machine learning Leverages domain expertise Allows customization of classifiers to organizational taxonomies ) Allows users to rapidly build new custom classifiers align to analysis requirements Provides automated performance scoring and ability to apply SLAs For further discussion and/or demo of Conversus, leading Machine Learning-As-A- Service Text Analytics platform, Please contact me at rkey@converseon.com