NLP WHAT S HAPPENING TO NLP?

Transcription

1 WHAT S HAPPENING TO NLP? A Guide to Synthesio s New & Improved Natural Language Processing Capabilities

2 WHAT IS HAPPENING TO NLP? Synthesio s Natural Language Processing (NLP) model is about to undergo a major upgrade, which will bring enhanced accuracy to sentiment tagging on mentions detected as English. While this update only affects English, future updates will bring the new ASA model to other languages. Your CSM is available to discuss the upgrade with you, but this guide will cover all the changes to the ASA model as well as questions we have anticipated. OVERVIEW OF SYNTHESIO S NLP CAPABILITIES Synthesio s NLP services consist of 2 core tools:! Automatic Sentiment Analysis (ASA)! Language Identification (langid) SYNTHESIO S PREVIOUS SENTIMENT MODEL The sentiment analysis model we have used in the past, while simple, provided a high level of precision in terms of positive and negative sentiment tagging. In practice, this meant it would only tag a mention as positive or negative if it had a high level of certainty. That is why many mentions that would seem positive or negative to a human observer were nevertheless tagged by the system as neutral. This was particularly true for small messages such as Tweets. THE NEW & IMPROVED MODEL In addition to having improved algorithms, our new sentiment model takes into account new data hand-coded by Synthesio NLP specialists and customers. With the help of our customers, we have been able to manually apply sentiment tags to a massive number of mentions in our database, enhancing the accuracy and reliability of our NLP algorithms. This means the platform will be more accurate at identifying positive, negative and neutral mentions in your dashboards going forward. In addition, future upgrades will allow us to deploy continuous incremental improvements on new code data. HISTORICAL SENTIMENT ANALYSIS The new sentiment model will only apply to data after the launch (i.e. only data going forward). However, upon request, we can re-batch your data so that it all gets reprocessed with the new model either for greater accuracy or to make it easier to perform comparisons over time.

3 UNDERSTANDING THE IMPROVEMENTS The new sentiment model has several improvements: 1. The new algorithms use a classification system instead of a rule-based system. This lets us be more precise when determining whether or not sentiment is present in a given mention. 2. The new model uses a complex decision system rather than a simple classifier. 3. New data has been added to the creation of the decision algorithm. This increases the number of contexts in which the platform can detect sentiment. 4. In developing the new models, we have used: 1. All the human and machine-coded data we had available up to the release date. 2. All mentions that were corrected by customers on automatic dashboards. 3. Several thousand new code entries on small mentions mostly Tweets in order to improve sentiment tagging precision on these. HOW OUR CUSTOMERS WORK WITH NLP Our proprietary text analytics services, NLP in platform 80% 20% Outsourced to other text analytics vendors

4 FAQ & ADDITIONAL NOTES What is NLP? Natural Language Processing (NLP) is an effort to combine artificial intelligence (machine learning) and linguistics to teach computer systems to understand the sentiment and meaning of natural human language. What does Synthesio s NLP offering consist of? Synthesio s NLP services consist of two core tools: Automatic Sentiment Analysis (ASA) and Language Identification (langid). How do Synthesio customers use NLP? 80% of our customers use our proprietary text analytics services (NLP in platform). The other 20% use third-party NLP software tools such as Lexalytics or Clarabridge. How did the old NLP model work? Our previous NLP model was rule-based, meaning it would only tag mentions as positive or negative if it had a high level of certainty. How does the new NLP model work? The new NLP model is classification-based, meaning it can identify the presence (or absence) of sentiment in more contexts and apply a positive/negative/neutral tag with greater precision. What changes can I expect in my dashboards as a result of these NLP updates? For most dashboards, the volumes of positive, neutral and negative mentions will shift slightly. In some cases there will be more significant changes, and we ll be reaching out to you directly if there will be larger changes. Does this mean my previous sentiment data was inaccurate? No. The previous model was not inaccurate just conservative in assigning a sentiment value. This update only improves upon a system that already led the industry in sentiment identification accuracy. How will this affect my data prior to the update? The new sentiment model will only apply to data after the launch (i.e. only data going forward). However, upon request, we can re-batch your data so that it all gets reprocessed with the new model either for greater accuracy or to make it easier to perform comparisons over time.

5 FAQ & ADDITIONAL NOTES Can I continue manually tagging mentions? Yes. Manually correcting mentions in your dashboards helps us improve our machine learning algorithms so we can continuously increase the accuracy of our automatic tagging system. What happens when I request a batch reprocessing of my data? There are two types of batch reprocessing, one that includes NLP and one that does not. As long as your request does not include NLP, your older sentiment data remains intact. Should you request a batch reprocessing including NLP, all of your sentiment analysis prior to the cut-over date will be reprocessed using the new ASA model. What happens when I modify dashboard scope (add or modify Topics/Sub-Topics, add countries or media types, etc.)? For any changes that will retrieve more mentions from our database, all mentions will retain sentiment from the time they were processed. In other words: - Mentions processed by Synthesio before the cutover date " old ASA - Mentions processed by Synthesio after the cutover " new ASA If you previously batch-reprocessed your dashboards to the new ASA model (per the above), any changes will require another NLP batch reprocessing to maintain consistency in sentiment model. Please make this request via your CSM. If you chose to retain the old ASA on pre-cutover mentions, the sentiment of added mentions from Synthesio s database will be consistent and require NO reprocessing. What happens when I request a back data purchase? When back data is added it will be processed using the new ASA model. Your historical results may have mixed sentiment analysis. We recommends an NLP batch reprocessing to insure sentiment consistency. Please make the request via your CSM. Can I continue to use the old ASA model? The new ASA model will be the default ASA used on all incoming data after the cut-over date.

6 Synthesio, named the leader in The Forrester Wave : Enterprise Listening Platforms, Q report, is a Global Social Intelligence Platform. Used by some of the world s top brands, and the agencies that support them, Synthesio is the framework for building social intelligence that provides real business results. Whether an organization s social team is built within Marketing or crosses multiple departments, business units or geographies, Synthesio helps teams listen, analyze, and engage with consumer conversations across social and mainstream media within one platform. Founded in 2006, Synthesio has offices in New York, Paris, London, and Singapore.