Part 3: Sentiment Analysis

Transcription

1 Part 3: Sentiment Anaysis The University of Sheffied, This work is icenced under the Creative Commons Attribution-NonCommercia-ShareAike Licence

2 What is Sentiment Anaysis? Aso known as Opinion Mining A recent discipine that studies the extraction of opinions using IR, AI and/or NLP techniques. More informay, it's about extracting the opinions or sentiments given in a piece of text Web 2.0 nowadays provides a great medium for peope to share things. This provides a great source of unstructured information (especiay opinions) that may be usefu to others (e.g. companies and their rivas, other consumers...)

3 It's about finding out what peope think...

4 Opinion Mining is Big Business Someone who wants to buy a camera Looks for comments and reviews Someone who just bought a camera Comments on it Writes about their experience Camera Manufacturer Gets feedback from customer Improve their products Adjust Marketing Strategies

5

6

7 It's not just about product reviews Much opinion mining research has been focused around reviews of fims, books, eectronics etc. But there are many other uses companies want to know what peope think finding out poitica and socia opinions and moods investigating how pubic mood infuences the stock market investigating and preserving community memories drawing inferences from socia anaytics

8 But there are ots of toos that anayse socia media aready... Streamcrab Semantria Socia Mention Sentiment140: TipTop:

9 Why not use existing sentiment apps? Easy to search for opinions about famous peope, brands and so on Hard to search for more abstract concepts, perform a non-keyword based string search e.g. to find opinions about Lady Gaga's dress, you can often ony search on Lady Gaga to get hits They're suitabe for a quick sanity check of socia media, but not reay for business needs And the opinion finding they do isn't aways very good...

10 Why are these sites unsuccessfu? They don't work we at more than a very basic eve They mainy use dictionary ookup for positive and negative words Or they use ML, which ony works for text that's simiar in stye They cassify the tweets as positive or negative, but not with respect to the keyword you're searching for First, the keyword search just retrieves any tweet mentioning it, but not necessariy about it as a topic Second, there is no correation between the keyword and the sentiment: the sentiment refers to the tweet as a whoe Sometimes this is fine, but it can aso go horriby wrong

11 Positive tweets about fracking Hep me stop fracking. Sign the petition to David Cameron for a #frack-free UK now! I' take it as a sign that the gods appaud my new antifracking country ove song. #Cameron wants to change the aw to aow #fracking under homes without permission. Te him NO!!!!!

12 Be carefu! Sentiment anaysis isn't just about ooking at the sentiment words It's a great movie if you have the taste and sensibiities of a 5- year-od boy. It's terribe Candidate X did so we in the debate ast night. I'd have iked the fim a ot more if it had been a bit shorter. Situation is everything. If you and I are best friends, then my gracefu swearing at you is different from if it s at my boss.

13 Death confuses opinion mining toos Opinion mining toos are good for a genera overview, but not for some situations

14 Nobody iked Leonard Nimoy

15 Or did they?

16 Opinion Mining for Stock Market Prediction It might be ony fiction, but using opinion mining for stock market prediction has been aready a reaity for some years Research shows that opinion mining outperforms eventbased cassification for trend prediction [Boen2011] Many investment companies offer products based on (shaow) opinion mining

17 Derwent Capita Markets Derwent Capita Markets aunched a 25m fund that makes its investments by evauating whether peope are generay happy, sad, anxious or tired, because they beieve it wi predict whether the market wi move up or down. Boen tod the Sunday Times: "We recorded the sentiment of the onine community, but we coudn't prove if it was correct. So we ooked at the Dow Jones to see if there was a correation. We beieved that if the markets fe, then the mood of peope on Twitter woud fa. "But we reaised it was the other way round that a drop in the mood or sentiment of the onine community woud precede a fa in the market.

18

19 But don't beieve a you read... It's not reay possibe to predict the stock market in this way Otherwise we'd be miionaires by now! In Boen's case. the advertised resuts were biased by seection (they picked the winners after the race and tried to show correation) The accuracy caim is too genera to be usefu (you can't predict individua stock prices, ony the genera trend) There's no rea agreement about what's usefu and what isn't

20 Who Wants to be a Miionaire? Ask the audience or phone a friend? Which do you think is better?

21 What's the capita of Spain? A: Barceona B: Madrid C: Vaencia D: Sevie

22 What's the height of Mt Kiimanjaro? A: 19,341 ft B: 23,341 ft C: 15,341 ft D: 21,341 ft

23 Go for the majority or trust an expert? It depends what kind of question you're asking In Who Wants to Be a Miionaire, peope tend to ask the audience fairy eary on, because once the questions get hard, they can't rey on the audience getting it right What's the height of Mt Kiimanjaro? A: 19,341 ft B: 23,341 ft C: 15,341 ft D: 21,341 ft What's the capita of Spain? A: Barceona B: Madrid C: Vaencia D: Sevie

24 Why bother with opinion mining? It depends what kind of information you want Don't use opinion mining toos to hep you win money on quiz shows Recent research has shown that one knowedgeabe anayst is better than gathering genera pubic sentiment from ots of anaysts and taking the majority opinion But ony for some kinds of tasks

25 Whose opinion shoud you trust? Opinion mining gets difficut when the users are exposed to opinions from more than one anayst Intuitivey, one woud probaby trust the opinion supported by the majority. But some research shows that the user is better off trusting the most credibe anayst. Then the question becomes: who is the most credibe anayst? Notions of trust, authority and infuence are a reated to opinion mining

26 A opinions are not equa Opinion Mining needs to take into account how much infuence any singe opinion is worth This coud depend on a variety of factors, such as how much trust we have in a person's opinion, and even what sort of person they are Need to account for: experts vs non-experts Spammers frequent vs infrequent posters experts in one area may not be expert in another how frequenty do other peope agree?

27 Trust Recommenders Two types of trust: reationship (oca) trust and reputation (goba) trust. Reationship trust: if you and I both rate the same things, and our opinions on them match cosey, we have high reationship trust. This can be extended to a socia networking group --> web of trust. Reputation trust: if you've recommended the same thing as other peope, and usuay your recommendation is cose to what the majority of peope think, then you're considered to be more of an expert and have high reputation trust. We can extend reationship trust to form custers of interests and ikes/disikes We can narrow reputation trust to opinions about simiar topics

28 Reated (sub)topics: genera Opinion extraction: extract the piece of text which represents the opinion I just bought a new camera yesterday. It was a bit expensive, but the battery ife is very good. Sentiment cassification/orientation: extract the poarity of the opinion (e.g. positive, negative, neutra, or cassify on a numerica scae) negative: expensive positive: good battery ife Opinion summarisation: summarise the overa opinion about something price:negative, battery ife: positive --> overa 7/10

29 Feature-opinion association Feature-opinion association: given a text with target features and opinions extracted, decide which opinions comment on which features. The battery ife is good but not so keen on the picture quaity Target identification: which thing is the opinion referring to? Source identification: who is hoding the opinion? There may be attachment and co-reference issues The camera comes with a free case but I don't ike the coour much. Does this refer to the coour of the case or the camera?

30 Opinion spamming

31 Spam opinion detection (fake reviews) Sometimes peope get paid to post spam opinions supporting a product, organisation or even government An artice in the New York Times discussed one such company who gave big discounts to post a 5-star review about the product on Amazon Coud be either positive or negative opinions Generay, negative opinions are more damaging than positive ones

32 How to detect fake opinions? Review content: exica features, content and stye inconsistencies from the same user, or simiarities between different users Compex reationships between reviews, reviewers and products Pubicy avaiabe information about posters (time posted, posting frequency etc) Detecting inconsistencies, contradictions, ack of entaiment etc. is aso reevant here

33 Opinion mining and socia media Socia media provides a weath of information about a user's behaviour and interests: expicit: John ikes tennis, swimming and cassica music impicit: peope who ike skydiving tend to be big risk-taker associative: peope who buy Nike products aso tend to buy Appe products Whie information about individuas isn't usefu on its own, finding defined custers of interests and opinions is If many peope tak on socia media sites about fears in airine security, ife insurance companies might consider opportunities to se a new service This kind of predictive anaysis is a about understanding your potentia audience at a much deeper eve - this can ead to improved advertising techniques such as personaised ads to different groups

34 Socia networks can trigger new events Not ony can onine socia networks provide a snapshot of current or past situations, but they can actuay trigger chains of reactions and events Utimatey these events might ed to societa, poitica or administrative changes Since the Roya Wedding, Piates casses became incrediby popuar in the UK soey as a resut of socia media. Why? Pippa Middeton's bottom is the answer! Pictures of her bottom are aegedy worth more than those of her face!

35 Socia media and poitics Twitter provides rea-time feedback on poitica debates that's much faster than traditiona poing. Socia media chatter can gauge how a candidate's message is being received or even warn of a popuarity dive. Campaigns that cosey monitor the Twittersphere have a better fee of voter sentiment, aowing candidates to fine-tune their message for a particuar state: paying to your audience". But appying compex agorithms to socia media is far from perfect for predicting poitics, e.g. you can't detect sarcasm reiaby. Nevertheess, Twitter has payed a roe in inteigence gathering on uprisings around the word, showing accuracy at gauging poitica sentiment.

36 Tracking opinions over time Opinions can be extracted with a time stamp and/or a geoocation We can then anayse changes to opinions about the same entity/event over time, and other statistics We can aso measure the impact of an entity or event on the overa sentiment about an entity or another event, over the course of time (e.g. in poitics) Aso possibe to incorporate statistica (non-inguistic) techniques to investigate dynamics of opinions, e.g. find statistica correations between interest in certain topics or entities/events and number/impact/infuence of tweets etc.

37 Viewing opinion changes over time

38 Some opinion mining resources Sentiment exicons Sentiment-annotated corpora

39 Bing Liu s Engish Sentiment Lexicon Avaiabe from Engish.rar List of strings Positive words: 2006 a+, abound, abounds, abundance, abundant Negative words: 4783 Exampes: 2-faced, abnorma, aboish, abominabe Usefu properties: incudes mis-speings, morphoogica variants, sang

40 The MPQA Subjectivity Lexicon GPL License; downoad from type - either strongsubj or weaksubj en - ength of the cue in words word1 - token or stem of the cue pos1 - part of speech of the cue, may be anypos (any part of speech) stemmed1 - y (yes) or n (no) priorpoarity - positive, negative, both, neutra

41 LIWC Linguistic Inquiry and Word Counts (LIWC) a database of categorized reguar expressions Costs around $90 Some sentiment-reevant categories with exampe reguar expressions beow, from

42 Try some different exicons Get sentiment scores for singe words from a variety of sentiment exicons Show how a variety of exicons score nove texts Spend a bit of time experimenting with these to see the difference

43 Probems with Using Sentiment Lexicons Sentiment words are context-dependent and ambiguous a ong dress vs a ong wak vs a ong battery ife the camera was cheap vs the camera ooked cheap I ike her vs Peope ike her shoud be shot. Soutions invove domain-specific exicons exicons incuding context (see e.g. Schar's GWAP methods constraining POS categories And we aso have to ook at things in the context other than sentiment words (negatives, adverbs, sarcasm etc.)

44 Summary In this section, we have introduced the concept of sentiment anaysis Why is it important? What are the typica probems associated with it? What probems are incurred when working with socia media? How can these be resoved? Next, we ook at how to actuay do sentiment anaysis in GATE