Predicting the Future With Social Media

Transcription

1 Predicting the Future With Social Media Sitaram Asur Social Computing Lab The Social Computing Lab focuses on methods for harvesting the collective intelligence of groups of people in order to realize greater value from the interaction between users and information. Bernardo A. Huberman Published on arxiv Cornell University March Maurizio Napolitano, SoNet group, - April 2010

2 SoNet Research Meetings These slides were used for an internal presentation of the SoNet group. Every week, one member of the SoNet group presents a research papers to the other members. The mentioned paper(s) are hence written by other researchers. Being internal presentations, these slides might be a bit rough and unpolished. You can find more information (including this presentation) about the SoNet group at

3 The question How social media content can be used to predict real-world outcomes? The case study: predicting box-office revenues for movies using the chatter from Twitter Why Twitter? several tens of millions of users who actively participate in the creation and propagation of content Why movies? The topic of movies is of considerable interest among the social media user community The real-world outcomes can be easily observed from box-office revenue for movies

4 Topics Viral marketing How buzz and attention is created for different movies How buzz and attention changes over time movies that are well talked about will be well-watched? Sentiments How are created How positive and negative opinions propagate How they influence people

5 What discovery Social media feeds can be effective indicators of real-world performance The rate at which movie tweets are generated can be used to build a powerful model for predicting movie box-office revenue. The predictions are better than those produced by the Hollywood Stock Exchange, the gold standard in the industry

6 The dataset TWITTER search API retweet 2.89 million tweets referring to 24 different movies period of 3 months (nov-feb) from 1.2 million users by using the movies keywords Armored ( ) Daybreakers ( ) Extraordinary Measures ( ) Leap Year ( ) Princess And The Fog ( ) Tooth Fairy ( ) Avatar ( ) Dear John ( ) From Paris With Love ( ) Legion ( ) Sherlock Holmes ( ) Transylmania ( ) The Blind Side ( ) Did You Hear About The Morgans ( ) The Imaginarium of Dr Parnassus ( ) Twilight: New moon ( ) Spy Next Door ( ) When in Rome ( ) The Book of Eli ( ) Edge of Darkness ( ) Invictus ( ) Pirate Radio ( ) The Crazies ( ) Youth in Revolt ( ) critical period = the time to the week before a release movie

7 Dataset charatecteristics Number of tweets per unique authors for different movies LIKE the box-office trends!!! y tweets x days lines movies

8 Dataset characteristics Number of tweets per unique authors for different movies ratio remains fairly consistent between 1 and 1.5 y tweets per authors x days lines movies

9 Dataset charatecteristics Log distribution of authors and tweets over the critical period POWER LAW Zipfian distribution A few authors generating a large number of tweets y log(frequency of authors) x log(number of tweets)

10 Dataset characteristics Distribution of total authors and the movies they comment on POWER LAW A majority of the authors talking about only a few movies y authors x number of movies

11 Attention and popularity Twitter and real world Prior to the release of a movie, media companies and and producers generate promotional information in the form of trailer videos, news, blogs and photos. We expect the tweets for movies before the time of their release to consist primarily of such promotional campaigns, geared to promote word-ofmouth cascades In Twitter: tweets and retweets referring a particular url (photos, trailer and other promotional material)

12 Attention and popularity Percentages of urls in tweets for different movies there is a greater percentage of tweets containing urls in the week prior to release than afterwards

13 Attention and popularity tweets with url VS retweets URLs and RETWEETs PERCENTAGES FOR CRITICAL WEEK Features Week 0 Week 1 Week 2 url retweet CORRELATION and COEFFICENT OF DETERMINATION (R 2 ) values for URLS and RETWEETs before release Features Correlation R 2 url retweet This result is quite surprising since we would expect promotional material to contribute significantly to a movie s box-office income

14 Prediction first weekend Box-office revenues Using the tweets referring to movies prior to their release, can we accurately predict the box-office revenue generated by the movie in its opening weekend? How use a quantifiable measure on the tweets? TWEETRATE number of tweets referring to a particular movie per hour Tweetrate mov = tweets mov Time hours the correlation of the average tweetrate with the box-office gross for the 24 movies considered showed a strong positive correlation, with a correlation coefficient value of 0.90

15 Prediction use the regression analisys! Prediction compared with the real box-office revenue information extracted from the Box Office Mojo website => POSITIVE RESULTS Regression analysis with: Time series values of the tweet rate for the 7 days before the release Thent released number of the theaters the movies were HSX Index Exchange the index of the Hollywood Stock

16 Prediction linear regression the results Features Adjusted R 2 p-value*** Avg Tweet-rate e-09 Tweet-rate timeseries e-09 Tweet-rate timeseries + thent e-12 HSX timeseries + thent e-10

17 Prediction Predicted vs Actual box office scores using tweet-rate and HSX predictors

18 Prediction Predicting prices Prediction of HSX end of opening weekend price Predictor Adjusted R 2 p-value*** HSX timeseries + thent e-10 Tweet-rate timeseries + thent e-11 The Hollywood Stock Exchange de-lists movie stocks after 4 weeks of release, which means that there is no timeseries available for movies after 4 weeks. In the case of tweets, people continue to discuss movies long after they are released Week-end Adjusted R 2 Jan Jan Jan Feb Coefficient of determination (R2) values using tweet-rate timeseries for different weekends

19 Sentiment Analysis investigate the importance of sentiments in predicting future outcomes For each tweet assign the label Positive, Negative or Neutral Clean data (no stop-words, removel url and userid, replace title, question, exclamations) Amazon Meccanical Turk (1000 workers) Use LingPipe DynamicLDClassifier Obtained an accuracy of 98% 1)Define two variables Positiveand NegativeTweets Subjectivity= Neutral Tweets Tweets with Positive Sentiment PNratio= Tweets with Negative Sentiment

20 Sentiment Analysis the subjectivity increases after release X Y movies subjectivity

21 Sentiment Analysis The positive and negative go in the same direction of the movies success X Y movies polarity

22 Sentiment Analysis regression analisys and polartiy (PNRatio) Predictor Adjusted R 2 p-value Avg Tweet-rate e-09 Avg Tweet-rate + thent a-09 Avg Tweet-rate + PNRatio e-12 Tweet-rate time series e-06 Tweet-rate timeseries + thent Tweet-rate timeseries + PNRatio e e-08 the sentiments do provide improvements, although they are not as important as the rate of tweets themselves

23 GENERAL PREDICTION MODEL FOR SOCIALMEDIA y= a A p P d D A : rate of attention seeking P : polarity of sentiments y= and reviews D : distribution parameter y denote the revenue to be predicted Є the error β values correspond to the regression coefficients

24 Bibliography D. M. Pennock, S. Lawrence, C. L. Giles, and F. A. Nielsen. The real power of artificial markets. Science, 291(5506): , Jan W. Zhang and S. Skiena. Improving movie gross prediction through news analysis. In Web Intelligence, pages , 2009.

25 These slides are released under Creative Commons Attribution-ShareAlike 2.5 You are free: to copy, distribute, display, and perform the work to make derivative works to make commercial use of the work Under the following conditions: Attribution. You must attribute the work in the manner specified by the author or licensor. Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. For any reuse or distribution, you must make clear to others the license terms of this work. Any of these conditions can be waived if you get permission from the copyright holder. Your fair use and other rights are in no way affected by the above. More info at