Senti-meter Scans the Twitterverse for Movie Sentiment: Oscar or Runner Up?

The Academy Awards are next Sunday, their 84th iteration, social-media aware as never before. The Academy invites you to “Join the Conversation” via the #Oscars and #CelebrateTheMovies Twitter hashtags and gives over major Web-page real estate to Oscar Blogs and to Twitter-derived Oscar Buzz. But Twitter’s an open platform, and tweets about movies are fair game for anyone to analyze. IBM, USC Annenberg School for Communications and Journalism, and the Los Angeles Times have given it a shot in the form for the Oscar Senti-meter.

The Oscar Senti-meter rates tweets that cite Academy Awards nominees in the Best Actress, Best Actor, and Best Picture categories and scores them on a more-positive/more-negative scale.

Oscar Senti-meter screenshotWhat do I like?

  1. The timeline control! Move left (earlier) or right (later) to see tweet volume and sentiment scoring for different dates.
  2. The listing of selected external events as part of the time control. Some of are quite relevant, for instance, the January 15 Golden Globe awards. Scroll to that date to see the effect on stats for Meryl Streep, for instance.
  3. There are at least some linguistic smarts built into the system (although it’s impossible to determine, by looking, how much). “Merryl Streep” and “dragon tattoo” are resolved to the correct, full names.

And what’s lacking, items for the To Do list for next go-around?

  1. Provide click-through to underlying tweets, and not just three, all of them or at least a large sample. The current tool is a tease, and it’s not as if the tweets are private or proprietary. But click-through isn’t, alone, quite enough…
  2. Show the sentiment classification of the underlying tweets. The LA Times tells us only that “the Senti-meter combs through a high volume of tweets daily and uses language-recognition technology… to gauge positive, negative and neutral opinions shared in the messages.” I want to see for myself how good the technology is.
  3. Animate the timeline control. A simple play button would march the display through the range of dates.
  4. Let me trace opinion across dates. All that’s needed is a trail attached to what’s now a bouncing dot although to be done effectively, it would be best to focus on a particular nominee and replace the current horizontal axis with a time axis.
  5. Let me filter tweets — for instance, to exclude tweets with the #GoldenGlobe hashtag — and dynamically recalculate based on my filtering.
  6. Let me explore how sentiment about Actors, Actresses, and Movies is linked. It would be interesting, for instance, to see if and how tweet volume and sentiment about Meryl Streep and the Iron Lady are linked.
  7. I want Sorting, Ranking, and Thresholds. Let me sort nominees, ascending and descending, within the Actors, Actresses, and Movies categories, by number of tweets in a given day and over all days. Let me restrict nominees to the Top N or Bottom M based on number of tweets and on positive or negative sentiment. Let me apply a threshold, for instance, show only nominees who have been mentioned in at least 100 tweets.
  8. Content enrichment. I’ve never heard of the movie Carnage, nor of actor Joseph Gordon-Levitt. How about a hyperlink, behind each name, for more information?
  9. Explain how the system works!

Frankly, I suspect that Senti-meter is classifying tweets at the message rather than at the feature level, that is, for individual Actresses, Actors, and Movies. I’d be happy to learn otherwise, but there’s no indication about method on the Senti-meter site.

I have one, last ask:

  1. Find a way to make Senti-meter useful, more than eye candy.

My last point is seemingly a tough one. The best way to make any information-delivery site or utility more useful is by designing it, from the start, to respond to business needs. If you don’t, the majority reaction will be a yawn, So What? (and I don’t mean the Miles Davis version), a few moments on-page dwell time and then the site visitor is on to the next thing. Is there a business need met by the Oscar Senti-meter? There could be, for instance, in predicting box-office and rental demand. But of course we assume that the Oscar-scoping interface is just a demonstration of the possibilities afforded by the technology and the implementation.

The types of interactivity I’ve described would boost Senti-meter’s business usefulness. So would provision of goal-aligned, beyond-polarity sentiment classification. A plug: We’ll discuss these topics and much more at the up-coming Sentiment Analysis Symposium, a conference I organize, May 7-8 in New York. I do come back to these topics frequently. I wrote about them in a recent review of Neurolingo’s beta Sentibet system. Sports betting, Oscar sentiment: Two peas in a pod. Sentibet differentiates Predictions, Feelings, and Wishes. I may think that Tilda Swinton is a great actress but hope that Michelle Williams wins the Oscar and expect Viola Davis to take the prize this year. Surely this granular, focused level of analysis isn’t beyond IBM’s capabilities, USC Annenberg’s research interests, or the L.A. Times‘s information-delivery mandate.

In sum, my review scores the Oscar Senti-meter moderately negative, with lacking outpolling like by 10 to 3. The IBM-USC-L.A. Times system is a worthy nominee, but not a winner. The players have the tools and resources to create something great, both technically strong and usable for a mass audience. I’m looking forward to the sequel.

2 thoughts on “Senti-meter Scans the Twitterverse for Movie Sentiment: Oscar or Runner Up?

  1. Seth, since I am a bit of a fan of this senti-meter app (I wasn’t personally involved so can’t claim any credit), I couldn’t resist responding to your post…

    I can’t disagree with most of what you said and in fact I feel you’ve drafted a really impressive set of requirements which I would also love to see in future releases of senti-meter. However, I did want to share my reasons for recently tweeting you that senti-meter was the “sexiest sentiment visualization ever”.

    These days you can’t throw a stone but you hit a sentiment app. They are everywhere, and for the most part they are utterly dull and unengaging. If I have to look at another pie chart I am going to be ill 🙁 What struck me novel about this approach was exactly their emphasis on creating an interaction experience. Combining dragging, clicking, and hovering, across every inch of available real-estate, to squeeze as much as possible into one visualization without making it over-complicated. I believe slick navigation is more important than us analytics folks often appreciate.

    Does senti-meter ultimately need to expose more of the underlying analytics? For sure! Does it need to extend with more advanced analytics and provide new visualizations to facilitate those navigation experiences? Absolutely! But I really hope that the next release doesn’t loose sight of the aesthetics and the navigation experience, because that’s what truly differentiates it.

    Even us analysts need a bit of bling 🙂

  2. Very interesting. I’m actually the project manager for this at USC. These are some excellent ideas about making this more visually engaging. When we embarked on this, we really didn’t know what kind of data we’d get, or how much. That made it a little difficult for us to conceive of of the functionality you describe. Also, making all tweets available, Straight, No Chaser (http://www.youtube.com/watch?v=bAJj2VdOfBQ), can put you in some sticky situations about what’s appropriate. People say some nasty things about Tilda Swinton that you don’t necessarily want to print in a family newspaper.
    You’re last point — how to make it more useful — is a good one. Think of where this kind of project is heading and how it could really meet business needs. Studios spend millions on marketing. but they don’t know how well it’s working until they see the box-office numbers. Connecting twitter sentiment on movies with studios’ marketing strategies opens up a lot of possibilities….

Leave a Reply to Anonymous Cancel reply