Of CinemaScore and Critics: Shouldn’t They Always Agree?


This past weekend, Andrew Dominik’s Killing Them Softly opened in the box-office.  This movie is Dominik’s third full-length film, after The Assassination of Jesse James by the Coward Robert Ford, a film he also wrote and directed.  Killing Them Softly opened to a good box office and positive critical praise, but another metric seems to indicate that the movie is actually a stinker.  The metric is the somewhat mysterious CinemaScore.

CinemaScore is a marketing-research firm based out of Las Vegas.  At any major release, their representatives show up and ask moviegoers questions in order to get an idea of how the average consumer will enjoy the movie.  These representatives will show up at 25 major cities and poll audience members just leaving the theater.  Sounds harmless and innocent enough, right?  It is.  If there’s any factor that press releases pay attention to though, it’s the final “what would you rate this movie?” question in which participants must rate the movie they just saw on an ordinal alphabetical scale of A+ to F.  Most movies score around a B, with a few venturing south toward C- territory, it’s pretty rare though, that a movie would see an F.  Killing Them Softly earned an F this weekend according to CinemaScore.

Most of the movies that received A+ ratings should come as no surprise: TitanicArgoThe King’s SpeechThe AvengersToy Story 2, etc…  Since the company started publishing its findings, there have only been a handful of F’s earned — 7 films actually, after Killing Them Softly‘s release.  According to CinemaScore an F means that the participant believed that the movie wasn’t only bad, but it should have never been released.  Here’s the list of the others F-rated movies with their current Rotten Tomatoes score (critic and audience):

  • Solaris (2002): a remake of the 1972 Russian film starring George Clooney and directed by Stephen Soderberg.  (65% and 54%)
  • Darkness (2002): the Anna Paquin vehicle directed by the same guy who made the fantastic REC.  (4% and 36%)
  • Wolf Creek (2005): a brutal thriller based on “true events” directed by Greg McLean. (53% and 49%)
  • Bug (2006): a thriller starring Ashley Judd and Michael Shannon, directed by William Friedkin (of The Exorcist and The French Connection).  (61% and 32%)
  • The Box (2009):  a thriller starring Cameron Diaz and James Marsden, directed by Richard Kelly (of Donny Darko).  (45% and 27%)
  • The Devil Inside (2012): a mockumentary, found-footage movie about a series of exorcisms.  (7% and 22%)
  • Silent House (2012): a thriller that takes place in real-time through one continuous shot starring Elizabeth Olsen.  (41% and 30%)

Looking at these numbers, I realize that Rotten Tomatoes may not be the best metric to compare CinemaScore to.  After all, Rotten Tomatoes is just an average bernoulli distribution — did you enjoy this movie? — with no real way to look at nuance.  For example, if Roger Ebert assigned a movie 2 out of 4 stars, and Peter Travers assigned the same movie a 0 out of 4 stars, Rotten Tomatoes would give the movie a 0 rating because, on average, the critics did not enjoy the movie.  Metacritic, on the other hand, attempts to quantify how much a critic enjoys a movie, usually by converting an assigned star-rating into a percentage.  For example, if Ebert gave a movie 3 out of 4 stars, it would warrant a 75% on Metacritic.  Here is the same list with Metacritic scores:

  • Solaris (2002): 65
  • Darkness (2002): 15
  • Wolf Creek (2005): 54
  • Bug (2006): 62
  • The Box (2009):  47
  • The Devil Inside (2012): 18
  • Silent House (2012): 49

So these numbers don’t vary too much.  In this list, it looks like there are some clear stinkers, and some movies that aren’t so bad.  Darkness and The Devil Inside probably “deserved” their F ratings.  According to Rotten Tomatoes and Metacritic, these two movies aren’t too good.  What about the others though?  Why would they receive an F?  Is it possible that the critics, whose job it is to determine whether or not movies are worth watching, dropped the ball?  Critics are often, well, criticized for being “out of touch” with the modern filmgoer — a notion that Michael Bay, M. Night Shyamalan, and Kevin Smith certainly seem to agree.  If film critics on average rate a movie a B or C, and public audiences rate it an F, does that mean the critics have failed?

CinemaScore ballot

Courtesy of CinemaScore.com

CinemaScore has been a mostly reliable indicator of how audiences will enjoy the movie on opening weekend.  Presumably, this is all done before the audience has heard much critical opinion toward the movie, and they are attending based solely on their thoughts of the premise of the movie, commercials, or cast listing.  In this light, it seems to make sense that Solaris and The Box and Killing Them Softly earned F’s: they feature A-list celebrities in movies that they aren’t normally associated with.  For Solaris, were people coming to theaters to see George Clooney in an action-packed space thriller?  For The Box, were people coming to see Cameron Diaz in a horror movie?  For Killing Them Softly, were people coming to see Brad Pitt as a gangster?  It’s quite possible that the F’s received by these movies are because folks showed up to the wrong movie, and not necessarily because they were bad movies.

The other three movies, BugWolf Creek, and Silent House are all horror movies that use unconventional methods in approaching the genre.  Again, it’s completely possible that audiences that showed up to these movies were under the impression they were in for jump-out-from-behind-the-corner scares.  Would these moviegoers be disappointed that Silent House was a psychological thriller that was all one continuous take?  Would they be prepared for the gruesome action in Wolf Creek?  Would they be disappointed that the titular bugs were just people losing their minds?

What appears to be disparate opinions between film critics and the moviegoing public might actually just be a product of circumstance.  Because CinemaScore is a snapshot of audiences on opening weekend, it doesn’t necessarily reflect the way that audiences will perceive it next week, or eventually on DVD.  Taking a look at Rotten Tomatoes, it does appear that the critics “get it right” in comparison to CinemaScore, assuming that an F on CinemaScore is equivalent to a 10% (or lower) on Rotten Tomatoes.  So what does that mean for CinemaScore?  Is it unreliable?  I will let you decide for yourself.  Here’s an exhaustive list of the movies the marketing-research firm has identified as A+ films, most that have been vindicated with Oscar nods and wins:

  • Driving Miss Daisy (1989)
  • A Few Good Men (1992)
  • The Fugitive (1993)
  • Schindler’s List (1993)
  • The Lion King (1994)
  • Titanic (1997)
  • Toy Story 2 (1999)
  • Harry Potter and the Chamber of Secrets (2002)
  • Antwone Fisher (2002)
  • Up (2009)
  • The Blind Side (2009)
  • Tangled (2010)
  • The King’s Speech (2010)
  • The Help (2011)
  • Soul Surfer (2011)
Tagged , , , , , , , , , , , , , , , , , , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: