The Numbers Are All Fake

People have been gaming RottenTomatoes the same way they've been gaming everything else.

As promised, here’s another free column. If you want to see more like this, consider becoming a paid subscriber. Failing that, share it with a friend! The more paid subscribers I get, the more I can focus on #Content full time. All #Content, all the time! More #Content faster! And if you already ARE a paid subscriber, know that I appreciate you from the bottom of my heart. I hereby grant you seven years of good luck and let it be known that you have very attractive genitals.

AI Generated

This past week, in a big piece on Vulture, it came out that a PR firm has essentially been artificially juicing RottenTomatoes scores by paying critics to write positive reviews for the movies the firm represented. The example cited was a 2022 movie called Ophelia, starring Daisy Ridley, which started out with a score of 46% recommended based on 13 reviews. Then a PR company called Bunker 15 got to work, paying some friendly critics to see it, and lobbying some others to flip their negative-listed reviews to positive. And voila, the score jumped from 46% to 62%, changing the rotting tomato icon to a fresh one.

This was just one anecdote in a much longer piece about the (perceived) power of RottenTomatoes and all the ways it gets gamed, misunderstood, and corrupted. It served as confirmation of a lot of things I’ve been writing about here — like trying to understand why an Inside Out review had almost gotten me fired, or why so many films seemed to have a RottenTomatoes-based marketing plans already built out before I’d even gotten my press screening invite.

The piece is full of gems from movie people admitting how important that little number had become to them, seemingly beyond all reason.

Where the trickery often begins — is that Rotten Tomatoes scores are posted after a movie receives only a handful of reviews, sometimes as few as five, even if those reviews may be an unrepresentative sample. This is sort of like a cable-news network declaring an Election Night winner after a single county reports its results. But studios see it as a feature, since, with a little elbow grease, they can sometimes fool people into believing a movie is better than it is.

I said this exact thing* a few weeks ago, without the comprehensive numbers to back it up, so it was nice to have confirmed.

Publicist No. 1 recalls working on a 2022 title that premiered to acclaim at a festival a few months before its release: “I wanted to screen it more widely, but the movie had a 100 and the studio didn’t want to damage that because they wanted to use the ‘100 percent’ graphic in their marketing. I said, ‘Why don’t we get a couple more reviews?,’ and they were like, ‘We just want the 100.’ ” The film** won an Oscar.

Again, this squares with my experience. Why was that one negative review I wrote such a big deal? Because before I wrote it, that film had a 100% next to it. In making the number go down, I likely ruined someone’s marketing plan. And those marketing plans buy ads. And those ads, unfortunately, end up paying a lot of the people who write the reviews. You can see how this would be a conflict of interest.

In some ways, getting $50 bucks from Bunker 15 to review some stinker no one has heard of seems much more straightforward.

An indie-distribution executive says, “I put in our original business plan that we should not do films that score less than 80%. Rotten Tomatoes is the only public stamp of approval that says, ‘This is of immense quality, and all critics agree.’”

Wow, a distribution executive saying “we should only distribute films that most people like.” What a genius. Give this person a raise.

Moreover, I’ve written at length about the impossibility of even two or three critics agreeing, let alone hundreds, but this is the central illusion RottenTomatoes, or any awards show, is selling — the myth of critical consensus. Furthermore, with RottenTomatoes, an 80% “score” just means that, at minimum, eight out of ten critics thought a movie was barely passable. It doesn’t mean they thought it was good, that it was a must-see, that you’ll still be thinking about it in a week, they just thought it was a hair better than terrible. Nonetheless, 80 is a number, and when you’re an executive, numbers are sexy.

Thus…

Studios are so scared of what the Tomatometer might say that some work with a company called Screen Engine/ASI, which attempts to forecast scores. (“According to the studios, the predictions are very close,” says another publicist).

That’s right, predicting RottenTomatoes scores. What an incredible grift. It’s funny to me, in a sick sort of way, how valuable an opinion becomes if you just repackage it as a prediction. Normal reviews are worth so little that Bunker 15 was finding people to do them for $50. Meanwhile, this company just puts “engine” in the title and claims they have the special sauce and the wallets just open for them. Business magic.

In any case, it didn’t take many steps before everyone involved in this process apparently just sort of forgot what these numbers were actually supposed to be for. Does any of the numbers gaming actually help put films on moviegoers’ radars or improve their box office chances? Not really, and we know this intuitively, because for all of the shenanigans, I’d still never heard of Ophelia before this article.

The article basically spells out as much:

Attempts to evince a relationship between movies’ Tomatometer scores and their financial success have yielded conflicting results. A 2017 study by the director of USC’s Data & Analytics Project concluded that “Rotten Tomatoes scores have never played a very big role in driving box office performance, either positively or negatively.” In 2020, an investigation by the Ringer found that Tomatometer scores do correlate with box-office returns, especially for comedies and horror films, but the authors admit that the pandemic may have scrambled moviegoing habits in ways that data may not fully account for yet.

Does the RottenTomatoes score actually matter to a movie’s success? Probably not. Maybe so. Who can say, really??

That number matters to the executive regardless, not necessarily for the behavior it was meant to illustrate, but because they’ve developed an emotional attachment to the number itself. And companies like Bunker 15 and Screen Engine/ASI are heavily invested in making you forget what the next steps were supposed to be. Versions of these companies exist in every industry. You like number, right? We make number go up.

Reading all of this was heartening and horrifying in equal measure, as it always is to realize you weren’t just imagining some phenomenon that sucks. But I also think it speaks to something much bigger than movies. We’ve gamified and fetishized numbers to the point that we no longer recognize their connection to the things they were designed to measure.

—

Elsewhere, Robert Silverman had a scoop at The Daily Beast this week, detailing how for years, Barstool Sports has been operating a network of burner accounts, allowing them to launder copyrighted material on their official channels without getting the channels banned. That practice — essentially putting some NFL footage or whatever on a burner Twitter or TikTok account, then reposting the burner on your official account — is probably nothing new or surprising to anyone who pays attention to the internet.

What was mostly of interest to me was the way Barstool has apparently been selling themselves to advertisers.

In a recent New York Post interview, a Barstool spokesperson claimed they reach one-third of all 18-34-year-olds in the country. The vast majority of that engagement, going by Barstool’s own metrics, comes from social media. That’s not how most media companies calculate their audience size, but for Barstool it’s essential.

So how do they calculate their audience size?

In SEC documents filed when Penn acquired a minority share in 2020, Barstool claimed to have 66 million “monthly unique visitors.”

To come up with that figure, subscribers and followers on Barstool’s social media are lumped together with podcast listeners, merchandise purchasers, and website readers. (Social makes up a significant majority.)

This raises the obvious question, if your numbers are huge largely because you’re the first to post copyrighted sports clips to social media, how valuable are those numbers?

By 2022, the number had jumped to 153 million people “who follow us and engage with us” each month, [Barstool CEO] Ayers Badan claimed. In July, a Barstool spokesperson claimed they hit 220 million. Now, it’s reportedly 250 million.

Say it with me: the numbers are all fake.

It’s hard to even begrudge Barstool for this. If people want to buy fake numbers, the most logical business solution is to produce some fake numbers to sell them.

—

Comcast and Disney are currently trying to sell Hulu, which the CEO of Comcast claims is worth at least $30 billion dollars. Meanwhile, the main demands of SAG and AFTRA, the writers and actors unions that have been striking against these companies for more than 100 days now, are that they be compensated based on some meaningful metric of how often their content is viewed (which, it may not shock you to learn, other countries already have).

The main sticking point is that the streamers… well, they don’t want to do that. And why don’t they want to do that? Because then they’d have to admit something that we all basically already know:

The numbers are all fake.

—

I’d like to believe that there’s a reckoning coming; that we’re all about to collectively claw back some small piece of the collective human experience from the hands of Big Data. In between eagerly checking my fantasy football scores and the metrics on one of my dumb tweets I recognize that that’s unlikely. And yet I nonetheless choose to find liberation in the fakeness of fake numbers. Only once we acknowledge that everyone is just kind of making shit up as they go can we be truly free to just make shit up as we go.

—

*Quoting myself:

“One funny thing that would happen to me a lot was that I’d get my press screening invitation for some big movie, and then a day before the screening or the day of, I’d get an email blast from a PR firm about how that movie was “certified fresh at 92% on RottenTomatoes!” or whatever. It was pretty rich addressing this email touting an aggregation “score” to me, a Tomatometer critic (I would never use this as a brag or proof of credentials, but it does seem relevant here) who hadn’t had a chance to review the film yet. It’d be like Wolf Blitzer telling you Biden’s Electoral Vote count before you’ve even received your ballot. This happened quite a bit, clearly being snubbed for this first round of reviews, with a marketing campaign clearly built around it.”

**So, a 2022 film that won an Oscar, which premiered at a film festival “a few months before,” that had a 100% recommended rating at some point. Anyone interested in trying to figure this out?

***Tooting my own horn a little bit again here, I’d like the record to show that I pointed out how weird it was that FTX, which was officially illegal to use in the US, had a supposedly “completely separate” trading platform not owned by FTX called FTX.us. FTX.us’s publicly-listed founder and CEO? You guessed it, Sam Bankman-Fried, the same guy as FTX. FTX filed for bankruptcy nine months later. Not that I think it took any special powers of perception to know that the whole thing was a fraud.