Lies, Damned Lies, and Statistics

Welcome back to Buffering, coming to you a day later than usual thanks to Monday’s celebration of our nation’s non-treasonous presidents. If you’re a regular reader of this space, odds are your media diet also includes several other Hollywood-adjacent newsletters, like Sean McNulty’s The Wakeup(part of The Ankler), Claire Atkinson’s Media Mix, and Ryan Faughnder’s Wide Shot for the Los Angeles Times, among many others. There are many excellent options! But there’s also no doubt that one of the buzziest industry columns comes from the keyboard of former THR boss Matt Belloni, one of the founders of newsletter start-up Puck. I’ve long thought of him — and this is a compliment!— as a kinder, gentler, more rational and far less ethically-challenged successor to the late Nikki Finke, a former New York writer who went on to terrorize Hollywood during the aughts. My colleague Nicholas Quah seems to agree, so he spent some time observing Belloni in his natural (apparently shoeless) habitat in Los Angeles, and you can read the result here. As for the newsletter you’re eyeballing right now, our topic is the debate over Nielsen’s streaming ratings, and what they do — and don’t — suggest about the kinds of programs streamers should be making. Thanks for reading. –Joe Adalian

Reach Joe on Twitter @tvmojoe or email buffering@vulture.com.


Enjoying Buffering? Share this email with your friends, and click here to read previous editions.

Earlier this week, politics website The Hill republished a story from one of its sister companies with a simple (but very clickable) headline: “The top streamed shows are almost all old. Why?” The article—a recap of Nielsen’s January snapshot of streaming ratings for 2023 revealing that Suits racked up more minutes of viewing than any other single series last year—was actually pretty straightforward and non-alarmist. The online response, however, was anything but, with ordinary viewers, journalists and Hollywood writers loudly trumpeting the data as proof audiences have rejected the streaming model of short-run series which wrap up after a few years.

Just one problem: It does no such thing.

To be sure, numbers from both Nielsen and Netflix bolster the idea that there’s value in streamers having shows with deep episode libraries, and that those same streamers should be greenlighting production of their own network-style programs rather than relying on stuff produced decades ago. (More on that later.) But while people spend a lot of time watching (and rewatching) older shows on streaming, it is wrong to look at the big numbers racked up by Suits, NCIS, or Grey’s Anatomy and conclude these shows are actually outperforming the biggest original titles on successful streamers such as Netflix and Max, or that short-run originals, as a category, are being rejected by audiences. It’s not that Nielsen’s raw numbers on consumption are faulty; they’re just misleading.

Let’s take a look at what Nielsen’s data says about Suits. As was hyped around the internet last month, the former USA Network original racked up just shy of 58 billion minutes of viewing time last year — more than any show, old or new, according to the ratings company. By contrast, another Netflix show — The Lincoln Lawyer — tallied just under 12 billion minutes of viewing. That was good enough to make it the tenth most popular streaming original of 2023, and the No. 20 series overall. So it seems pretty simple, right? Suits was a bigger hit last year than Lincoln Lawyer, with an audience nearly five times as big as Suits.

Except, no. Sure, according to Nielsen, Americans as a group collectively spent more time watching the TV show Suits than they did Lincoln Lawyer. It occupied a lot of our collective attention, and to be clear, as library shows go, it was a legit monster hit which produced a real pop culture moment. But unlike its TV sample, Nielsen’s publicly released streaming data is not set up to allow for direct apples-to-apples comparison between most shows, particularly when it comes to new titles versus old ones. That 58 billion number for Suits is based on viewing of 140 roughly 45-minute-long episodes, both on Netflix and Peacock. By contrast, Lincoln Lawyer racked up its 12 billion minutes with just 20 episodes. I’m not going to go all A Beautiful Mind on you working out the math, but if you look at Nielsen’s numbers on a per-episode basis, Suits averaged 414 million minutes of viewing for every show in its library, while Lincoln Lawyer generated about 600 million per episode. I’m not even considering the fact that some of Suits’s numbers include viewership on Peacock (no doubt a fraction of the overall total but also probably not insignificant) or that the vast majority of Lincoln’s 2023 tally was generated on the strength of the 10 new episodes of season two which debuted in August (whereas every episode of Suits was “new” to Netflix this summer).

Even when you’re comparing two library shows, Nielsen’s streaming data still lends itself to easy misinterpretation. For instance, the company’s 2023 list of acquired (a.k.a. library) titles has Friends ranked No. 8 with 25 billion minutes streamed on Max while NCIS is No. 3 with 39.4 billion via Netflix and Paramount+. If you just take a quick glance at a top 10 chart, you’d think NCIS is the better performer. But the naval crime drama is an hourlong procedural with a whopping 443 episodes in its catalog (or a little bit less than 20,000 available minutes of TV, per my rough math), while the one with all the goofy Gen X icons is a half-hour comedy with 243 episodes (or about 5,400 minutes of potential viewing). So even though both shows are streaming hits and very valuable parts of their respective platforms, when you adjust for library size and program length, the average episode of Friends gets more than twice the streaming time as NCIS.

Part of what makes the narrative around Nielsen’s streaming ratings so problematic is the fact that folks (reporters, entertainment industry folks, and pop culture fans on social media) are used to looking at Nielsen’s far more established linear TV ratings, where things are a lot simpler. Instead of messing with minutes consumed, or measuring up all the viewership of every episode that might air over the course of a season or calendar year, Nielsen’s public-facing household survey focuses on both the average number of viewers per episode of a series, as well as a show’s average audience over the course of a season. And that’s true whether we’re talking about a current series still airing once a week in primetime or a library show which can be seen on cable multiple times a day.

In Nielsen’s primetime rankings for last week, for example, a single telecast of CBS’s Young Sheldon is listedas the most-watched entertainment series of the week, with 8 million same-day viewers. But during that same week, cable’s TBS aired Sheldon predecessor The Big Bang Theory dozens of times, including 14 times on February 13 alone. If you added up the audience which checked out all of those Big Bang episodes throughout the seven-day frame — even just the ones which aired in primetime — the total number would easily pass the 8 million that Sheldon got on CBS. In other words, Americans spent more time watching Big Bang on TBS last week than they did Young Sheldon on CBS. But not a single soul who works in television would argue that Big Bang was the bigger show last week because comparing the audience for a single network telecast with dozens of cable telecasts just isn’t a thing, and never has been.

What has long been a thing, even before the streaming era, is the reality that a few big library shows airing on cable or local TV stations often generate larger overall audience consumption throughout the course of a week than many big network shows of the day. To demonstrate this, I looked up the ratings for a random Wednesday in February of 2015. On that night, Fox’s red-hot Empire drew 13 million same-day viewers for Fox. It was a monster hit! But on that same evening, TBS aired six episodes of the Big Bang Theory, and the cumulative viewership for those episodes was over 15 million. A decade earlier, circa 2005, I probably would have found something similar by comparing ratings for reruns of Seinfeld to those of whatever was airing in primetime back then.

This is not to argue the data showing big consumption of library shows is meaningless — not at all. Netflix wouldn’t pay hundreds of millions of dollars to get the rights to a show like Seinfeld if there wasn’t value. Audiences absolutely like watching their favorite shows again and again — but this was true even when the new shows on TV were also 22-episode sitcoms and procedurals. Back in the 1980s, ‘90s, and ‘00s, cable networks and local TV stations used to pay crazy sums of cash for reruns of Friends or Home Improvement. Studios like Sony and Warner Bros. TV would literally make billions in profit selling those older shows, even as the Big Four kept churning out more traditional TV shows.

While it’s frustrating to see streaming ratings misinterpreted or even weaponized, it’s also entirely understandable why this is happening. Except for Netflix, streamers are super protective of their internal data and the many various metrics that actually get used to determine popularity and success in 2024. If they were more transparent and open about the incredibly granular data they have, there’d be a lot less confusion over what gets watched and how. And while Nielsen has tried to step up to fill the info gap, the company is a business whose clients are networks, advertisers, studios, and streamers; they save their best data for paying customers. Plus, it’s not exactly an easy task to figure out the perfect metric for streaming ratings, or one which closely resembles the decades-old linear numbers.

After all, in the digital TV universe, there are no timeslots or schedules to measure. On-demand streaming and linear TV are two entirely different ecosystems and business models. It’s why Nielsen decided total time spent viewing during the course of a week would be the simplest way to report on performance of streaming titles, and there’s definitely some logic to that. The tricky part, however, has been how the company reports and categorizes that consumption data: It only publicly reports on total minutes consumed for the full library of a title’s available episodes, vs. individual seasons or episodes; it doesn’t adjust for runtime or series length; and it releases data for library shows and original series in the same weekly release. Unfortunately, that has resulted in a less-than-nuanced picture of what’s actually popular.

Netflix’s own system for reporting data, while also far from perfect and plagued by its own biases and limitations, actually offers a better representation of what’s connecting with audiences. The streamer now uses time spent watching a title divided by total runtime — what it calls “views” — to even out the differences between shorter and longer titles. It also breaks out titles by season so that shows which can take weeks to binge don’t get an edge over a new show whose eight-episode run can be finished in a couple days. Even with these adjustments, older library titles still end up in the streamer’s top 10, as season one of Fox’s House and Syfy’s Resident Alien did last week globally (and season six of Young Sheldon did in the U.S.). But they usually rank below Netflix’s own originals, which I think fits in with the theory that most people who pay $15 or $20 a month for Netflix aren’t doing so just for older shows that they’ve probably seen before. (Tubi and Pluto do a much better job of that anyway, and they’re free.)

This brings us back to something I mentioned at the start of this story. While big numbers for shows like Suits are not evidence that audiences have rejected the idea of short-run series or shows that feel like movies, the people who complain that streamers should not be abandoning so many of the hallmarks of network TV are absolutely correct. Given programming budgets that stretch into the billions, it borders on malpractice that streamers aren’t greenlighting more 20-episode series that run for five or six years. Indeed, I actually wrote an entire story about this last summer in the wake of the summer of Suits. You can read it here, but one of my key points was that streamers like Netflix, which have been able to rely on a healthy broadcast/cable ecosystem to feed them a steady supply of mega-bingeable library series, have so succeeded in diminishing their linear competitors that the pipeline for such shows is starting to dry up:

It made some sense during the first decade of streaming for platforms to rely almost exclusively on buzzy, star-packed shows with huge budgets because they were great tools to get people to commit to a new type of TV and the idea of paying for a single service. Early on, everyone in streaming sort of wanted to be the new HBO, and they made a lot of HBO-like shows. But streaming is more mature, those shows cost a lot of money, and when everyone else is also making “prestige” shows, it’s a lot harder for your prestige play to stand out. Plus, streamers need a variety of formats and genres: If they’re going to replace the cable bundle, they need to replace all of it. They’ve done a good job developing premium cable shows, and more recently, we’ve seen a surge in the kinds of slightly elevated dramas you’d once find on TNT or USA, like The Night Agent or Reacher. Netflix and Peacock have done a stellar job launching unscripted franchises, from Love Is Blind to The Traitors.

But streamers should also figure out a way to make the kinds of shows which worked on broadcast TV for decades: Broad appeal, lots of episodes, and perhaps most importantly these days, modest budgets. Hulu, Paramount+, and Peacock do at least get some network-style shows from corporate siblings ABC, CBS, and NBC. Unfortunately, those networks have all had their scripted content budgets slashed in the past five years and have little room to develop new shows that aren’t part of existing franchises. Meanwhile, Netflix, Prime Video and, Max have so far shown little interest in figuring out how to make scripted shows which can quickly amass a hundred or more episodes — even though the one thing Nielsen’s streaming ratings do prove is that audiences absolutely will watch more episodes of a show they love if they actually have more episodes to watch.