As the clock runs out on 2019, Netflix has provided its own “year-end” lists of the most popular releases on its platform in the United States across this past year.
Combining its film and TV offerings into a single list, released as a Twitter thread, the streaming giant announced that the No. 1 title was Murder Mystery, its Adam Sandler and Jennifer Aniston-led comedy about a couple caught up in a tangled murder investigation.
The title, which debuted in June, was hailed as having the biggest opening weekend ever for a Netflix original film, with 30,869,863 accounts watching it within its first three days. Those numbers suggest the movie would have been a sizable hit in theaters, as Fortune reported earlier this year, though how much of one is a complicated question given the tricky and uneven way in which Netflix measures viewership.
Indeed, the “popularity” of Netflix titles, as measured by Netflix, is a metric that should raise eyebrows and—minus the externally regulated measurement of viewing numbers, something the company has long refused to engage in—can be considered little more than a promotional bump by the streaming giant.
To wit, all but one of the 10 most popular titles ranked by Netflix were Netflix originals. The full list reads:
- Murder Mystery
- Stranger Things 3
- 6 Underground
- The Incredibles 2
- The Irishman
- The Witcher
- Triple Frontier
- Extremely Wicked, Shockingly Evil and Vile
- The Umbrella Academy
- The Highwaymen
Limited to movies, the list reads:
- Murder Mystery
- 6 Underground
- The Incredibles 2
- The Irishman
- Triple Frontier
- Extremely Wicked, Shocking Evil and Vile
- The Highwaymen
- Spider-Man: Into the Spider-Verse
- Ralph Breaks the Internet: Wreck-It Ralph 2
- Secret Obsession
And the series ranking was as follows:
- Stranger Things 3
- The Witcher
- The Umbrella Academy
- Dead to Me
- You: Season 2
- When They See Us
- Unbelievable
- Sex Education
- 13 Reasons Why: Season 3
- Raising Dion
Other lists in Netflix’s same thread broke out the top nonfiction series, comedy specials, documentaries, international releases, and kids and family series.
According to CNET, the company’s “popularity” rankings do not subscribe to Netflix’s typical model of counting a view as an account having watched 70% of a program’s entire runtime. Instead, these rankings hinge on a title having been watched for two minutes, a dubious metric given that Netflix promotes its originals aggressively and employs autoplay on many of them as “top recommendations” for what a given viewer should watch next. According to a company spokesperson contacted by The Wall Street Journal, the programs were measured this way so as to put short-form and long-form programming on a more equal playing field, helping out offerings like three-and-a-half-hour crime epic The Irishman against more digestible series like the third season of Stranger Things, episodes of which are around an hour.
Additionally, no program that had been on Netflix before this past year, including evergreen-viewing sitcoms like Friends and The Office, was accounted for. Only the first four weeks of a program’s viewership is measured in this ranking, which likely accounts for the absence of slower-grown hits, like stand-up sketch series I Think You Should Leave with Tim Robinson, which only became a word-of-mouth sensation on social media months after its debut.
But given that four-week window, you may ask, how could 6 Underground (which arrived just over two weeks ago) or The Witcher (which premiered the Friday before Christmas) possibly be ranked so highly? How could You‘s second season have become such a hit within just a few days out? That’s because Netflix is also leaning on projections, anticipating future viewings, to dictate its rankings of the programs that haven’t yet been out for four weeks.
Even with all of these caveats disqualifying the rankings from being taken seriously, they raise an irritating number of questions. How is it possible that the final season of Orange is the New Black, released in July, had fewer viewings than the barely promoted first season of Raising Dion? If Triple Frontier led content chief Ted Sarandos to tell high-up executives at Netflix to cut back on spending on risky projects, as The Information reported earlier this year, what is there to be made of its ranking within the service’s most popular overall offerings? (Such curiosities would be a non-issue if the company didn’t keep its ratings hidden, but real transparency from Netflix is rare.)
Netflix’s ratings have always been subject to skepticism, given the uneven and unverified way in which they’re rolled out. More often than not, announcements of a particular statistic are employed to play up Netflix’s success in the original content space, as with the Murder Mystery ratings release or a comparable one for the post-apocalyptic thriller Bird Box last year. The goal is less to keep a record of its successes and failures and more to bang its own gong, hopefully convincing others in Hollywood of its continuing prosperity in the process.
As Netflix stands to lose the bulk of its licensed content within the coming years to competitors like Disney+, HBO Max, and Peacock, its success will hinge on its ability to continue making popular, conversation-dominating series itself—or at least on its ability to convince shareholders and Hollywood talent that it’s doing so.
Not for nothing—the non-Netflix titles that did make its ranked list are mostly Disney offerings, including The Incredibles 2 and Ralph Breaks the Internet: Wreck-It Ralph 2. All will be gone from Netflix by the end of 2020, suggesting that Netflix’s popularity contest next year could truly turn into an all Netflix affair.
More must-read stories from Fortune:
—Hasbro’s toy box is bigger than ever with $3.8 billion Entertainment One takeover
—What did and didn’t work at the box office in 2019
—Aldis Hodge on going to “that dark place” for death row drama Clemency
—Little Women director Greta Gerwig and cast reveal how they reinvented a feminist classic
—How Netflix transformed the peak TV terrain in the 2010s
Follow Fortune on Flipboard to stay up-to-date on the latest news and analysis.
