“Gosh I love election nights,” tweeted CNN’s Harry Enten late last night. The Upshot’s Nate Cohn made a cheeky Twin Peaks reference at the moment his site’s infamous needle abruptly swung from predicting a Donald Trump win in Georgia to a Joe Biden win. At least someone was having fun. Never has this form of journalism’s close relationship to the analytics movement in sports media seemed more obvious, or more irritating. And never has the type of analysis they peddle felt more useless.
“The narrative here fairly dumb overall,” FiveThirtyEight’s Nate Silver tweeted at 10:15 p.m. on election night, defending against accusations of a 2016-style polling miss as it became clear that Joe Biden was not racking up the early wins that Democrats had been hoping for. Silver’s tweet could serve as something of a mission statement for him and the other members of his coterie of data-driven political analysts: Pundits and traditional campaign journalists are in thrall to soft “narratives” about momentum and rely on anecdotes and conversations with voters. On the other hand, they believe, if you bloodlessly process a significant enough quantity of polling data, you can see the truth of what’s happening in the race. However the 2020 presidential election turns out in the end, it’s become clear that the polling analyses are themselves a “narrative,” one that can also obscure as much as it reveals.
Data journalism is only as good as the data that goes into it, and it’s already clear that the polls had some big misses this year. The New York Times and the Washington Post ran polls in the past week showing Biden with 17- and 11-point leads in Wisconsin, respectively. Biden led by 10 points in the Upshot’s final polling average for the state. (This was after steps were reportedly taken to avoid the polling misses that the state saw in 2016.) As of Wednesday morning, it’s still very possible Biden will win the state, but it will clearly be much closer than that. Just before the election, the New York Times’ Upshot projected that even with a 2016-level polling error, Biden would win or essentially be tied in Florida. Trump won the state with—as of this morning—a lead of about 3½ points. The Senate picture looks a lot grimmer for Democrats than polls anticipated, and races predicted to be tight, like South Carolina’s closely watched contest between Lindsey Graham and Jaime Harrison, were handily won by Republicans. Cook Political Report editor and widely cited polling guru Dave Wasserman tweeted on Wednesday morning, “Polls (esp. at district-level) have rarely led us more astray & it’s going to take a long time to unpack.”
If Biden ultimately wins, pollsters and the data journalists who rely on them will claim some vindication. To be fair, FiveThirtyEight’s final projections gave Biden less than a 1-in-3 chance of a landslide. The popular vote projections are likely to be pretty accurate. Yet, frustratingly, pollsters can also marshal a defense of their methods if Trump manages a surprise win. After the polling misfires of 2016, Silver and other data journalists went to great lengths to remind readers that no matter how promising the polls looked, a Trump victory could not be ruled out. “A 10 percent chance of winning is not a zero percent chance. In fact, that is roughly the same odds that it’s raining in downtown Los Angeles. And it does rain there,” noted FiveThirtyEight’s final projection.
It’s not all that comforting to Democrats today to know that 9 out of 10 times this election happens in the greater multiverse, Biden will win it. As former FiveThirtyEight writer Mona Chalabi put it, assuming the voice of FiveThirtyEight’s much-derided Fivey Fox mascot, “No matter what happens, I will find a way to say ‘I told you so! That’s how probabilities work!’ ”
It’s not that we should stop trusting polls entirely. They are a flawed but vital tool for campaigns to know where to devote resources, and for campaign journalists to use in reporting. But an entire industry of pundits and soothsayers have turned polling analysis into something more like a religion while proclaiming it a science. Meanwhile, it is increasingly unclear why these projections are useful at all.
The worst offender in this regard is the Upshot’s probability needle, which infamously crept from an 85 percent likelihood of a Clinton win at the start of election night to Trump victory in 2016. This year, they brought back the needle, in limited form—only employing it for a handful of crucial early states. But last night, in Georgia’s closely watched race, the Upshot’s projection swung from around a 90 percent chance for Trump early in the counting to a 64 percent chance for Biden this morning. Cohn seemed oddly taken aback by the level of support for Biden in Atlanta. Readers who went to bed early ended up with just as much information as those who followed the needle’s tremors minute by minute—and they likely woke up with much lower blood pressure.
In truth, the polls only reveal one sliver of the reality of elections, and especially this election. But unlike in baseball, which Silver first made a name for himself analyzing before shifting to politics, this game doesn’t always have a predictable set of rules that all players abide by. There’s much more noise in the signal that can interfere with an algorithm. Just look at the efforts by Trump and state Republicans to invalidate mail-in ballots, or the deployment of discriminatory signature-matching rules, or the weeks of Postal Service slowdowns, or the hundreds of thousands of would-be voters thwarted by delayed naturalization ceremonies, or the misinformation targeting certain types of voters to get them to stay home. It’s like using advanced metrics to predict the outcome of a baseball game where you don’t know how many players each team will be putting on a field.
The polling gurus portray themselves as objective number-crunchers, unswayed by human bias or emotion. But in truth, they are in the reassurance business. Over the past weeks and months, after any troubling piece of news came out about lagging minority turnout or legal challenges to mail-in voting, Democrats could check FiveThirtyEight and see Biden’s odds at 89 out of 100 (or, if they were really feeling glum, check the Economist’s G. Elliott Morris and see them at 19 out of 20) and feel like things were still under control.* But they’re hawking a false sense of certainty—and, presumably, racking up earth-shattering levels of web traffic in the process.
It’s been clear for weeks now that the most likely scenario was a close election that would come down to one or two Upper Midwestern states and that we likely wouldn’t know the result for a day or two after election night. If Biden does win and Democrats feel disappointed that the victory wasn’t more commanding, FiveThirtyEight’s sunny odds of 89 out of 100 probably has something to do with that.
Polls are not going anywhere, nor should they. But it should also be clear now that 2016 was not an anomaly. We’ve now been burned enough times that we have no excuse not to accord the chaos and complexity of 160 million human beings’ behavior the respect it deserves.
Correction, Nov. 4, 2020: This piece originally misspelled G. Elliott Morris’ middle name.
Support our 2020 coverage
Slate is covering the election issues that matter to you. Support our work with a Slate Plus membership. You’ll also get a suite of great benefits.
Join Slate Plus
