What if the Polls are Actually Right This Time?

Attempting to make sense of a strange period in American electoral history

For almost a decade now, Donald Trump has been systematically blowing up almost everything we thought we knew about how American politics functions at the national level. His insurgent 2015 primary candidacy was supposed to generate an initial burst of enthusiasm, some early polling momentum, and then fizzle out once the GOP establishment got their shit together and rallied around an establishment alternative.¹ Everyone from pundits, to prediction markets, to polling aggregators and forecasters expected him to lose handily to Hillary Clinton in the general election, only for him to overperform his polls by just enough to demolish the Blue Wall and ascend to the White House. The list of Trump scandals that would have ended the career of any other politician contains more line items than your average Cheesecake Factory menu.

The 2020 election was supposed to be the final repudiation of the curious but most-certainly temporary phenomenon that was Trumpism. Amid anemic approval ratings, upside-down favorability ratings, and a mismanaged global pandemic that sent the economy into a recession, Democrats managed to shrug off their own internal divisions and belatedly rallied around a boring center-left establishment nominee. Pollsters, who had missed the education realignment that had allegedly produced the spectacular failures of prognostication four years earlier, reweighted their models correct for the new reality of education polarization. The adults were back in charge.

Trump, however, was in no mood to cooperate. He proceeded to beat his polls by an even larger margin than 2016, coming within a whisker of winning a second term despite trailing by more than 8 points in FiveThirtyEight’s national polling average heading into Election Day.

So here we are, in 2024, somehow still doing this same dance. And if there’s one point that everyone seems to agree on, it’s that the race between Trump and Democratic nominee Kamala Harris is too close to call. But lurking in the background of every discussion of the 2024 election is the question of whether Trump will overperform the polls again. Because if he does, that’s basically the whole ball game.

1. The Dilemma

Before evaluating the question of whether we can expect the polls to underestimate Trump again in 2024, it’s worth briefly fleshing out some of the challenges that the polling industry faces as a whole. For some, these challenges are basically insurmountable, and the entire enterprise of political polling is fundamentally broken beyond repair. “Ignore the polls, just vote!” is their mantra, and if you’re in this category you should probably stop reading right here, because the question of whether polling is useless, or has just suffered a couple of unusually bad cycles, is at the core of the analysis to follow. Broadly, these challenges fall into a few different categories:

1. Nonresponse Bias: This occurs when certain groups of people are less likely to respond to polls. For example, supporters of certain political candidates might be less willing to participate in surveys, leading to skewed results.

2. Shy Voter Effect: Some voters might be reluctant to disclose their true preferences to pollsters, especially if their choice is controversial. This can lead to underreporting of support for certain candidates.

3. Sampling Issues: Polls rely on sampling a representative slice of the population. If the sample isn’t truly representative, the poll results can be inaccurate. This can happen due to outdated sampling methods or difficulties in reaching certain demographics.

4. Late Swings: Voters may change (or make up) their minds in the final days of the campaign, after most polls have been conducted. This can result in polls failing to capture the final shifts in voter sentiment.

5. Turnout Predictions: Polls often have to estimate who will actually turn out to vote. If these estimates are off, the poll results can be significantly skewed. Unexpectedly high or low voter turnout can thus lead to inaccuracies.

6. Herding: This is when pollsters adjust their results to align more closely with other polls, sometimes leading to a false consensus. This can mask true voter intentions and lead to widespread inaccuracies.

There are essentially two schools of thought on why both the 2016 and the 2020 elections produced major failures in polling. The first is that there’s something unique to Trump’s appeal as a candidate. He mobilizes certain types of voters when he’s on the ballot, and only when he’s on the ballot, who are basically impossible to account for through either survey methodology or weighting. Some combination of response bias, a Shy Trump Effect, and sampling issues have failed to properly identify a sizeable population of largely rural, white, marginal voters that Trump is consistently able to turn out. This is why the polls missed so badly in 2016 and 2020, but were pretty accurate in 2018 and 2022.

The second is that political polling suffered from two separate and almost entirely unrelated failures in the 2016 and 2020 elections. These failures were not easily foreseeable at the time, but with hindsight, they can be corrected for using proper methodology, and are not expected to reoccur in 2024.

In 2016, the polls missed the mark largely because of the effects of education polarization, leading them to significantly underestimate Trump’s support among white, working-class voters, particularly in the Midwest. There were also unusually high numbers of undecided voters heading into the home stretch of the campaign, and those undecideds broke heavily for Trump:

For instance, according to the American Association for Public Opinion Research, some voters did not decide whose name to write on their ballot until the last minute, making them difficult to account for.

And some voters were shy about their support for Trump due to his controversial rhetoric during the 2016 campaign. As a result, they did not always admit their voting intentions to pollsters.

But other factors were direct results of methodological oversight.

“People didn’t factor in educational representation,” said Matin Mirramezani, chief operating officer at Generation Lab, a polling organization that specifically targets young voters. “Education is a lesson learned from 2016.”

In 2020, this education bias had mostly been corrected for, but an entirely new problem of COVID/George Floyd-related partisan response bias then reared its head, leading to an even greater underestimation of Trump’s support. Democrats were not only much more engaged amid the Resistance memes and social justice protests, but also more likely to take surveys because they were stuck at home without anything better to do. As Nate Cohn put it in the New York Times:

In the aftermath of the 2020 election, most pollsters concluded that the polls probably underestimated Donald J. Trump because of something called nonresponse bias. In short, Mr. Trump’s supporters were less likely to respond to surveys than Joe Biden’s supporters, even among people who had the same demographic characteristics.

The problem here, as I’m sure you’ve noticed, is that it’s very difficult to distinguish these two hypotheses given the limited data we have available. There have been only two general elections and two midterm elections in the years since Trump became a national political figure, and the patterns in those elections (big misses in 2016 and 2020; decent performance in 2018 and 2022), are broadly consistent with both explanations. So, is there any way for us to say which one is more likely?

2. The Data

The following section is based on analysis of the national and state polling averages published by RealClearPolitics and Nate Silver/FiveThirtyEight during the 2016, 2020, and 2024 campaigns, along with the actual results for 2016 and 2020.² For consistency, I used the FiveThirtyEight numbers for 2016 and 2020, and the Silver Bulletin numbers for 2024.³ The topline results are interesting, both in terms of the national numbers and the state polls.

First, despite persistent controversies about their averaging methods and poll-inclusion criteria, RCP’s polling averages were consistently closer to the final vote totals than those produced by Nate Silver/FiveThirtyEight. RCP underestimated Trump’s national margin by 1.1% in 2016 and by 2.7% in 2020, while Silver’s numbers missed the mark by 1.8% and 3.9% respectively. At the state level, the polling averages did even worse: RCP underestimated Trump by an average of 1.5% in 2016 and 2.3% in 2020 across the 15 competitive states for which comparable data was available in all three elections, while Silver was again further off the mark, missing by 3.1% and 4.0% respectively.

What if we limit the analysis to the specific subset of states that are expected to be decisive in 2024? Looking at the polling averages and final results in Arizona, Georgia, Michigan, Nevada, North Carolina, Pennsylvania, and Wisconsin, RCP underestimated Trump by 1.8% in 2016, but actually did slightly better in 2020, averaging only 1.2% below his actual performance. Silver’s numbers look even worse: he underestimated Trump’s support by 3.2% in 2016 and 3.7% in 2020.

So far, none of this is looking especially encouraging for Harris boosters. The polls underestimated Trump both nationally and at the state level in both 2016 and 2020, and this effect was particularly pronounced in the states that are expected to be closest in 2024, with the exception of RCP’s 2020 numbers.

The polling averages for 2024 exhibit a similar divergence between the RCP and Silver averages. As of Monday 9/23, RCP had Harris ahead by 0.2% in the entire sample of competitive states, and trailing by just 0.1% in the seven 2024 swing states. Silver, meanwhile, had Harris ahead by 0.7% in the competitive states and 0.6% in the seven swing states. Did I mention that this election is expected to be close?

Overall, the polling in 2024 is considerably more bullish on Trump than it was in either 2016 or 2020. Either Trump is running significantly ahead of his performance in those two prior campaigns, or the same patterns of polarization and negative partisanship that made those elections so close are still present, and the polls are now more accurately reflecting those trends. So which is it?

One final data point that I’ve not really seen analyzed anywhere else, but which might provide some insight into that question. We know that the polls missed the mark in 2016, and then by an even larger margin in 2024, but how closely do the current 2024 polling averages correspond to the final results in those prior elections? As it turns out, both the RCP and Silver polling averages do a much better job of “predicting” the results of the 2020 election than the polls conducted during the 2020 campaign did. RCP’s 2024 polling average overestimates Trump’s 2020 performance by just 0.8% in the full competitive state sample and 1.0% in the seven swing states, while Silver’s numbers slightly understate Trump’s 2020 electoral strength, by 0.2% and 0.3% in the respective samples. The gap with 2016 is slightly larger, reflecting Trump’s stronger performance in that election, but still broadly in line with the accuracy of the actual 2016 polls.

3. Conclusion

What can we take away from this analysis? Returning to the original hypotheses, the congruence between the 2024 polling averages and the 2020 results suggest that we’re living in one of two realities. Either (1) Trump is doing significantly better against Harris than he was against Biden, and the polls are once again underestimating his support, in which case we’re heading for an election in which he sweeps all or most of the swing states and wins with upwards of 300 electoral votes; or (2) we’re living in a highly polarized nation where changes of one or two points in the national popular vote translate into narrow and shifting margins in the swing states, and Harris’s current lead puts her right in the sweet spot between Clinton and Biden’s popular vote performances where the election balances on a knife edge.

To me, the data seems somewhat more consistent with the second scenario. Trump’s favorability rating (-8.4) does represent an improvement in his popularity compared to 2020 (-12.8) or 2016 (-16.4). But on the flip side, Harris (+1.1) is considerably more popular than Hillary Clinton was in 2016 (-12.6), albeit somewhat less so than Joe Biden was in 2020 (+6.2).

All of this seems to wash out into something approximately the reality suggested by the polls: Harris is much better liked than Clinton was, and slightly less so than Biden was, but she faces a slightly more popular version of Trump than either of them had to run against. So, her lead is larger than Clinton’s in 2016 but still smaller than Biden’s was in 2020. The underlying reality has not really changed in any significant respect, but it was obscured by the effects of unexpected education polarization that biased the polls against Trump in 2016, and the effects of nonresponse bias that further biased them against him in 2020. With those biases now either absent or corrected for, the polls, which we must remember were pretty good in both 2018 and 2022, may have finally converged with reality.

1. This had happened to an almost comical degree in 2012, as a whole series of anyone-but-Romney candidates (Rick Perry, Herman Cain!, Newt Gingrich, Rick Santorum) surged into the lead in primary polling, only to collapse as quickly as they had emerged.

2. The state polling averages include the following states: Arizona, Florida, Georgia, Iowa, Michigan, Minnesota, Nevada, New Hampshire, North Carolina, New Mexico, Ohio, Pennsylvania, Texas, Wisconsin, and Virginia. Basically, this sample includes states that were close in 2016 but have since become less competitive, states that have remained competitive throughout the entire period, and states that were less competitive in 2016 but have since become closer.

3. Nate Silver’s proprietary poll-aggregation and forecasting models formed the basis for the FiveThirtyEight averages and predictions in 2016 and 2020. When Silver left Disney in May 2023, he retained the intellectual property rights to these models, and they are now used at Silver Bulletin. FiveThirtyEight’s 2024 models were newly created under his successor, G. Elliot Morris.