Survey experts have yet to figure out what caused the most significant polling error in 40 years in Trump-Biden race
- Written by W. Joseph Campbell, Professor of Communication Studies, American University School of Communication
More than six months after the astonishing polling embarrassment[1] in the 2020 U.S. elections, survey experts examining what went wrong are uncertain about what led to the sharpest discrepancy between the polls and popular vote outcome since Ronald Reagan defeated Jimmy Carter in a near-landslide[2] in 1980.
Lingering questions about the misfire in 2020, in which voter support for then-President Donald Trump was understated in final pre-election polls, suggest that troubles in accurately surveying presidential elections could be deeper and more profound than previously recognized. If the source of the polling miscall isn’t clear, then addressing and correcting it obviously becomes quite challenging.
Moreover, as I discussed in my 2020 book “Lost in a Gallup[3],” polling failures in presidential elections since 1936[4] rarely have been repetitive. Just as no two elections are alike, no two polling failures are quite the same.
Over the years, pollsters have anticipated tight presidential elections when landslides have occurred[5]. They have signaled the wrong winner in closer elections. The estimates of venerable pollsters have been singularly in error[6]. Wayward exit polls[7] have thrown Election Day into confusion[8] by identifying the losing candidate as the likely winner. Off-target state polls have confounded expected national outcomes, which essentially was the story in 2016.
A voter walks to a booth to fill out their ballot at Public School 160 on Nov. 3, 2020, in the Brooklyn borough of New York City.
David Dee Delgado/Getty Images[9]
Support that wasn’t there
In 2020, overall, election polls pointed to Democrat Joe Biden’s winning the presidency. But the polls overstated support for Biden and underestimated backing for Trump no matter how close to the election the poll was conducted and regardless of the methods[10] pollsters chose. Surveys in races for U.S. senator and governor were beset by similar shortcomings.
Those were among the key findings described recently at the annual conference of the American Association for Public Opinion Research[11], which was convened online. The organization had recruited a task force[12] of 19 experts in survey research who examined the 2020 election polls in detail and reported being unable, so far, to pinpoint specific causes of polling errors.
Their findings did make clear, however, that the 2020 miscall was the most significant in 40 years[13].
Polls in the presidential race in 2020 collectively overstated Biden’s lead by 3.9 percentage points, the task force chair, Joshua Clinton of Vanderbilt University, said in a presentation at the conference.
This marked the fourth presidential election in the past five in which the national polls, at least to some extent, overstated support for Democratic candidates.
Masking dramatic miscalls
Averaging the polling errors, as the task force did in conducting its analysis, is broadly revealing about the extent of those errors. But it has the effect of masking several dramatic miscalls in late-campaign polls conducted in 2020 by, or for, leading news organizations. The final CNN poll[14] had Biden ahead by 12 points. Surveys for The Wall Street Journal-NBC News[15] and by the Economist-YouGov[16] had Biden winning by 10 percentage points as the campaign wound down. A few polls, such as Emerson College’s survey[17], came close in estimating the outcome.
Biden won the popular vote by 4.5 percentage points.
Clinton, the Vanderbilt professor, said the task force eliminated several prospective causes of polling error in 2020, including those that likely distorted survey results in key states in 2016 when Trump unexpectedly won an Electoral College victory. Those factors included undecided voters swinging to Trump late in the campaign and a failure by some pollsters to adjust survey results to account for varying levels of education.
White voters without college degrees were understood to have voted heavily for Trump in 2016, but those voters were underrepresented in some polls in key states such as Pennsylvania and Wisconsin, where Trump won narrowly and surprisingly.
A source of the miscalls in 2020, Clinton said, may have been that Republicans were less inclined than Democrats to agree to be interviewed by pollsters.
If that’s so, it’s not entirely clear why that happened. And that prospect troubles pollsters and survey research experts.
Courtney Kennedy[18], director of survey research at Pew Research Center, said while moderating a panel discussion at the conference that “what keeps me from getting a good night’s sleep these days is the prospect … Republicans, or maybe certain types of Republicans, seem like they’re less inclined to participate in polls these days than Democrats.”
This may be a tough problem for pollsters to overcome, she said, adding, “It would be a real challenge” to calibrate poll-taking to capture such nuanced distinctions.
Likewise, it is unclear whether Trump’s sharp criticism of pre-election polls[19] in 2020 dissuaded his supporters from participating in surveys.
“So it’s possible that these may be short-term phenomena that will abate when Trump is not on the ballot,” Daniel Merkle[20], president of the American Association for Public Opinion Research, said in a speech[21] recorded for conference-goers.
[Over 106,000 readers rely on The Conversation’s newsletter to understand the world. Sign up today[22].]
“On the other hand, it could be a broader issue of conservatives becoming less likely to respond to polls in general because of a decline in social trust, or for some other reasons. It will take further evaluation to understand this nonresponse issue and to adjust for it. This may not be an easy task,” Merkle said.
Screenshot of a Wall Street Journal story about its poll with NBC News, showing Biden with a 10-point lead over Trump in the waning days of the 2020 campaign.
The Wall Street Journal[23]
Overblown characterizations
In the immediate aftermath of the 2020 election, several media critics asserted that polling seemed “irrevocably broken[24]” and faced “serious existential questions[25].”
Such alarming characterizations appear overblown; polls are not going to melt away[26]. After all, election polling represents a slice of a multibillion-dollar industry that includes consumer and product surveys of all types.
And if election polling survived the debacle of 1948[27] – when President Harry S. Truman defied predictions of pollsters[28] and pundits to win reelection – then it surely will live on after the embarrassment of 2020.
References
- ^ polling embarrassment (theconversation.com)
- ^ near-landslide (news.bbc.co.uk)
- ^ Lost in a Gallup (www.ucpress.edu)
- ^ since 1936 (www.cambridge.org)
- ^ landslides have occurred (news.bbc.co.uk)
- ^ singularly in error (news.gallup.com)
- ^ Wayward exit polls (www.washingtonpost.com)
- ^ thrown Election Day into confusion (www.latimes.com)
- ^ David Dee Delgado/Getty Images (www.gettyimages.com)
- ^ regardless of the methods (www.aapor.org)
- ^ American Association for Public Opinion Research (www.aapor.org)
- ^ task force (www.aapor.org)
- ^ most significant in 40 years (www.wsj.com)
- ^ CNN poll (cdn.cnn.com)
- ^ Wall Street Journal-NBC News (www.wsj.com)
- ^ Economist-YouGov (docs.cdn.yougov.com)
- ^ Emerson College’s survey (emersonpolling.reportablenews.com)
- ^ Courtney Kennedy (www.pewresearch.org)
- ^ sharp criticism of pre-election polls (www.nytimes.com)
- ^ Daniel Merkle (twitter.com)
- ^ said in a speech (www.youtube.com)
- ^ Sign up today (theconversation.com)
- ^ The Wall Street Journal (www.wsj.com)
- ^ irrevocably broken (www.washingtonpost.com)
- ^ serious existential questions (www.theatlantic.com)
- ^ polls are not going to melt away (slate.com)
- ^ debacle of 1948 (www.latimes.com)
- ^ defied predictions of pollsters (www.baltimoresun.com)
Authors: W. Joseph Campbell, Professor of Communication Studies, American University School of Communication
