Better but not stellar: Pollsters faced familiar complaints, difficulties in assessing Trump-Harris race
- Written by W. Joseph Campbell, Professor Emeritus of Communication, American University School of Communication
An oracle erred badly. The most impressive results were turned in by a little-known company in Brazil. A nagging problem reemerged, and some media critics turned profane in their assessments.
So it went for pollsters in the 2024 presidential election. Their collective performance, while not stellar, was improved from that of four years earlier[1]. Overall, polls signaled a close outcome[2] in the race between former President Donald Trump and Vice President Kamala Harris.
That is what the election produced: a modest win for Trump.
With votes still being counted in California and a few other states more than a week after Election Day, Trump had received 50.1% of the popular vote to Harris’ 48.1%[3], a difference of 2 points. That margin was closer than Joe Biden’s win by 4.5 points over Trump in 2020[4]. It was closer than Hillary Clinton’s popular vote victory in 2016[5], closer than Barack Obama’s wins in 2008[6] and 2012[7].
There were, moreover, no errors among national pollsters quite as dramatic as CNN’s estimate in 2020[8] that Biden led Trump by 12 points[9].
This time, CNN’s final national poll said the race was deadlocked[10] – an outcome anticipated by six other pollsters, according to data[11] compiled by RealClearPolitics.
The most striking discrepancy this year was the Marist College poll[12], conducted for NPR and PBS. It estimated Harris held a 4-point lead nationally at campaign’s end.
‘Oracle’ of Iowa’s big miss
In any event, a sense lingered[13] among critics that the Trump-Harris election had resulted in yet another polling embarrassment, another entry in the catalog of survey failures in presidential elections[14], which is the topic of my latest book, “Lost in a Gallup[15].”
Comedian Jon Stewart gave harsh voice to such sentiments, saying of pollsters on his late-night program on election night, “I don’t ever want to fucking hear from you again. Ever. … You don’t know shit about shit, and I don’t care for you.”
Megyn Kelly, a former Fox News host, also denounced pollsters, declaring on her podcast[17] the day after the election: “Polling is a lie. They don’t know anything.”
Two factors seemed to encourage such derision – a widely discussed survey of Iowa voters released the weekend before the election and Trump’s sweep of the seven states where the outcome turned.
The Iowa poll injected shock[18] and surprise[19] into the campaign’s endgame, reporting that Harris had taken a 3-point lead[20] in the state over Trump. The result was likened to a “bombshell[21]” and its implications[22] seemed clear: If Harris had opened a lead in a state with Iowa’s partisan profile, her prospects of winning elsewhere seemed strong, especially in the Great Lakes swing states of Wisconsin, Michigan and Pennsylvania.
The survey was conducted for the Des Moines Register by J. Ann Selzer[23], a veteran Iowa-based pollster with an outstanding reputation in opinion research. In a commentary[24] in The New York Times in mid-September, Republican pollster Kristen Soltis Anderson declared Selzer “the oracle of Iowa.” Rachel Maddow of MSNBC praised[25] Selzer’s polls before the election for their “uncanny predictive accuracy.” Ratings released in June by data guru Nate Silver gave Selzer’s polls an A-plus grade[26].
But this time, Selzer’s poll missed dramatically[27].
Trump carried Iowa by 13 points, meaning the poll was off by 16 points – a stunning divergence for an accomplished pollster.
“Even the mighty have been humbled” by Trump’s victory, the Times of London said of Selzer’s polling failure[28].
Selzer said afterward[29] she will “be reviewing data from multiple sources with hopes of learning why that (discrepancy) happened.”
It is possible, other pollsters suggested, that Selzer’s reliance on telephone-based surveying contributed to the polling failure. “Phone polling alone … isn’t going to reach low-propensity voters or politically disengaged nonwhite men,” Tom Lubbock and James Johnson wrote in a commentary[30] for The Wall Street Journal.
These days, few pollsters rely exclusively on the phone[31] to conduct election surveys; many of them have opted for hybrid approaches[32] that combine, for example, phone, text and online sampling techniques.
Surprise sweep of swing states
Trump’s sweep of the seven vigorously contested swing states surely contributed to perceptions[33] that polls had misfired again.
According to RealClearPolitics[34], Harris held slender, end-of-campaign polling leads in Michigan and Wisconsin, while Trump was narrowly ahead in Arizona, Georgia, Pennsylvania, North Carolina and Nevada.
Trump won them all, an outcome no pollster anticipated – except for AtlasIntel of Sao Paulo, Brazil, a firm “about which little is known,” as The New Republic noted[35].
AtlasIntel estimated Trump was ahead in all seven swing states by margins that hewed closely to the voting outcomes. In none of the swing states did AtlasIntel’s polling deviate from the final vote tally by more than 1.3 points, an impressive performance.
AtlasIntel did not respond to email requests I sent requesting information about its background and polling technique. The company describes itself[36] as “a leading innovator in online polling” and says it uses “a proprietary methodology,” without revealing much about it.
Its founder and chief executive is Andrei Roman[37], who earned a doctorate in government at Harvard University. Roman took to X, formerly Twitter, in the election’s aftermath to post a chart that touted[38] AtlasIntel as “the most accurate pollster of the US Presidential Election.”
It was a burst of pollster braggadocio reminiscent of a kind that has emerged periodically[39] since the 1940s. That was when polling pioneer George Gallup[40] placed two-page advertising spreads in the journalism trade publication “Editor & Publisher” to assert the accuracy of his polls in presidential elections.
Underestimating Trump’s support again
A significant question facing pollsters this year – their great known unknown – was whether modifications made to sampling techniques would allow them to avoid underestimating[41] Trump’s support, as they had in 2016 and 2020.
Misjudging Trump’s backing is a nagging problem for pollsters. The results of the 2024 election indicate that the shortcoming persists[42]. By margins ranging from 0.9 points to 2.7 points, polls overall understated Trump’s support in the seven swing states, for example.
Some polls misjudged Trump’s backing by even greater margins. CNN, for example, underestimated Trump’s vote by 4.3 points in North Carolina[43], by more than 6 points in Michigan and Wisconsin[44] as well as Arizona[45].
Results that misfire in the same direction suggest that adjustments to sampling methodologies were inadequate or ineffective for pollsters in seeking to reach Trump backers of all stripes.
References
- ^ was improved from that of four years earlier (theconversation.com)
- ^ signaled a close outcome (thehill.com)
- ^ 50.1% of the popular vote to Harris’ 48.1% (www.nbcnews.com)
- ^ Joe Biden’s win by 4.5 points over Trump in 2020 (www.cnn.com)
- ^ Hillary Clinton’s popular vote victory in 2016 (www.cnn.com)
- ^ Barack Obama’s wins in 2008 (www.presidency.ucsb.edu)
- ^ and 2012 (www.presidency.ucsb.edu)
- ^ quite as dramatic as CNN’s estimate in 2020 (theconversation.com)
- ^ Biden led Trump by 12 points (www.cnn.com)
- ^ deadlocked (www.cnn.com)
- ^ according to data (www.realclearpolling.com)
- ^ Marist College poll (maristpoll.marist.edu)
- ^ a sense lingered (www.theguardian.com)
- ^ survey failures in presidential elections (theconversation.com)
- ^ Lost in a Gallup (www.ucpress.edu)
- ^ Screenshot, YouTube (www.youtube.com)
- ^ declaring on her podcast (www.newsweek.com)
- ^ injected shock (www.natesilver.net)
- ^ surprise (www.poynter.org)
- ^ a 3-point lead (www.desmoinesregister.com)
- ^ bombshell (www.msn.com)
- ^ implications (www.usatoday.com)
- ^ J. Ann Selzer (x.com)
- ^ commentary (www.nytimes.com)
- ^ Rachel Maddow of MSNBC praised (www.facebook.com)
- ^ gave Selzer’s polls an A-plus grade (www.natesilver.net)
- ^ missed dramatically (thefederalist.com)
- ^ said of Selzer’s polling failure (www.thetimes.com)
- ^ said afterward (www.desmoinesregister.com)
- ^ wrote in a commentary (www.wsj.com)
- ^ few pollsters rely exclusively on the phone (theconversation.com)
- ^ opted for hybrid approaches (www.pewresearch.org)
- ^ perceptions (www.spectator.co.uk)
- ^ RealClearPolitics (www.realclearpolling.com)
- ^ as The New Republic noted (newrepublic.com)
- ^ describes itself (ropercenter.cornell.edu)
- ^ Andrei Roman (scholar.google.com.br)
- ^ touted (x.com)
- ^ reminiscent of a kind that has emerged periodically (theconversation.com)
- ^ polling pioneer George Gallup (wapo.st)
- ^ allow them to avoid underestimating (www.nbcnews.com)
- ^ shortcoming persists (abcnews.go.com)
- ^ North Carolina (www.cnn.com)
- ^ Michigan and Wisconsin (www.cnn.com)
- ^ Arizona (www.cnn.com)
Authors: W. Joseph Campbell, Professor Emeritus of Communication, American University School of Communication
