With memories of embarrassments still fresh, election pollsters face big tests in 2022 midterm elections
- Written by W. Joseph Campbell, Professor of Communication Studies, American University School of Communication
When it became clear his poll had erred in the 2021 New Jersey governor’s race, Patrick Murray, director of the Monmouth University Polling Institute, acknowledged:
“I blew it.”
The campaign’s final Monmouth poll[1] estimated Gov. Phil Murphy’s lead over Republican foe Jack Ciattarelli at 11 percentage points – a margin that “did not provide an accurate picture of the state of the governor’s race,” Murray later said in a newspaper commentary. Murphy won by 3.2 points.
It was a refreshingly candid acknowledgment by an election pollster.
More broadly, the error was one of several in the recent past[2] and looms among the disquieting[3] omens confronting pollsters in the 2022 midterm elections. Will they be embarrassed again? Will their polls in high-profile U.S. Senate and gubernatorial races produce misleading indications of election outcomes?
Such questions are hardly far-fetched or irrelevant, given election polling’s tattered recent record. A few prominent survey organizations in recent years have given up on election polling[4], with no signs of returning.
Treat polls warily
It is important to keep in mind that polls are not always in error, a point noted in my 2020 book, “Lost in a Gallup: Polling Failure in U.S. Presidential Elections[6].” But polls have been wrong often enough over the years that they deserve to be treated warily and with skepticism.
For a reminder, one need look no further than New Jersey in 2021 or, more expansively, to the 2020 presidential election[7]. The polls pointed to Democrat Joe Biden’s winning the presidency but underestimated popular support for President Donald Trump by nearly 4 percentage points overall.
That made for polling’s worst collective performance in a presidential campaign in 40 years, and post-election analyses were at a loss to explain[8] the misfire. One theory was that Trump’s hostility[9] to election surveys dissuaded supporters from answering pollsters’ questions.
In any case, polling troubles in 2020 were not confined to the presidential race: In several Senate and gubernatorial campaigns, polls also overstated support for Democratic candidates. Among the notable flubs was the U.S. Senate race in Maine[10], where polls signaled defeat for the Republican incumbent, Susan Collins. Not one survey[11] in the weeks before the election placed Collins in the lead.
She won reelection[12] by nearly 9 points.
Recalling the shock of 2016
The embarrassing outcomes of 2020 followed a stunning failure in 2016, when off-target polls in key Great Lakes states confounded expectations of Hillary Clinton’s election to the presidency[13]. They largely failed to detect late-campaign shifts in support to Trump, who won a clear Electoral College victory despite losing[14] the national popular vote.
Past performance is not always prologue in election surveys; polling failures are seldom alike. Even so, qualms about a misfire akin to those of the recent past have emerged during this campaign.
In September 2022, Nate Cohn, chief political analyst for The New York Times, cited the possibility of misleading polls in key races, writing[15] that “the warning sign is flashing again: Democratic Senate candidates are outrunning expectations in the same places where the polls overestimated Mr. Biden in 2020 and Mrs. Clinton in 2016.”
There has been some shifting in Senate polls since then, and surely there will be more before Nov. 8. In Wisconsin, for example, recent surveys suggest Republican incumbent Ron Johnson has opened a lead over Democratic challenger Mandela Barnes. Johnson’s advantage was estimated[16] at 6 percentage points not long ago in a Marquette Law School Poll.
The spotlight on polling this election season is unsurprising, given that key Senate races[17] – including those featuring flawed candidates in Pennsylvania and Georgia[18] – will determine partisan control of the upper house of Congress.
Worth doing?
Polling is neither easy nor cheap if done well, and the field’s persistent troubles have even prompted the question whether election surveys are worth the bother.
Monmouth’s Murray spoke to that sentiment, stating[19]: “If we cannot be certain that these polling misses are anomalies then we have a responsibility to consider whether releasing horse race numbers in close proximity to an election is making a positive or negative contribution to the political discourse.”
He noted that prominent survey organizations such as Pew Research and Gallup quit[20] election polls several years ago to focus on issue-oriented survey research. “Perhaps,” Murray wrote, “that is a wise move.”
Questions about the value[21] of election polling run through the history of survey research and never have been fully settled. Early pollsters such as George Gallup and Elmo Roper were at odds about such matters.
Gallup used to argue[22] that election polls were acid tests, proxies for measuring the effectiveness of surveys of all types. Roper equated election polling to stunts[23] like “tearing a telephone book in two” – impressive, but not all that consequential.
Pollsters in 2016 predicted Democratic presidential candidate Hillary Clinton would win some states that she actually lost.
Screenshot, New York Times[24]
Who is and isn’t responding
Experimentation, meanwhile, has swept the field, as contemporary pollsters seek new ways of reaching participants and gathering data.
Placing calls to landlines and cellphones – once polling’s gold standard methodology[25] – is expensive and not always effective, as completion rates in such polls tend to hover in the low single digits. Many people ignore calls from numbers they do not recognize, or decline to participate when they do answer.
Some polling organizations have adopted a blend of survey techniques, an approach known as “methodological diversity.” CNN announced[26] in 2021, for example, that it would include online interviews with phone-based samples in polls that it commissions. A blended approach, the cable network said, should allow “the researchers behind the CNN poll to have a better understanding of who is and who is not responding.”
During an online discussion last year[27], Scott Keeter[28] of Pew Research said “methodological diversity is absolutely critical” for pollsters at a time when “cooperation is going down [and] distrust of institutions is going up. We need to figure out lots of ways to get at our subjects and to gather information from them.”
So what lies immediately ahead for election polling and the 2022 midterms?
Some polls of prominent races may well misfire. Such errors could even be eye-catching.
But will the news media continue to report frequently on polls in election cycles ahead?
Undoubtedly.
After all, leading media outlets, both national and regional, have been survey contributors for years, conducting or commissioning – and publicizing – election polls of their own.
References
- ^ campaign’s final Monmouth poll (www.monmouth.edu)
- ^ one of several in the recent past (theconversation.com)
- ^ looms among the disquieting (www.politico.com)
- ^ have given up on election polling (time.com)
- ^ Mark Makela/Getty Images (www.gettyimages.com)
- ^ Lost in a Gallup: Polling Failure in U.S. Presidential Elections (www.ucpress.edu)
- ^ to the 2020 presidential election (www.washingtonpost.com)
- ^ at a loss to explain (theconversation.com)
- ^ hostility (www.nytimes.com)
- ^ Maine (www.bangordailynews.com)
- ^ Not one survey (www.realclearpolitics.com)
- ^ won reelection (www.brookings.edu)
- ^ confounded expectations of Hillary Clinton’s election to the presidency (www.nytimes.com)
- ^ who won a clear Electoral College victory despite losing (apnews.com)
- ^ writing (www.nytimes.com)
- ^ estimated (law.marquette.edu)
- ^ Senate races (www.politico.com)
- ^ Georgia (www.msn.com)
- ^ stating (www.nj.com)
- ^ quit (www.politico.com)
- ^ value (www.politico.com)
- ^ Gallup used to argue (doi.org)
- ^ Roper equated election polling to stunts (www.ucpress.edu)
- ^ Screenshot, New York Times (www.nytimes.com)
- ^ gold standard methodology (poll.qu.edu)
- ^ announced (www.cnn.com)
- ^ online discussion last year (ecornell.cornell.edu)
- ^ Keeter (www.pewresearch.org)
Authors: W. Joseph Campbell, Professor of Communication Studies, American University School of Communication
