In its troubled hour, polling could use an irreverent figure to reset expectations
- Written by W. Joseph Campbell, Professor of Communication Studies, American University School of Communication
Polling is hardly a flamboyant field that attracts a lot of colorful characters. It is a rather reserved profession that now finds itself under siege in the aftermath of yet another polling surprise[1] in a national election.
The field is buffeted by intense criticism[2] – by even extreme claims[3] that it may be doomed – following mischaracterizations in national polls that former Vice President Joe Biden was bound for a blowout victory[4].
Many preelection polls suggested it was to be a “blue wave” election[5] in which Biden would easily take over the White House, while fellow Democrats would sweep to control in the Senate and fortify their majority in the House of Representatives.
The 2020 election was closer and more complex than most national polls indicated[6], and it marked the second successive polling surprise[7] in a U.S. presidential election. In 2016, polls in key Great Lakes states underestimated support for Donald Trump, states that were crucial to his winning the White House.
In its troubled hour, polling could use a prominent, outspoken and irreverent character who knows the profession’s intricacies and whose default isn’t to defensiveness. Such a figure could place polling’s latest misstep in useful and plausible perspective, and do so candidly, without seeming too haughty or arcane about it.
The answer is ‘No’ to the question posed in a Nov. 3 Wall Street Journal story.
Wall Street Journal[8]
‘To prove we’re not yellow’
Polling has no such colorful, outspoken character now. It did once, in Burns (“Bud”) Roper[9], the Iowa-born son of a pioneer in modern survey research, Elmo Roper. Bud Roper was disarming enough to tell a newspaper reporter in the 1950s: “I guess the main reason we do these election polls at all is to prove we’re not yellow,” or cowardly.
Roper[10], who died in 2003, was in polling much of his adult life, entering his father’s market research firm after World War II. He retired as the company’s chairman in 1994. He was around when the Roper poll dramatically miscalled the 1948 presidential election[11], predicting that Thomas E. Dewey would defeat President Harry Truman by 15 percentage points.
Truman won reelection by 4.5 points, which meant Roper’s polling error was a staggering 19.5 percentage points – almost as dreadful as the Literary Digest failure[12] in 1936, when the venerable magazine’s mail-in survey erroneously pegged Alf Landon to unseat President Franklin D. Roosevelt by a wide margin.
The 1936 debacle occurred at the dawn of modern opinion research and, as I write in my latest book, “Lost in a Gallup: Polling Failure in U.S. Presidential Elections[13],” it left a legacy of nagging doubt about the effectiveness of polling in estimating election outcomes.
Nonetheless, it is also true that journalists, and the public, inevitably turn to polls[14] – and the illusion of precision they offer – in seeking clarity about the dynamics of a presidential campaign. Even after the back-to-back embarrassments [15] in 2016 and 2020, election polling is surely not destined for collapse or dissolution. Polling may be an unglamorous profession; it also is a hardy[16] one.
Bud Roper was willing to criticize his profession.
Screenshot, The Roper Center for Public Opinion Research[17]
Bud Roper’s long career traced fairly well polling’s entrenchment in American politics and culture. He once said that he entered the field when it was somewhere between “a kooky off-the-wall and an established industry.”
In some ways, Roper’s most noteworthy contribution was candor and a refreshing disinclination to take survey research all that seriously. In that sense, he was like his father, who began conducting preelection polls in 1936 but came to doubt their value.
In the run-up to the 1948 election, for example, Elmo Roper equated polling to “a stunt, like balancing cocktail glasses on top of each other or tearing a telephone book in two. It’s impressive. It has a certain fascination. But it tells us very little that we wouldn’t find out even if poll-taking had never been invented.”
Bud Roper similarly tended toward colorful outspokenness. He was not hesitant to call out his profession for its shortcomings and flaws.
Associated Press journalists in the Washington bureau tabulate election returns Nov. 5, 1940, keeping the score on both electoral and popular votes for the nation.
AP Photo[18]
‘Largely art’
In 1984, at a time when election polling was going through another rough patch, Bud Roper said in a speech[19] to the American Association for the Advancement of Science, “Our polling techniques have gotten more and more sophisticated, yet we seem to be missing more and more elections.”
Roper was frank about some of polling’s unresolved headaches, such as differentiating between likely and unlikely voters – a determination crucial to a survey’s accuracy.
“One of the trickiest parts of an election poll is to determine who is likely to vote and who is not,” Roper once said, adding with characteristic frankness, “I can assure you that this determination is largely art.”
The likely-voter conundrum[20] remains a defiant and persistent problem. It also is an important reason that election polling is a blend of art and science, which Roper liked to emphasize. In fact, he said[21] it tended to be more art than science.
“I have heard it said that opinion research is half art and half science,” Roper stated in an address to members of the American Association for Public Opinion Research[22] at the close of his yearlong presidential term in 1983. “I would say that a good deal more than half is art and correspondingly less than half is science.”
Roper held some out-of-the-mainstream ideas about polling. He was not enamored with surveys conducted by telephone, noting they too often interrupted respondents and disrupted their routines. Roper argued, somewhat vaguely, a solution to the sharp decline in response rates[23] to telephone surveys was to “go back to personal interviews. Telephone won’t do it, internet won’t do it, email won’t do it,” he said[24] late in his life.
He added: “I don’t have all the answers as to how, but if [the problem of declining response rates] is not solved, I think the industry as we’ve known it is going to be – oh, it’ll survive, but it’s going to survive with worse and worse results every time we go up.”
Taking responsibility for a bad poll
Roper was not one to sidestep controversy. He conceded error without hesitation when, in 1993, his company conducted a survey for the American Jewish Committee that suggested 22% of Americans doubted[25] the Holocaust had occurred.
It was a surprising, controversial and off-target finding that Roper soon questioned, noting the question’s wording included a double negative and should have been rephrased. When the question was revised and posed in a separate survey, only 1.1%[26] of the respondents said they doubted the Holocaust.
[Deep knowledge, daily. Sign up for The Conversation’s newsletter[27].]
Roper said he regretted[28] that the original poll’s finding “served to misinform the public, to scare the Jewish community needlessly and to give aid and comfort to the neo-Nazis who have a commitment to Holocaust denial.”
In saying so, Roper showed he could stand up and take responsibility for a bad poll. It’s a lesson that has enduring relevance.
References
- ^ polling surprise (theconversation.com)
- ^ intense criticism (www.theatlantic.com)
- ^ extreme claims (thehill.com)
- ^ blowout victory (thehill.com)
- ^ a “blue wave” election (fivethirtyeight.com)
- ^ polls indicated (projects.fivethirtyeight.com)
- ^ polling surprise (theconversation.com)
- ^ Wall Street Journal (www.wsj.com)
- ^ Roper (www.nytimes.com)
- ^ Roper (www.latimes.com)
- ^ miscalled the 1948 presidential election (www.nytimes.com)
- ^ Literary Digest failure (www.jstor.org)
- ^ Lost in a Gallup: Polling Failure in U.S. Presidential Elections (www.ucpress.edu)
- ^ inevitably turn to polls (theconversation.com)
- ^ back-to-back embarrassments (www.cnbc.com)
- ^ hardy (slate.com)
- ^ Screenshot, The Roper Center for Public Opinion Research (ropercenter.cornell.edu)
- ^ AP Photo (newsroom.ap.org)
- ^ said in a speech (www.ucpress.edu)
- ^ likely-voter conundrum (www.pewresearch.org)
- ^ said (www.aapor.org)
- ^ American Association for Public Opinion Research (www.aapor.org)
- ^ decline in response rates (www.pewresearch.org)
- ^ said (www.youtube.com)
- ^ doubted (www.jstor.org)
- ^ 1.1% (www.jta.org)
- ^ Sign up for The Conversation’s newsletter (theconversation.com)
- ^ he regretted (www.latimes.com)
Authors: W. Joseph Campbell, Professor of Communication Studies, American University School of Communication
