Rethinking reporting on polls in time for midterm elections

The Associated Press^[1], a nonprofit news wire published by 1,300 papers and broadcasters^[2], has updated its stylebook^[3] to clarify that “poll results that seek to preview the outcome of an election must never be the lead, headline or single subject of any story ^[4].”

This change is a clear response to the 2016 U.S. presidential election, where pre-election polls routinely predicted Hillary Clinton as the winner.

“The 2016 election was a reminder that polls aren’t perfect,” said AP’s Deputy Managing Editor for Operations David Scott. “They’re unquestionably a piece of the story, but never the whole story.”

The failure of pollsters to predict the 2016 election accurately has led to increased criticism of polls. Some individuals have even claimed polls are bad for democracy^[5].

The AP consulted with two prominent polling organizations, the Pew Research Center^[6] and NORC^[7] in developing these narrow and direct guidelines. They do not suggest journalists stop talking about polls, or that citizens should stop caring about polls. Instead, they guide reporters to talk about polls in a more informed way, so they can better inform citizens.

However my research^[8], completed with Cindy Kam^[9], suggests that journalists and citizens should not completely ignore polls, as they provide some benefits for citizens.

When an election is seen as competitive, citizens become more engaged^[10] and think more deeply about the candidates. Additionally, citizens are motivated to learn more^[11] about candidates who show improvements in their polling numbers. Polls can signal that an election is competitive, or that a candidate is a viable option, and this can encourage citizens to start paying more attention.

In other words, while the AP is right that a single poll often does not give a clear picture of an election, it can change how citizens think about the election. In our research, we provide citizens with a single piece of polling information. This single poll influences how citizens search for information about the candidates. When a candidate is seen as more viable, or when an election is more competitive, citizens tend to look for more policy relevant information about the candidates.

Citizens who are consuming information about polling can become better consumers of polls by following a few important guidelines.

1) Consider how the polling is done. All survey research takes a sample, or group of responses, from a population they are trying to make inferences about. With predictive election polls, this is especially difficult. Pollsters are trying to make predictions about a population – actual voters – that they cannot sample from, since individuals notoriously overreport^[12] their intention to vote.

To alleviate this problem, most polls turn to models of “likely voters.”^[13] These models are difficult to make precise and may have led to the failures to predict^[14] the 2016 election. Many organizations, such as Gallup^[15], are transparent in their methods for modeling likely voters. Others keep their methods secret. To help promote voter literacy, journalists should give preference to organizations that are transparent in their methods and take time to explain the poll’s methodology to readers.

2) Talk about margins. Most polling stories report a margin of error, but provide little explanation about what this means. The margin of error^[16] simply means how accurate the poll is. For example, a polling number of 52 percent with a margin of error of plus or minus 4 means that the “best” estimate of a candidates polling number is between 48 percent and 56 percent of the vote. If citizens begin to think of polling numbers as ranges, rather than exact estimates, this will lead to a better understanding of what polls really mean.

3) Keep it local. It is important to question how valuable national polls are in predicting presidential elections. Due to the Electoral College, the winner of the national popular vote can lose – and has lost, in two of the last five presidential elections – the presidency. A national level poll showing a candidate with a small, or even moderate, lead does not necessarily serve as an accurate predictor of the election outcome. State-level polling is more informative, but typically is conducted less often and has higher rates of error^[17], than national level polls. State polls are becoming increasingly expensive^[18], due to lower response rates, leading many polling organizations to cut back operations.

Polls can benefit democracy, but only when they are reported in an honest – rather than a misleading – way.

References

1. ^{^} Associated Press (www.ap.org)
2. ^{^} 1,300 papers and broadcasters (www.ap.org)
3. ^{^} stylebook (www.apstylebook.com)
4. ^{^} “poll results that seek to preview the outcome of an election must never be the lead, headline or single subject of any story (blog.ap.org)
5. ^{^} bad for democracy (www.usnews.com)
6. ^{^} Pew Research Center (www.pewresearch.org)
7. ^{^} NORC (www.norc.org)
8. ^{^} my research (scholar.google.com)
9. ^{^} Cindy Kam (scholar.google.com)
10. ^{^} citizens become more engaged (www.journals.uchicago.edu)
11. ^{^} citizens are motivated to learn more (www.journals.uchicago.edu)
12. ^{^} notoriously overreport (editions.lib.umn.edu)
13. ^{^} “likely voters.” (www.pewresearch.org)
14. ^{^} failures to predict (www.pewresearch.org)
15. ^{^} Gallup (news.gallup.com)
16. ^{^} margin of error (ropercenter.cornell.edu)
17. ^{^} higher rates of error (today.yougov.com)
18. ^{^} increasingly expensive (www.minnpost.com)