Election polls are more accurate if they ask participants how others will vote

Most public opinion polls correctly predicted the winning candidate in the 2020 U.S. presidential election – but on average, they overestimated the margin by which Democrat Joe Biden would beat Republican incumbent Donald Trump.

Our research into polling methods has found that pollsters’ predictions can be more accurate if they look beyond traditional questions. Traditional polls ask people whom they would vote for if the election were today, or for the percent chance^[1] that they might vote for particular candidates.

But our research into people’s expectations^[2] and social judgments^[3] led us and our collaborators, Henrik Olsson^[4] at the Santa Fe Institute and Drazen Prelec^[5] at MIT, to wonder whether different questions could yield more accurate results.

Specifically, we wanted to know whether asking people about the political preferences of others in their social circles and in their states could help paint a fuller picture of the American electorate. Most people know quite a bit about the life experiences of their friends and family^[6], including how happy and healthy they are and roughly how much money they make. So we designed poll questions to see whether this knowledge of others extended to politics – and we have found that it does.

Pollsters, we determined, could learn more if they took advantage of this type of knowledge. Asking people how others around them are going to vote and aggregating their responses across a large national sample enables pollsters to tap into what is often called “the wisdom of crowds^[7].”

What are the new ‘wisdom-of-crowds’ questions?

Since the 2016 U.S. presidential election season, we have been asking participants in a variety of election polls: “What percentage of your social contacts will vote for each candidate?^[8]”

In the 2016 U.S. election, this question predicted that Trump would win, and did so more accurately than questions asking about poll respondents’ own voting intentions.

The question about participants’ social contacts was similarly more accurate^[9] than the traditional question at predicting the results of the 2017 French presidential election, the 2017 Dutch parliamentary election, the 2018 Swedish parliamentary election and the 2018 U.S. election for House of Representatives.

In some of these polls, we also asked, “What percentage of people in your state will vote for each candidate?” This question also taps into participants’ knowledge of those around them, but in a wider circle. Variations of this question have worked well in previous elections^[10].

How well did the new polling questions do?

In the 2020 U.S. presidential election, our “wisdom-of-crowds” questions were once again better at predicting the outcome of the national popular vote than the traditional questions. In the USC Dornsife Daybreak Poll^[11] we asked more than 4,000 participants how they expected their social contacts to vote and which candidate they thought would win in their state. They were also asked how they themselves were planning to vote.

The current election results show a Biden lead of 3.7 percentage points^[12] in the popular vote. An average of national polls^[13] predicted a lead of 8.4 percentage points. In comparison, the question about social contacts predicted a 3.4-point Biden lead^[14]. The state-winner question predicted Biden leading by 1.5 points. By contrast, the traditional question that asked about voters’ own intentions in the same poll predicted a 9.3-point lead.

Why do the new polling questions work?

We think there are three reasons that asking poll participants about others in their social circles and their state ends up being more accurate than asking about the participants themselves.

First, asking people about others effectively increases the sample size of the poll. It gives pollsters at least some information about the voting intentions of people whose data might otherwise have been entirely left out. For instance, many were not contacted by the pollsters, or may have declined to participate. Even though the poll respondents don’t have perfect information about everyone around them, it turns out they do know enough to give useful answers.

Second, we suspect people may find it easier to report about how they think others might vote than it is to admit how they themselves will vote^[15]. Some people may feel embarrassed to admit who their favorite candidate is. Others may fear harassment. And some might lie because they want to obstruct pollsters. Our own findings suggest that Trump voters might have been more likely than Biden voters to hide their voting intentions, for all of those reasons.

Third, most people are influenced by others around them. People often get information about political issues from friends and family – and those conversations may influence their voting choices^[16]. Poll questions that ask participants how they will vote do not capture that social influence. But by asking participants how they think others around them will vote, pollsters may get some idea of which participants might still change their minds.

Other methods we are investigating

Building on these findings, we are looking at ways to integrate information from these and other questions^[17] into algorithms that might make even better predictions of election outcomes.

One algorithm, called the “Bayesian Truth Serum^[18],” gives more weight to the answers of participants who say their voting intentions, and those of their social circles, are relatively more prevalent than people in that state think. Another algorithm, called a “full information forecast^[19],” combines participants’ answers across several poll questions to incorporate information from each of them. Both methods largely outperformed the traditional polling question and the predictions from an average of polls^[20].

[Deep knowledge, daily. Sign up for The Conversation’s newsletter^[21].]

Our poll did not have enough participants in each state to make good state-level forecasts that could help predict votes in the Electoral College. As it was, our questions about social circles and expected state winners predicted that Trump might narrowly win the Electoral College. That was wrong, but so far it appears that these questions had on average lower error than the traditional questions in predicting the difference between Biden and Trump votes across states.

Even though we still don’t know the final vote counts for the 2020 election, we know enough to see that pollsters could improve their predictions by asking participants how they think others will vote.

References

1. ^{^} percent chance (academic.oup.com)
2. ^{^} people’s expectations (priceschool.usc.edu)
3. ^{^} social judgments (www.santafe.edu)
4. ^{^} Henrik Olsson (www.santafe.edu)
5. ^{^} Drazen Prelec (economics.mit.edu)
6. ^{^} quite a bit about the life experiences of their friends and family (dx.doi.org)
7. ^{^} the wisdom of crowds (www.penguinrandomhouse.com)
8. ^{^} What percentage of your social contacts will vote for each candidate? (www.nature.com)
9. ^{^} similarly more accurate (www.youtube.com)
10. ^{^} in previous elections (academic.oup.com)
11. ^{^} USC Dornsife Daybreak Poll (election.usc.edu)
12. ^{^} Biden lead of 3.7 percentage points (cookpolitical.com)
13. ^{^} An average of national polls (projects.fivethirtyeight.com)
14. ^{^} predicted a 3.4-point Biden lead (osf.io)
15. ^{^} to admit how they themselves will vote (healthpolicy.usc.edu)
16. ^{^} may influence their voting choices (psycnet.apa.org)
17. ^{^} integrate information from these and other questions (osf.io)
18. ^{^} Bayesian Truth Serum (science.sciencemag.org)
19. ^{^} full information forecast (osf.io)
20. ^{^} average of polls (projects.fivethirtyeight.com)
21. ^{^} Sign up for The Conversation’s newsletter (theconversation.com)