Political forecast models aren't necessarily more accurate than polls – or the weather

As the presidential election approaches, everyone wants to know who will win.

But nobody wants to wait until the election is actually over and the votes are all counted up and double-checked.

In an effort to predict the winner weeks, or even months, in advance, pollsters take to the phones and the internet, and academics take to spreadsheets of statistics.

Some of these analysts boast impressive track records, but take caution from a political scientist who delves into the data^[1] frequently: These methods may not necessarily be more accurate than any other method of predicting the future. For some, it’s not so different from consulting Ouija boards and reading tea leaves.

The next Nostradamus?

Several political analysts have made names for themselves as predictors of election outcomes.

In the wake of the 2016 election, one political predictor, James Campbell at the University at Buffalo^[2], a longtime professor of political science, said forecasting models had been more accurate than the widely swinging public opinion polls. He listed several examples, along with how well they had predicted the election’s outcome^[3].

One of the people on his list was Stony Brook University political scientist Helmut Norpoth^[4], who back in March 2016, eight months before Election Day, had declared there was a 91% chance that Donald Trump would win^[5]. He claims to have a system capable of predicting the winner of every election outcome but two^[6], all the way back to 1912.

Instead of relying on polls, Norpoth’s analysis, called “The Primary Model^[7],” looks at the results of primary elections. For 2020, he observes that Trump won the New Hampshire and South Carolina primaries by wide margins, and therefore predicts the president will do better than Biden^[8], who split those primaries with Bernie Sanders.

American University historian Allan Lichtman^[9] was another star political forecaster, who called the 2016 election for Trump in September 2016^[10]. He has identified 13 factors he calls “keys to the White House^[11],” which include whether one candidate is an incumbent, whether the nomination was contested, whether there is a third party challenge, and Lichtman’s own assessment of the national economic conditions, the presence of a major scandal or major policy changes, as well as his views of the candidates’ charisma.

Lichtman claims he has been right about every presidential election since 1984^[12], and says he predicts Trump will lose to Biden in 2020^[13].

Is it a nice day in Wyoming?

Pomona College economist Gary Smith^[15] warns that these sorts of methods are not necessarily as robust as they may seem. Statistically speaking, he notes, “any 10 observations can always be predicted perfectly^[16] … with nine … explanatory variables.”

To demonstrate this, he used the high temperature on Election Day in five small cities^[17] across the country to create a prediction for the 2016 election, which matched up very well – at least from 1980 to 2016.

That and other examples he provides are reminders that with enough data, “spurious correlations^[18]” are everywhere – such as the famous example that from 2000 to 2009, the divorce rate in Maine^[19] was very closely matched to the per-capita consumption of margarine in the U.S.

NFL fans may recall the “Washington Rule^[20],” which claimed that if the Washington, D.C., football team won its last home game before Election Day, the party in the White House would keep it. Sportswriters claimed it would predict every election from 1940 to 2000 – but since then, it has only gotten the 2008 result correct, and has been largely discarded.

Hindsight in 2020 is 20/20

Of course, scholars’ political forecasting models do incorporate information that could be linked to the elections. For instance, I believe that party unity, economic performance, scandals and incumbency are some of the most important factors in how elections turn out.

[Insight, in your inbox each day. You can get it with The Conversation’s email newsletter^[22].]

But economist and weather-based prognosticator Smith is correct when he points out that some of these systems “predict past elections astonishingly well^[23] and then do poorly with new elections and must be tweaked, after the fact, to ‘correct’ for these mispredictions.” In fact, both Lichtman and Norpoth have made changes^[24] to their analysis methods over time^[25].

They may need more tweaks in 2020, in part because they leave out factors that haven’t been important in the past, but might be vital now. For instance, election officials across the country are expecting a flood of mail-in ballots and early voting. The New York Times finds that a record 76% of Americans^[26] can vote by mail, and Gallup polling says 64% of Americans support voting by mail^[27]. Those figures are far beyond even the 40% of votes cast in those ways^[28] in the 2016 election. In the past, when new methods of voting have emerged, outcomes have been harder to predict^[29].

The forecasts may be interesting, and – like the polls – often grab headlines, but you probably don’t want to bet too much money based on what they say.

References

1. ^{^} political scientist who delves into the data (scholar.google.com)
2. ^{^} James Campbell at the University at Buffalo (arts-sciences.buffalo.edu)
3. ^{^} how well they had predicted the election’s outcome (centerforpolitics.org)
4. ^{^} Stony Brook University political scientist Helmut Norpoth (www.stonybrook.edu)
5. ^{^} 91% chance that Donald Trump would win (www.foxnews.com)
6. ^{^} every election outcome but two (www.newsweek.com)
7. ^{^} The Primary Model (primarymodel.com)
8. ^{^} president will do better than Biden (www.newsweek.com)
9. ^{^} American University historian Allan Lichtman (www.american.edu)
10. ^{^} in September 2016 (www.realclearpolitics.com)
11. ^{^} keys to the White House (www.american.edu)
12. ^{^} right about every presidential election since 1984 (www.cnbc.com)
13. ^{^} predicts Trump will lose to Biden in 2020 (thehill.com)
14. ^{^} benedek via Getty Images (www.gettyimages.com)
15. ^{^} Pomona College economist Gary Smith (www.pomona.edu)
16. ^{^} any 10 observations can always be predicted perfectly (mindmatters.ai)
17. ^{^} the high temperature on Election Day in five small cities (mindmatters.ai)
18. ^{^} spurious correlations (www.tylervigen.com)
19. ^{^} divorce rate in Maine (www.tylervigen.com)
20. ^{^} Washington Rule (content.time.com)
21. ^{^} Gary Smith, Mind Matters (mindmatters.ai)
22. ^{^} You can get it with The Conversation’s email newsletter (theconversation.com)
23. ^{^} predict past elections astonishingly well (mindmatters.ai)
24. ^{^} made changes (news.stonybrook.edu)
25. ^{^} over time (primarymodel.com)
26. ^{^} record 76% of Americans (www.nytimes.com)
27. ^{^} 64% of Americans support voting by mail (news.gallup.com)
28. ^{^} 40% of votes cast in those ways (fivethirtyeight.com)
29. ^{^} outcomes have been harder to predict (time.com)