Why the margin of error matters more than ever in reading 2024 election polls – a pollster with 30 years of experience explains
- Written by Doug Schwartz, Director of the Quinnipiac Poll, Quinnipiac University
In just about any discussion of a poll about the very close presidential race between Vice President Kamala Harris and former President Donald Trump, you’ll hear the phrase “within the poll’s margin of error[1].” Those words signal that it is a tight race with no clear leader[2], even if one of them has a slightly larger percentage of support, like 48% to 47%[3].
As the director of the Quinnipiac University Poll[4], which has been taking the pulse of the public on policy issues and elections for the past 30 years, I’ve noted that people have been paying more attention to this technical term since at least 2016.
In that year, some polls in Florida, for example, indicated that Hillary Clinton was just a couple of percentage points ahead[5] of Trump. Journalists and the public largely – and incorrectly – understood that apparent popular-vote lead to mean Clinton was likely to win[6].
But those 1 or 2 percentage points were within their polls’ margins of error. And Clinton lost Florida. In a poll about a political race, the margin of error tells readers the likely range of results of an election.
What is a margin of error?
A poll is one or more questions asked of a small group of people and used to gauge the views of a larger group of people. The margin of error is a mathematical calculation of how accurate the poll results are – of how closely the answers given by the small group match the views held by the larger group.
If everyone in the larger group were polled, there would be no margin of error. But it’s complicated, difficult and expensive to contact that many people. The U.S. Census Bureau spent US$13.7 billion over several years[7] in its most recent effort to count every person in the United States every 10 years, and it still wasn’t able to include exactly everyone[8].
Pollsters don’t have that kind of time – or money – so they use smaller samples of the population. They seek to identify representative samples[9] in which all members of the larger group have a chance to be included in the poll.
The group size is important
The calculation of how close the poll is to the views of the larger population is based on the size of the group that is polled.
For example, a sample of 600 voters will have a larger margin of error – about 4 percentage points – than a sample of 1,000 voters, which has a margin of error of just over 3 percentage points.
The way the sample is chosen also matters: In 1936, the Literary Digest magazine polled people on the presidential election by mailing surveys to telephone owners, car owners and country club members[10]. Everyone in this group was relatively affluent, so they were not representative of the whole U.S. voting population. Calculating a margin of error would have been meaningless because the sample did not capture all segments of the population.
A concrete example
Let’s use an example of how to understand the margin of error. If a poll shows that 47% of the polled group support Candidate A, and the margin of error is plus or minus 3 percentage points, that means that the percentage in the population supporting Candidate A is likely to be between 44% (47 minus 3) and 50% (47 plus 3).
One quick note: Most polls report margins of error alongside another technical term, “confidence interval[13].” In the most rigorous reporting of polls, you might see a sentence near the end that says something like “The margin of error is plus or minus 3 percentage points, at a 95% confidence interval.” What all that means is this: Imagine if 100 different random samples of the same size were selected from the larger group, and then asked the same questions in the poll. The 95% confidence interval means that 95% of the time, those other polls’ responses would be within 3 percentage points of the answers reported in this one poll.
Comparing support between candidates
The concept of margin of error gets more complex when looking at the differences in support between two candidates. If a margin of error is plus or minus 3 percentage points, the margin of error on the difference between them is about double – or 6 percentage points, in this example.
That’s because the margin of error here is a combined one, and refers to not just the percentage voting for Candidate A but also to the percentage voting for the other candidate.
To look back at 2016 again, the final Quinnipiac University Poll in Florida before Election Day[14] showed Clinton with 46% support and Trump with 45% support. The margin of error was 3.9 percentage points, which meant Clinton was likely to get between 42.1% and 49.9% of the vote, and Trump was likely to get between 41.1% and 48.9% of the vote.
The actual result was that Trump won Florida with 48.6%[15], as compared with Clinton’s 47.4%. Those results were within our poll’s margin of error, meaning we were correct to declare it “too close to call” – and we would have been wrong to say Clinton was ahead.
2024 will be a close election
In the current election cycle, many media reports about polls are not including information[16] about the margin of error.
Leaving out that information, or downplaying its significance, may help media outlets provide a quick, simple picture about the state of the race. Technology can seem precise in the modern age of the internet and artificial intelligence.
But polling is not as precise. It is an inexact science. It’s a pollster’s job to capture snapshots of the complexities of human nature at a particular time. People’s minds can change, and new information can arise as the campaigns unfold.
With the presidential election in its final weeks, our polls have been finding a fairly tight and steady race[17], with most voters telling us their minds are made up. Because the difference between the presidential candidates is within the margin of error in swing states, the election polling in autumn 2024 is telling Americans to hold their breath and make sure they vote, because it is likely to be a squeaker.
References
- ^ within the poll’s margin of error (www.nbcnews.com)
- ^ tight race with no clear leader (www.realclearpolling.com)
- ^ like 48% to 47% (poll.qu.edu)
- ^ director of the Quinnipiac University Poll (www.qu.edu)
- ^ Clinton was just a couple of percentage points ahead (www.realclearpolling.com)
- ^ Clinton was likely to win (www.nytimes.com)
- ^ US$13.7 billion over several years (www.gao.gov)
- ^ wasn’t able to include exactly everyone (www.census.gov)
- ^ representative samples (www.pewresearch.org)
- ^ mailing surveys to telephone owners, car owners and country club members (www.cambridge.org)
- ^ Zieben007 via Wikimedia Commons (commons.wikimedia.org)
- ^ CC BY-SA (creativecommons.org)
- ^ confidence interval (www.investopedia.com)
- ^ Quinnipiac University Poll in Florida before Election Day (poll.qu.edu)
- ^ Trump won Florida with 48.6% (www.nytimes.com)
- ^ not including information (www.nytimes.com)
- ^ fairly tight and steady race (poll.qu.edu)
Authors: Doug Schwartz, Director of the Quinnipiac Poll, Quinnipiac University
