How to know which impeachment polls to believe – and which to skip

Pollsters are trying their best to track public opinion about the House Democrats’ decision to initiate an impeachment inquiry against President Donald Trump.

Currently, it is doubtful the required two-thirds of the Senate, now controlled by Republicans^[1], would vote to convict Trump. But senators’ votes could change by the time they receive the articles of impeachment from the House.

That is why tracking the changes in the president’s support in the polls is so important. Members of Congress will factor in what the public thinks as well as their own assessments of the president’s behavior.

For a variety of reasons, these are difficult measurements to make. As with all polling, the results depend to some extent on who is polled, what they are asked and when.

Whom are you asking?

New polls on the topic are being released every couple of days, and journalists are reporting on the latest shifts in opinions.

Many^[2] stories^[3] link attitudes on impeachment to the president’s level of support as measured in his approval ratings.

Attitudes on impeachment – and presidential approval – reflect the current state of political polarization in the U.S. Trump’s approval ratings are determined by strong support among Republicans and disapproval among Democrats; party identification^[4] has become the primary determinant of most attitudes in the U.S.

Most polls involving political issues focus on one of three target populations^[5]: adults 18 years of age and older, registered voters or likely voters.

Each of these is a subset of the previous one. Most importantly, each subset is generally more Republican-leaning as well^[6].

Presidential approval^[7] is commonly measured for all adults. But questions about candidate support^[8] – relevant as the U.S. works through the primaries and next year’s election approaches – are typically analyzed for registered or likely voters.

As these samples get smaller by definition and therefore more Republican, this usually mean greater support for Donald Trump and less for impeachment.

Political groups will sometimes produce press releases based on selected samples^[9], such as people “in Republican-held congressional seats and National Republican Congressional Committee targets.” These groups are obviously more likely to be partisan and unrepresentative of the national population.

Partisanship is strongly related to attitudes about impeachment. Consumers should look for differences in opinions by party identification in any single poll. Changing attitudes over time could be related to differences in the partisan composition of samples from different polls, even from the same polling organization.

Democrats are generally more willing to participate in polls while Republicans disproportionately refuse. This potential problem^[10] has appeared in some polls in the past. This makes it especially important to compare the partisan makeup of poll samples from different firms, as well as whether the partisan composition is changing over time for the same polling organization.

What are you asking?

But what does “support” for impeachment mean?

Pollsters are asking a variety of questions^[11] to measure that concept.

Some use the phrase “start an impeachment inquiry”^[12] while others use the phrase “impeach President Donald Trump and remove him from office^[13],” asking for a harsher judgment. “Support” is generally higher in response to the first question than in the second.

Due to the variations in question wordings used in different polls, the search for shifts in opinions over time can be sensitive to the wording used in the polls.

Some polls are asking about how high a priority impeachment and removal of Trump^[14] should be.

In some polls, the impeachment question includes an explicit “don’t know” option^[15], while in others it is a volunteered response. The proportion of “don’t know” responses is quite variable right now from poll to poll, but I expect it to decline over time as people are exposed to more news coverage.

Comparing results from different polls using different wording can be confusing and misleading, so it is best to compare polls conducted by the same organization at different points in time to see whether support for impeachment is stable, increasing or decreasing.

When are you asking?

Public opinion on impeachment will change as more information becomes available from additional sources. Those sources could include a second whistleblower^[16] and documents disclosed by the congressional inquiry^[17].

Polls about impeachment have been conducted^[18] since Robert Mueller began his investigation. However, the number of polls increased substantially^[19] after news about President Trump’s phone conversation with the president of Ukraine.

These opinions were pretty stable until that disclosure, with slightly more Americans opposing impeachment than not^[20]. Support for impeachment among Democrats was about twice as high as among Republicans, with self-described Independents split evenly.

Since the disclosure of the phone call, support for impeachment among all groups has increased. It increased the most among Democrats, but also among Republicans, and now a majority of Americans support impeachment.

In my view as a polling researcher^[21], the most consequential shifts in opinion on impeachment will be among Republicans and Independents, as Democrats are already pretty solid in their views.

References

1. ^{^} two-thirds of the Senate, now controlled by Republicans (www.msnbc.com)
2. ^{^} Many (www.cnn.com)
3. ^{^} stories (www.cnbc.com)
4. ^{^} party identification (www.pewresearch.org)
5. ^{^} target populations (blog.minitab.com)
6. ^{^} generally more Republican-leaning as well (fivethirtyeight.com)
7. ^{^} Presidential approval (www.gallup.com)
8. ^{^} questions about candidate support (www.people-press.org)
9. ^{^} produce press releases based on selected samples (www.nrcc.org)
10. ^{^} This potential problem (fivethirtyeight.com)
11. ^{^} asking a variety of questions (www.huffpost.com)
12. ^{^} “start an impeachment inquiry” (www.cbsnews.com)
13. ^{^} impeach President Donald Trump and remove him from office (www.newsweek.com)
14. ^{^} how high a priority impeachment and removal of Trump (www.politico.com)
15. ^{^} includes an explicit “don’t know” option (morningconsult.com)
16. ^{^} a second whistleblower (www.apnews.com)
17. ^{^} documents disclosed by the congressional inquiry (www.cnn.com)
18. ^{^} Polls about impeachment have been conducted (www.washingtonpost.com)
19. ^{^} the number of polls increased substantially (docs.google.com)
20. ^{^} with slightly more Americans opposing impeachment than not (fivethirtyeight.com)
21. ^{^} as a polling researcher (scholar.google.com)