The Asterisk on Kamala Harris’s Poll Numbers

The pollsters believe they’ve learned from mistakes of 2020. Of course, they believed the same thing last time.

By Gilad Edelman

Aug 22, 2024 07:07 PM

One month after her entry into running for the presidency, Kamala Harris has a tiny but clear advantage against Donald Trump, if the polls are to be believed. However, after two presidential elections polls may seem like an odd thing to do.

The 2016 presidential election is in the collective memory as possibly the most notorious polling mishap ever, however 2020 was a far more shrewd choice. The polls from four years ago underestimated the strength of Trump’s popularity, even though they accurately predicted an eventual Joe Biden win. A detailed review of the American Association for Public Opinion Research found that the polls for 2020 were among the least accurate in a long time, exaggerating Biden’s lead with an average 3.9 percentage points nationwide as well as 4.3 percentage points for the states in the final couple of weeks leading up to the campaign. (In 2016 however the national polling forecast the margin of popular vote for Hillary Clinton very accurately.) In the report by the New York Times, Biden led by 10 points in Wisconsin but was able to win by a fraction of a point. He also won Michigan by 8 points and then won by three points; he was ahead in Pennsylvania by 5 points and was able to win by 1. In the moment of time, Harris is up in all three states, however in a smaller amount than Biden was. An error of this magnitude would suggest that she’s actually lower and is in danger of losing her seat in the Electoral College.

The pollsters realize they screwed the 2020 election badly. They’re optimistic the lessons they learned are applicable to their next attempt. Of course, they believed the same thing last time.

What caused the polls to get worse from 2016 until 2020, while everyone was watching? In the aftermath of Trump’s surprise 2016 victory, the public-opinion-research industry concluded that the problem was educational polarization. When pollsters would have made it a decision to include sufficient whites without university degrees in their surveys they wouldn’t have misjudged Trump in such a way. In the race for 2020 they concentrated on rectifying this error.

It was not a success. While polls in 2020 showed more non-college educated white voters, they proved to be more disproportionately non-college educated white voters who favored Biden. The consensus now is the fact that Republican people are more likely not to take part in polls at all regardless of their education levels. (To say it in a more scientific way the partisanship of a voter is in direct correlation with the willingness to take an opinion, if only in the event that Trump has been a candidate.) Don Levy, the director of the Siena College Research Institute, that conducts polls on behalf of The New York Times The phenomenon is described as “anti-establishment response bias.” The more a person is distrustful of establishment institutions that are mainstream, like the media and pollsters it is more probable they will be to support Trump.

Learn: The voting crisis is a disaster to American democracy

Levy said to me that in 2020, the staff who were on the phone for Siena frequently complained of being verbally abused by shady Trump supporters. “In plain English, it was not uncommon for someone to say, ‘I’m voting for Trump–fuck you,'” and then to hang up before taking the remainder of the questionnaire according to him. (So what’s that “shy Trump voter” hypothesis.) In 2020, these votes were not considered. In 2019 they’re. Levy said to me that incorporating the “partials” in 2020 would have erased more than half of the error rate in Siena’s study.

There’s still the other half. Another problem is that the majority of pollsters have opted out of live calls to go with polls that are online or via text which means they don’t have angered partials to count. So, pollsters are experimenting with different variations of the same method to get more likely Trump voters into their database. If a smaller percent of voters who are Republican-leaning are responding to polls perhaps you need to make contact with a bigger amount of them.

It may sound like a simple thing however it is an uncomfortable change for the business. Public pollsters have historically adhered to the categories that are politically neutral that are used in censuses when assembling or weighing their sample: age, race, gender, and other such categories. The idea suggested that if one created your sample according to different demographics–if you counted the correct number of white people as well as Latinos as well as evangelicals and atheists as well as women and men, an precise picture of the country’s the balance of power would be evident.

“In 2016, the feeling was that the problem we had was not capturing non-college-educated white voters, particularly in the Midwest,” Chris Jackson who is the head of U.S. public polling at Ipsos said to me. “But what 2020 told us is that’s not actually sufficient. There is some kind of political-behavior dimension that wasn’t captured in that education-by-race crosstab. So, essentially, what the industry writ large has done is, we’ve started really looking much more strongly at political variables.”

The pollsters of the past were reluctant to incorporate such variables due to the fact that predicting the composition of voters is a hazy science. If it wasn’t there was no need for polls at all in the first place. However, after the failure of 2020, the polling industry has no other option. “There’s no avoiding coming up with a hypothesis as to the composition of the electorate,” Matt Knee, who manages polls and analysis for Republican campaigns and me. “Choosing to throw up your hands on the most important predictor of how someone’s going to vote, and saying ‘That’s not a valid thing to include in my hypothesis,’ just doesn’t make sense.”

Read The return of the people machine

Some pollsters are using state-level voter lists to gain an appropriate mix between Democrats and Republicans within their sample. Another method is to utilize “recalled vote”: asking those who voted in 2020 and ensuring that the composition of the respondents is consistent with actual results. (If states voted 60 percent in support of Trump however, if, for example, only 50 percent of respondents indicate that they voted Trump then the pollster would orally call more Trump voters or weigh their responses more in the aftermath of the poll.)

Each method has its own limitations. The registration of parties doesn’t align perfectly with voter preferences. Certain states, like Michigan and Wisconsin do not even have party registrations, which means pollsters must rely on models of political partisanship that is based on factors like gender, age or religion. The recall vote could be more uncertain: A lot of people make up or deny their voting records. A lot of people claim that they voted but they actually did not, and a few voters who voted for a loser claim that they did vote for the winner. Levy said that when Siena tried using a recalled votes from 2022 onwards, this caused certain result less exact.

But, pollsters can see some evidence of optimism. “People who told us they voted for Trump in 2020 are responding at the same rates as people who told us they voted for Biden in 2020,” said Jackson from Ipsos and that “suggests we’re not having a really strong systemic bias.” The New York Timespoll master Nate Cohn made a similar observation in an earlier interview with The New Yorker: Democrats were much likelier to respond to Timespolls in 2020, however the last time around, “it’s fairly even–so I’m cautiously optimistic that this means that we don’t have a deep, hidden non-response bias.” A different thing about 2020 and today the present: There’s no pandemic. Many researchers believe Democratic electors were more likely respond to questions in 2020 since there was a higher likelihood to Republicans to be home, with nothing to do.

What’s evident at this point is that the race is not far off as Harris appears to be in a better situation over Biden was. Natalie Jackson, a Democratic pollster at GQR Research, told me that if Harris’s results are merely the result of more energized Democrats having the energy to take polls seriously and take part in polls, then Democrats would see an identical increase in general congressional polls. However, the fact that they’re not indicates that the shift is real. “Trump’s numbers haven’t moved,” Jackson stated. “This is all shifting from third party or undecided to Democrat.”

Similar to Olympic athletes Pollsters in the political world have four years of perfecting their skills, but they don’t know whether their efforts were effective until it’s time to make any changes. The bias in nonresponse that plagued the 2020 polls isn’t easy to correct. According to the definition of pollsters, they know nothing about people who do not talk to them. If Trump beats the polls time, it’s because, despite all the years there is something about his supporters that remains undiscovered.