The 4 key questions for pollsters
1. Do respondents match the electorate demographically in terms of sex, age, education, race, etc.? (This was a problem in 2016.)
2. Do respondents match the electorate politically after the sample is adjusted by demographic factors? (This was the problem in 2020.)
3. Which respondents will vote?
4. Should the pollster trust the data?
To show how the answers to these questions can affect poll results, I use a national survey conducted from October 7 - 14, 2024. The sample included 1,924 self-reported registered voters drawn from an online, high-quality panel commonly used in academic and commercial work.
After dropping the respondents who said they were not sure who they would vote for (3.2%) and those with missing demographics, the unweighted data give Harris a 6 percentage point lead - 51.6 % to 45.5% - among the remaining 1,718 respondents.
*Adjusting for demographic factors
*Adjusting for political factors
*Adjusting for likely voters
*Do I trust this data?
After all of this, there is still a critical limitation. Even if we correctly predict what the 2024 electorate will look like in terms of demographics and partisanship, our adjustments only work if the voters who took the poll have the same views as similar voters who did not take the poll. This is a foundational assumption of polling.
The performance of polls thus depends on the opinions of both voters and pollsters in ways that are often hard to discern. This election year, are the polls virtually tied because voters are tied, because pollsters think the race is tied, or both?
We would all do better to temper our expectations about pre-election polls. It's impossible to ensure that the polls will reliably predict close races given the number of decisions that pollsters have to make. And it's often hard for consumers of polls to know how much the results reflect the opinions of the voters or the pollsters.
