"Don't Trust the Election Forecasts" " Must-Read Justin Grimmer electionlawblog.org

I'm a political scientist who develops and applies machine learning methods, like forecasts, to political problems. The truth is we don't have nearly enough data to know whether these models are any good at making presidential prognostications. And the data we do have suggests these models may have real-world negative consequences in terms of driving down turnout.

Statistical models that aggregate polling data and use it to estimate the probability of each candidate winning an election have become extremely popular in recent years. Proponents claim they provide an unbiased projection of what will happen in November and serve as antidotes to the ad hoc predictions of talking-head political pundits. And of course, we all want to know who is going to win.

But the reality is there's far less precision and far more punditry than forecasters admit ...

In our paper, we show that even under best-case scenarios, determining whether one forecast is better calibrated than another can take 28 to 2,588 years. Focusing on accuracy " whether the candidate the model predicted to win actually wins " doesn't lower the needed time either. Even focusing on state-level results doesn't help much, because the results are highly correlated. Again, under best-case settings, determining whether one model is better than another at the state level can take at least 56 years " and in some cases would take more than 4,000 years' worth of elections ... .