We can learn from the failure of opinion polls

For centuries, ancient Greeks trekked to the Oracle of Delphi in an effort to find out the future. Pretty silly, right? If only they’d had professional polling companies to consult instead.

Over the last two years, pollsters have been catastrophically wrong more often than not. They didn’t just call the presidential election wrong. They got Brexit wrong too. And many of the presidential primaries. And our general election last year. And Israel’s general election a few months earlier. And the US midterm elections, in 2014.

Yet, despite their track record of failure, pollsters are still treated as reliable forecasters. Why?

Perhaps because, a few years ago, it seemed like polls were getting better and better. UK pollsters got the 2010 election spot on. US statistician Nate Silver was hailed as a genius for forecasting the right result in every state in the 2012 presidential election.

But opinion polling isn’t a hard science. Forecasts are shaped not just by the raw data but also by weighting adjustments for different groups. The methodology rests on assumptions that reflect what happened last time around – which is no guide, necessarily, to what will happen the next time.

In framing their assumptions, one thing pollsters seem to have repeatedly failed to account for is shy voters on the right. In the general election, the referendum, and the presidential election, polls underestimated their numbers, apparently because they were reluctant to disclose their voting intentions publicly.

Why is that? Could it be because Leave and Trump voters were dismissed as nasty, stupid and racist by establishment elites? By stigmatising large portions of the electorate, elites may have unwittingly created the statistical illusion of their own strength.

The anger at both Trump and Brexit derives, in part, from epistemic hubris. People were shocked to realise that the stats weren’t as reliable as they had supposed. That they didn’t know their own countries nearly as well as they had thought.

But the best way to react to this realisation isn’t rage, but humility. Recognising the limits of our knowledge ought to encourage us to investigate more.

Assuming total knowledge is part of what’s polarising Western societies. It’s the complacency that encourages people never to leave the Facebook filter bubble, or meet anyone who thinks – or votes – differently.

Understanding – rather than just railing against – seismic political shifts requires recognising that people can’t easily be reduced to statistics in a survey.