Why the polls were wrong - and will never be right again

We’re sorry, this feature is currently unavailable. We’re working to restore it. Please try again later.

Advertisement

This was published 3 years ago

Opinion

Why the polls were wrong - and will never be right again

A widely predicted "blue wave" has not eventuated for Joe Biden. This is a failure of mainstream opinion polling.

By Salvatore Babones

The 2020 US elections got off to a roaring start as Joe Biden scored a thumping victory in Dixville Notch, New Hampshire. His five votes there, in the first town in the country to report, launched his "path to victory" with a perfect 100 per cent result. Unfortunately for Biden and the Democrats, the whole country is not Dixville Notch, though you might not know that if your only source of political information were the mainstream opinion polls.

The consensus of major national polls tracked by FiveThirtyEight gave Biden a stunning 8.4 percentage point lead on November 1. Reality sunk in the minute Florida started reporting its election results, indicating a narrow victory for the Donald in what is legally his home state. The consensus of Florida state polls tracked by FiveThirtyEight suggested that Biden would cruise to victory there, as in the rest of the country. How could they be so wrong – again?

The blue wave didn't arrive for Joe Biden.

The blue wave didn't arrive for Joe Biden.Credit: Getty

There was one poll that got Florida right – again. The final Rasmussen Poll gave the Sunshine State to Donald Trump by 1 point. Rasmussen was also the major poll that came closest to predicting the outcome of the 2016 election. Its secret? Be anti-social. Instead of using human poll-takers like the other major polls, Rasmussen uses a pre-recorded voice.

Human beings are social animals, and that's just as true when they're answering a telephone survey as when they're arguing on Twitter. In an America where Trump supporters are routinely called "racist" (or worse), it's no surprise that many of them prefer to keep their political leanings to themselves.

Often ridiculed as the "shy Trump voter" hypothesis, the technical name for this phenomenon is social desirability bias. It's the most obvious explanation for the fact that while other pollsters predicted a Biden blowout, the Rasmussen poll showed a narrow lead for Trump.

No social relationship, no social desirability bias. Or at least, that's the theory. But there's another reason why all the polls are more error-prone than ever, including Rasmussen's. It's the dirty little secret the pollsters don't want you to know: virtually no one is answering their questions.

Model error is here to stay. In fact, it's likely to get worse.

In the age of the mobile phone, very few people answer calls from unlisted numbers, and even fewer want to talk to a pollster – who, for all they know, may be a fraudster in disguise. The Pew Research Centre reports that its response rates have plummeted from 36 per cent two decades ago to just 6 per cent now. And Pew is a not-for-profit outfit that doggedly attempts to contact every sampled phone number at least seven times. Commercial polling firms don't have that luxury.

No major commercial polling company is brave enough to reveal its response rate. Rumours are that they're down to about 3 per cent. That's a very thin foundation on which to predict a presidential election. The tiniest inconsistency between the characteristics of that 3 per cent and those of the electorate as a whole could invalidate the entire industry.

Advertisement

The pollsters do their heroic best to model the likely behaviour of the masses from the self-reports of a few phone-answerers, but all such models are approximations. They inevitably introduce error. Model error may be even bigger than the sampling error that goes into calculating the "error margins" that are often reported alongside polling data. Or it may not be. No one knows but the pollsters, and they're not saying.

Loading

Social desirability biases may diminish once the popularising figure of Donald Trump eventually departs the political stage, but model error is here to stay. In fact, it's likely to get worse.

The models that help pollsters extrapolate from the 3 per cent to the whole electorate rely heavily on exit poll data: in-person surveys conducted on election day just after people have voted. But with the majority of votes being cast by mail in this election – 102 million of them – the 2020 exit polls will be next to useless. As a result, future poll predictions will stray further and further from reality.

Political polling is good fun, but it should be treated more as entertainment than as serious politics. By all means, seek out all the data. Then make your own judgment. It's likely to prove as accurate as any pollster's, and if you happen to turn out right, the "I told you so" will be all your own.

Salvatore Babones is an American political sociologist and an associate professor at the University of Sydney. He is the author of The New Authoritarianism: Trump, Populism, and the Tyranny of Experts.

Trump Biden 2020

Understand the election result and its aftermath with expert analysis from US correspondent Matthew Knott. Sign up to The Sydney Morning Herald's newsletter here, The Age's here, Brisbane Times' here and WAtoday's here

Most Viewed in World

Loading