How polls are a threat to US democracy

Andrew Therriault was certainly one of those readers. Therriault was director of data science for the Democratic National Committee for two years before leaving in June 2016. Speaking on the phone, he said those forecasts might have helped Trump win: When its this close, anything that could even plausibly have an effect could be a reason why the outcome happened. I dont think it had a sizable effect. But theres a possibility that a not-so sizable effect was sizable enough.

Therriaults reluctance to endorse the underdog effect theory as the only explanation makes sense. There are other equally compelling explanations like bandwagon effect theory, which would argue that forecasts showing Clinton would win helped voters rally around her rather than Trump.

Seeming to hedge his bets, Trumps rhetoric during the election made use of both appeals to voters; the candidate would routinely claim that he was worth supporting because he was a winner and because he could lose.

Trumps strategy could be understood using a cruder political science theory: ass-covering. A similar theory could be used to understand Silvers final election prediction, in which he said that Clinton would probably win but that she could also lose to Trump, or win by a landslide. Similarly, once Trumps victory was confirmed, Silver described the result as shocking, but not surprising.

After 8 November, Silver seems very, very unwilling to say that FiveThirtyEight was wrong, insisting instead that the sites forecast was more accurate than others. Theres some truth to that claim. In predicting the popular vote, Silver had projected that Clinton would win 49% and Trump 45% (in the end, Hillary Clinton won 48% and Trump won 46%). And, when looking at state-level results, Silvers model guessed better than eight other forecasts.

But many readers werent interested in those statistical nuances they simply had a question: Who will be the next US president? and believed, based on forecasts, that the answer was Hillary Clinton.

But polling is inaccurate, and its getting worse something even political forecasters admit.

Speaking on WNYC days after the result, Silver told the host: To me, its a miracle that the polls arent off by more. The Upshot had a similar line in its postmortem, writing: It was the biggest polling miss in a presidential election in decades. Yet in many ways, it wasnt wholly out of the ordinary.

The Americans who did cast a ballot on election day were not simply black or white, male or female, wealthy or poor. They included middle-class Native American women and low-income college-educated black men it is very hard to capture that demographic complexity in the 1,000 respondents you are contacting in a poll. American society is only becoming more diverse, and fewer Americans today are willing to talk to a pollster for up to 30 minutes for nothing in return.

In the 1980s, around 60% of those contacted to take part in a poll would do so. By 2012, response rates had fallen to 9%, and by 2016, Therriault told me that when he was with the DNC, I would have killed to get 9%. Those falling response rates are problematic. They make conducting polls an expensive business because more slammed phones mean more people are needed to make those calls.

Some elections pollsters used a panel instead, contacting the same thousand or so individuals every week to reduce those costs and get a more accurate picture of how their opinion was changing over the course of the election.

But pollstersalso found that after a candidate had received negative media coverage (eg for having a private email server or for being accused of sexual assault for the 12th time), respondents simply wouldnt pick up the phone to avoid being questioned about the scandals and whether they would continue to support their candidates. That made it hard for pollsters to know what those people would do in voting booths.

Forecasts rely heavily on those polls, but they also use historical data about factors such as turnout and voter behavior. In the past, they have been a good guide. The 2012 presidential election wasnt so dramatically inconsistent with the 2008 election which is one reason why Silver and others correctly predicted the outcome. But, to state the obvious, 2016 is no ordinary year. And, in the absence of using the past as a guide, its important to look around at the present. And the present can look very different depending on who you are in America.

In that same WNYC interview, aired on 11 November, Silver said of Trump:

I, I think naively, thought that a candidate who made those sorts of appeals that were often based on populism, nationalism and racial appeals, you know, I thought that was not something that a sophisticated country like ours would go for in large enough numbers for him to win the primary, let alone become president.

That view may not have been shared by non-white polling analysts, of which there are fewin the American media.In the year and a half that I worked at FiveThirtyEight, I was the only non-white staff writer there. Unlike for Silver, the fact that racism was alive and well in America was no surprise to me. That affected my skepticism when reading polling numbersI was convinced that they were off and underestimated Trumps support (although exit polling data suggests that the poorest Americans might not have voted for Trump, as I thought at the time).

The inaccurate polling numbers did not just affect what forecasters wrote. Based on her campaigns own internal data, Clinton decided not to visit Maine, Wisconsin or Minnesota, and it was only in the last week of the race that she campaigned in Michigan. In the end, she lost Wisconsin and Michigan, and only narrowly won Maine and Minnesota.

Political polling has repeatedly proved unreliable recently in Britains Brexit vote, the 2015 UK election and the Israeli election the same year. But even when the numbers are correct, are polling calculations healthy for democracy? In the US, the media and the publics shared obsession with polling fuelled an entire election season, including three televised candidate debates, that virtually ignored policy. Headlines focused instead on scandals and their potential effect on the horserace numbers.

Midterms are two years away, so there might be a pause before forecasting dominates the US media again in quite the same way. But polling isnt going anywhere if Trumps candidacy is any guide, Trump as president will obsessively quote his approval ratings and use polling numbers to craft populist policies.

It will take months to gather the voter file data necessary to accurately understand the inaccuracy of the polls in 2016. But its not too early to question whether polls threaten democracy.