Submitted by Douglas
Does it sound familiar? Just rewind to 2013 and the Barbados general election. A particular pollster/political guru had the DLP to get whitewashed; or as we would say in local lingo – ‘catspraddled’. Well, we all know the outcome; the DLP was returned to office, and that pollster/pundit, who had egg all over his face, now spends every waking hour excoriating the government on air and in print. Big Joker!
Heard him a week or so ago pontificating that the Scottish people deserve better and a ‘Yes’ vote would shake-up Whitehall. However, when a caller tried to put him on the spot and asked him to predict the outcome, he was somewhat non-committal and hedging; probably remembering how he and the Belle Tower made a fool of themselves in 2013. I am waiting to hear that pollster on last night’s result. I am sure he will do a volte-face; that is his accustomed mo.
Scotland’s Referendum is an object lesson for the self-righteous. People Power always wins out. For the record, I was hoping and wishing for a ‘No’ result. It makes sense. Alex Salmond wanted to have his cake and eat it too – aka- the best of both worlds.
Scotland’s ‘No’ Vote: A Loss for Pollsters and a Win for Betting Markets
by Justin Wolfer
Thursday’s Scottish referendum was interesting not just for what it said about Britain, but also for what it said about the state of political forecasting. I’m calling it a loss not only for the pro-independence movement — the “No” campaign won 55.3 percent of the vote — but also for the pollsters.
To be fair, I should start by acknowledging that most of the election-eve polls correctly predicted a majority No vote, but they all underestimated the margin, and many missed by quite a lot. The polls were volatile; they often gave conflicting signals; and it took them until the last few weeks to even start to suspect that this would be a close race. The major polls in the past week ranged from a 6-point lead for the Yes vote to a 7-point lead for the No vote.
And this wide range wasn’t because of wild fluctuations in public opinion. It was the result of two surveys that were taken within a day of each other.
The prediction markets, on the other hand, yielded much more reliable forecasts. Despite the demise of Intrade, these markets remain extremely active, and over at Betfair, bettors rated the chances of a No vote at around 80 percent, an estimate that remained remarkably stable over the past week, fluctuating by only a few points.
British bookies were laying similar odds. According to The Financial Times, a Ladbrokes spokesman argued earlier this week that the referendum would be the biggest political betting event in history, noting that his firm had taken more money in bets than the last British general election and American presidential election combined. Betting on the likely winning margin also suggested that the No vote was most likely to win by around 4 points. Yes, bettors underestimated the winning margin, but they were still closer than the election-week polling average.
My own research with Microsoft’s David Rothschild suggests that pollsters could do a better job if they learned from prediction markets. Instead of focusing on whom people say they plan to vote for, ask them instead to focus on who they think will win. Typically, asking people who they think will win yields better forecasts, possibly because it leads them to also reflect on the opinions of those around them, and perhaps also because it may yield more honest answers.
It’s an idea with particular relevance to the case of the Scottish referendum. As Stephen Fisher, an associate professor of political sociology at the University of Oxford, has noted, there is a historical tendency for polling to overstate the likelihood of success of referendums, possibly because we’re more willing to tell pollsters we will vote for change than to actually do so. Such biases are less likely to distort polls that ask people who they think will win. Indeed, in giving their expectations, some respondents may even reflect on whether or not they believe recent polling.
And in this election, too, voters’ expectations yielded a much clearer signal. A recent IPSOS/More poll showed that voters’ intentions were so evenly balanced as to be within the margin of error, even as the share of the population who expected the No vote to win held a robust 11-point lead over those expecting a successful Yes vote. Lesson: The electorate knew who would win, even as most pollsters failed to ask them.
There’s also a lesson here that’s relevant to the debate about the importance of whether to adjust polling results for “fundamentals,” as Leo, the Upshot’s election forecasting model, does. If you take polling results literally, you can be forced to predict a win even when the polling points in an unlikely direction (as when it suggested Herman Cain would win the Republican nomination). But prediction markets, like sophisticated statistical models, can also take into account other information that would temper such a prediction. In the Scottish case, the “fundamental” worth noting is that it’s hard to get a referendum to pass.
But my real beef with the polls concerns how badly they’ve failed at making useful long-run forecasts. Predicting what will happen tomorrow is never that hard, while predicting what will happen in several months or years is not only difficult but also much more useful.
Thanks to Oliver Lee, an amateur psephologist who has tracked the betting odds day by day, and the good folks at UK Polling Report, who have tracked the major polls, we can compare the very different narratives they’ve given over the longer run. Don’t be fooled by the fact that very similar numbers have a very different meaning: When a pollster says you’re going to win only 30 percent of the vote, you’re set to lose in a landslide, whereas when a market says you have a 30 percent chance to win, it’s saying you’ve got a real chance. And that’s basically what happened.
Throughout most of the campaign, few gave the pro-independence supporters a chance. From the early surveys in 2012 and 2013, all the way through to those run as recently as June of this year, most polls registered support for the Yes campaign as running in the mid-to-low 30s. A few nudged above 40 percent, but far more registered support in the 20s. It’s no surprise, then, that the news media largely ignored the referendum, and that pundits basically wrote it off as a sideshow.
It took a shocking YouGov survey earlier this month, which pointed to a winning Yes vote, to finally awaken pundits from their slumber. But in fact, throughout the entire campaign, prediction markets judged the race as being very much alive, and on average throughout 2012-13, the average betting odds suggested around a one-in-four chance that the pro-independence groups would succeed. True, the independence campaign ultimately failed, but this seemed a better assessment of the actual risks than polls suggesting an electoral blowout.
Ultimately, the Scots didn’t end up voting for change on Thursday. But perhaps this election will change political punditry instead. I’m betting that next time Scottish independence comes up for a vote, political forecasters will rely more on prediction markets or on surveys asking voters what they expect will happen.
Justin Wolfers is a senior fellow at the Peterson Institute for International Economics and professor of economics and public policy at the University of Michigan.
The blogmaster invites you to join the discussion.