@Anneinak draws our attention to a fantastically interesting paper by Alan Renwick (who teaches at one of my old stomping grounds, the University of Reading) in which we get a pretty persuasive argument. The conclusion of the argument is that in major referenda like the one coming up on whether or not Britain will remain in the EU, support for the "yes" position tends to decrease quickly in the months and weeks leading up to the referendum itself. This bar chart (from Renwick's piece) does a pretty good job of laying out the evidence[c]:
I feed the numbers into a spreadsheet (though I had to do a little guesstimating because of the lack of granularity along the vertical axis), and it looks like the support for the yes/change side of the referenda drops in the month or so before the event on average by 11.6% but with high variability: 14.7%. The whole thing looks something like this:
Let's take this at face value and see what we find. We could start with the Financial Times, which has some nice meta-polling data, represented in the first graph above[d]. An unweighted average of the polls taken so far in March results in 0.46 for staying in the EU, 0.42 for leaving it, and 0.12 undecided. Let's leave the undecided folks out of the picture, just as Renwick does. (I assume that the bias toward to status quo is largely a result of undecided types coming in at the last moment with a vote for keeping things as they are. I can't recall whether Renwick's paper addresses this matter, and I'm too busy/lazy to look through it again.) That means that we're looking at a probability of about 0.52 (=0.46/(0.46+0.42)) of staying and a probability of about 0.48 (=0.42/(0.46+0.42)) of leaving the EU. It looks like what we're supposed to expect on average a -11.62% drop in support for the a "yes" result in the referendum (i.e., a result that leads to a change in the status quo). In this case, that's a drop in support for the Brexiting to 0.425 (0.48-(0.48*0.1162)) and, conversely, an opposition to Brexiting of 0.575 (1-(0.48-(0.48*0.1162))). This would be an overwhelming win for the forces of sanity over the eurosceptics, in my humble opinion. Given the standard deviation is 0.06 (=14.69 % of 0.425), the range of expectation looks something like this:
So if this model is correct, then we should expect a mere 10% (or so) chance of a vote to leave the EU. But is the model correct? Several reasons for doubt concerns the data points themselves, though there are ways to try to account for these concerns.
First, many of the referenda took place in nations with very small (by global standards) populations, and small populations make polling very difficult. Sweden had only 8 million citizens in 1980. Ireland had only 5 million citizens in 1986. I expect that polls for a country of 65 million citizens in 2016 are likely to be a bit more predictive, all other things being equal. I suppose that the best way to try to take this into consideration would be to discount polls taken in smaller countries. As I have no principled way to do this, I won't try. But I'd be pleased as punch if someone else did.
Second, several countries had many referenda. Bloody Ireland! How many referenda do you need, mates? Yes, democracy and all that, I suppose. But still.... Anyway, we can easily control for that by removing multiple referenda in a single country and replacing them with a single average for referenda in that country. The results look like this:
As is obvious, the results are less dramatic, though only slightly so. We should expect 5.2% less support for Brexit at the polls on June 23, down to 42.8%, with a standard deviation of 6.3%. That leaves about a 13% chance of the pro-Brexit forces getting for than 50% of the vote. Using Olympic rules (tossing out the highest and lowest scores) tames the data even further. We should expect 4.9% less support for Brexit at the polls on June 23, down to 43.1%, with a standard deviation of 5.4%. But it still means less than an 11% chance of Brexit. I assume that Renwick is himself is making some data-shaping modifications similar to the one's I'm playing with here, since he himself is (as of the date of this writing) giving Brexit a 13% chance, as you can see from his blog.
Another reason for doubt concerns the similarity between the kind of referenda used to build the model and the referendum to be voted upon in late June. In some cases, the referenda are very similar to what's going on in the UK right now, viz., the Danish referendum of 1992[e]. But the 1980 Swedish referendum on nuclear power and the 1997 Welsh referendum on creating an assembly with devolved powers[g] are less so. The way to handle this, I think, would be to create a similarity scale which would allow us to discount the importance of referenda that are less like the Brexit referendum on the horizon. But I work (more or less) for a living (again, more or less) and don't have the time on my hands right now.
Finally (not really finally, but this is the last thing that came to mind while typing this up), there's the matter of local issues that might matter. I don't know, e.g., much about Swedish elections (and that's an understatement), but perhaps there are typical patterns of voting that are true in Sweden but are not true in the UK that have relevance here. Maybe Swedish voters regularly are especially risk averse at election time but not so prior to the election, while UK voters aren't. Who the hell knows? But if I were going to put all my eggs in this particular basket, I'd want to find out.
All that said, I feel fairly confident that this tendency to shift toward the status quo is real, and it's made me a lot more confident that Britain will vote to remain in the EU. So I'm moving from 61% to 71% likelihood of the UK voting against Brexit.
@Anneinak draws our attention to a fantastically interesting paper by Alan Renwick (who teaches at one of my old stomping grounds, the University of Reading) in which we get a pretty persuasive argument. The conclusion of the argument is that in major referenda like the one coming up on whether or not Britain will remain in the EU, support for the "yes" position tends to decrease quickly in the months and weeks leading up to the referendum itself. This bar chart (from Renwick's piece) does a pretty good job of laying out the evidence[c]:
(https://pythagorassite.files.wordpress.com/2016/03/renwick___don_t_trust_your_poll_lead___in_cowley_and_ford__2014__pdf.png?w=978)
I feed the numbers into a spreadsheet (though I had to do a little guesstimating because of the lack of granularity along the vertical axis), and it looks like the support for the yes/change side of the referenda drops in the month or so before the event on average by 11.6% but with high variability: 14.7%. The whole thing looks something like this:
(https://pythagorassite.files.wordpress.com/2016/04/screenshot_4_1_16__6_37_pm.png?w=920)
Let's take this at face value and see what we find. We could start with the Financial Times, which has some nice meta-polling data, represented in the first graph above[d]. An unweighted average of the polls taken so far in March results in 0.46 for staying in the EU, 0.42 for leaving it, and 0.12 undecided. Let's leave the undecided folks out of the picture, just as Renwick does. (I assume that the bias toward to status quo is largely a result of undecided types coming in at the last moment with a vote for keeping things as they are. I can't recall whether Renwick's paper addresses this matter, and I'm too busy/lazy to look through it again.) That means that we're looking at a probability of about 0.52 (=0.46/(0.46+0.42)) of staying and a probability of about 0.48 (=0.42/(0.46+0.42)) of leaving the EU. It looks like what we're supposed to expect on average a -11.62% drop in support for the a "yes" result in the referendum (i.e., a result that leads to a change in the status quo). In this case, that's a drop in support for the Brexiting to 0.425 (0.48-(0.48*0.1162)) and, conversely, an opposition to Brexiting of 0.575 (1-(0.48-(0.48*0.1162))). This would be an overwhelming win for the forces of sanity over the eurosceptics, in my humble opinion. Given the standard deviation is 0.06 (=14.69 % of 0.425), the range of expectation looks something like this:
(https://pythagorassite.files.wordpress.com/2016/04/normal_distribution_calculator_with_step_by_step_explanation.png?w=978)
So if this model is correct, then we should expect a mere 10% (or so) chance of a vote to leave the EU. But is the model correct? Several reasons for doubt concerns the data points themselves, though there are ways to try to account for these concerns.
First, many of the referenda took place in nations with very small (by global standards) populations, and small populations make polling very difficult. Sweden had only 8 million citizens in 1980. Ireland had only 5 million citizens in 1986. I expect that polls for a country of 65 million citizens in 2016 are likely to be a bit more predictive, all other things being equal. I suppose that the best way to try to take this into consideration would be to discount polls taken in smaller countries. As I have no principled way to do this, I won't try. But I'd be pleased as punch if someone else did.
Second, several countries had many referenda. Bloody Ireland! How many referenda do you need, mates? Yes, democracy and all that, I suppose. But still.... Anyway, we can easily control for that by removing multiple referenda in a single country and replacing them with a single average for referenda in that country. The results look like this:
(https://pythagorassite.files.wordpress.com/2016/04/screenshot_4_1_16__9_47_pm.png?w=920)
As is obvious, the results are less dramatic, though only slightly so. We should expect 5.2% less support for Brexit at the polls on June 23, down to 42.8%, with a standard deviation of 6.3%. That leaves about a 13% chance of the pro-Brexit forces getting for than 50% of the vote. Using Olympic rules (tossing out the highest and lowest scores) tames the data even further. We should expect 4.9% less support for Brexit at the polls on June 23, down to 43.1%, with a standard deviation of 5.4%. But it still means less than an 11% chance of Brexit. I assume that Renwick is himself is making some data-shaping modifications similar to the one's I'm playing with here, since he himself is (as of the date of this writing) giving Brexit a 13% chance, as you can see from his blog.
(https://pythagorassite.files.wordpress.com/2016/04/first_forecast_for_the_brexit_referendum___elections_etc.png?w=978)
Another reason for doubt concerns the similarity between the kind of referenda used to build the model and the referendum to be voted upon in late June. In some cases, the referenda are very similar to what's going on in the UK right now, viz., the Danish referendum of 1992[e]. But the 1980 Swedish referendum on nuclear power and the 1997 Welsh referendum on creating an assembly with devolved powers[g] are less so. The way to handle this, I think, would be to create a similarity scale which would allow us to discount the importance of referenda that are less like the Brexit referendum on the horizon. But I work (more or less) for a living (again, more or less) and don't have the time on my hands right now.
Finally (not really finally, but this is the last thing that came to mind while typing this up), there's the matter of local issues that might matter. I don't know, e.g., much about Swedish elections (and that's an understatement), but perhaps there are typical patterns of voting that are true in Sweden but are not true in the UK that have relevance here. Maybe Swedish voters regularly are especially risk averse at election time but not so prior to the election, while UK voters aren't. Who the hell knows? But if I were going to put all my eggs in this particular basket, I'd want to find out.
All that said, I feel fairly confident that this tendency to shift toward the status quo is real, and it's made me a lot more confident that Britain will vote to remain in the EU. So I'm moving from 61% to 71% likelihood of the UK voting against Brexit.
--
Endnotes [a] and [b] removed.
[c] http://centaur.reading.ac.uk/38059/1/Renwick,%20'Don't%20Trust%20Your%20Poll%20Lead',%20in%20Cowley%20and%20Ford%20(2014).pdf
[d] https://ig.ft.com/sites/brexit-polling/
[e] https://en.wikipedia.org/wiki/Danish_Maastricht_Treaty_referendum,_1992
[f] https://en.wikipedia.org/wiki/Swedish_nuclear_power_referendum,_1980
[g] https://en.wikipedia.org/wiki/Welsh_devolution_referendum,_1997