Why the surprise? Labour’s poll slippage was predicted and is on trend

Ah, the panic. You can see it setting in as there is a blip downwards in the polls. Two consecutive polls show Labour and the Tories neck and neck, and we have a letter to the Guardian. As Uncut’s Atul Hatwal noted last week, there are now rumblings on the Labour right.

But while it is a perfectly respectable aim to ask Miliband to change course on a raft of policy areas, one cannot help but think it is a conversation we should have been having two or three years ago.

The good news is that this jitteriness is based on very little change in the actual prognosis.

To explain: political journalists are not, in the main, statisticians. Neither are politicians. And so both groups often subscribe to a mathematical fallacy, and it’s this: the polling of today is our best indicator of a general election result in X years time. It’s not. It’s a very rough guide which fails to account for the cycle of the parliamentary term, and in particular an opposition’s mid-term bounce. For the hard of maths, you can skip the next nerdy paragraph and trust us on this.

Our best guess – the expected value – of a general election vote-share lead is not equal to the value of our polling lead now. It’s equal to the value of our polling lead plus our expectation of how much that lead is going to change in between now and then. Trouble is, that second bit is crucial and historically, it’s not zero. In short, it is reasonable to argue that we shouldn’t just extrapolate today’s poll out to 2015 in a straight horizontal line. For an opposition party, it should be a line that inclines downwards.

A year ago, I wrote at Uncut about the research of Leo Barasi, from which one can conclude an approximate value of how much the polling lead would change over the last two year of a parliament, based on historical precedent: if you even out short-term fluctuations, on trend it would drop about 12%. Given that the lead – two years out – was then 11%, one might reasonably assume a 6% fall per year*.

Which is pretty much exactly what has happened.

Average polling now shows a 3% lead for Labour. And 5% is what one might expect from this model, one year out. It’s within a reasonable margin for error. We can also reasonably expect that it will be around -1% (call it zero, for the sake of argument) at the general election in a year’s time. It may fluctuate up and down a few per cent, but that’s about what we should expect to see.

So, on the one hand, we shouldn’t panic: the situation hasn’t suddenly changed.

On the other, there is another reading, from which we can take two lessons.

The first is that we should stop saying how there is a strong probability that Ed Miliband is going to be Prime Minister on the back of today’s opinion polls, whatever they say.

This has been the refrain on the left of the party almost non-stop since early 2012. But the assumption is not just restricted to the left: even my good colleague Stephen Bush, a centrist blogger of some repute, wrote in the NS two weeks ago of “an election that is overwhelmingly likely to send Ed Miliband to Downing Street”. But it is really not so. What’s more, we can argue that that has not been the case for at least the last two years, and probably well before.

The second is a lot more worrying: if we wanted to change the balance of probabilities to allow some margin for error in our success, we really should have started a long time ago. At this point, we seem to be less rainmaking and more at the mercy of cruel, fluctuating fate.

To play devil’s advocate, we can construct an alternative hypothesis for our polls to date: that we have been simply enjoying the fruits of a classic opposition mid-term bounce. That although we may feel we have been making the political weather from time to time, we have in fact more often merely been experiencing the near-random fluctuations in the polls that happen with the daily grind of politics. It may not be quite that bad, but we should reflect on the fact that it’s still a plausible explanation.

Now, the current prediction of Electoral Calculus is of a Labour majority of 40 seats based on current polling. For a start, even if we were to forget the trend effect of a further 6% loss of poll lead between now and the election, this is not any kind of a win Labour could want.

Anyone who disagrees need only take a look at the government of the unfortunate John Major, cursed with a majority of 21 and a constant hostage to his backbenchers. Not forgetting by-elections, which incumbents mostly lose, and defections, which make matters worse.

Furthermore, as pollster Deborah Mattinson noted at the start of this year, no party has ever won a majority from here. And if we do take into account this further above-mentioned loss of 6% on historical trend, we would easily be looking at some kind of a hung parliament with the Tories the largest party. Rather like 2010, in fact. A few more per cent short-term fluctuation – rain on polling day or whatever – and that could be a Tory majority. And that is if there is no additional trend downward between now and 2012, which might happen as a simple result of economic recovery or a collapse in the UKIP vote after the Euros.

In any event, as John Rentoul observes from the same Electoral Calculus front page, current polling is already hovering perilously close to a hung parliament anyway, even on the basis of today’s average poll.

It is quite simple: we cannot tell which of these scenarios will take place, and quite possibly will not until the poll itself. We are still on a knife-edge. The game has not changed.

But for Labour to have any kind of security about winning, let alone a majority, it needs to. Sharpish.

*Obviously we assume the downward trend is linear, but even if it is not (e.g. a steeper fall first then a shallower gradient, or vice versa), sooner or later it is likely to tend towards that linear trend line.

UPDATE: I am indebted to Ben Mitchell for pointing out this piece of research by Stephen Fisher of Oxford University – it is the first piece of research I have seen (apart from Leo Barasi’s) which attempts to reconcile historical polls with election results, although I am sure there are others.

Essentially it shows a similar, radically different result from what most pollsters are showing, including Electoral Calculus, only a little worse for Labour. It fully predicts a Tory win (313 seats vs 279 seats for Labour), although not a Tory majority. Even if the analysis is correct, I would still say it is too close to call, but let’s see how things look in six months.

This post first published at Labour Uncut and selected for Progress’ What We’re Reading