Post Election Round-up; How Accurate Was My Calculator?

As outlined at the very inception of this blog, it’s no easy task predicting local and regional election results on the basis of national polls. The traditional method, Universal Swing, simply assumes that the change in votes will be the same everywhere – e.g., if a party is down 3% nationally, it’ll be down 3% in every constituency and region. In reality, we know that won’t be the case. In trying to account for that with my model, I used regional swings based on approximating 2015 Westminster results to Holyrood regions and seeing how the result in each region varied from the national result.

Now, as established yesterday, my actual final prediction was off significantly. But so too were the polls that informed that prediction (more on that in the next post). To see how how accurate my model was in and of itself, I’ve plugged the actual national vote shares into it.

Model vs Actual

Looking just at the headline figures, the calculator is spot on for three of the five parties in parliament – Labour, Greens and the Lib Dems. It over-estimates the SNP by 4 seats and therefore under-estimates the Tories by 4. It’s not a million miles off, then, giving me a national success rate of 121 of 129 MSPs. But what about individual regional results?

Central Real

In Central, it gets all the constituencies right, but my model had Central as the Tories second worst area and gave them only two regional seats. In reality, they won three – with Labour suffering an even larger decline in Central than expected. From a Green point of view, the model was also just a shade too pessimistic about Kirsten Robb’s chances, albeit she still unfortunately missed out.

MSPs Correct; 15 of 16.

Glasgow Real

Based on most polling, and with the model viewing Glasgow as the Tories worst region, I would have expected the Greens to beat the Tories here. However, with the Tories doing so well nationally, even the model had them squeaking ahead – but Labour were seen to still be strong enough to prevent either party from getting two MSPs. Labour’s Glasgow collapse was even worse than expected however and sadly for Zara Kitson the Greens didn’t do quite so well, meaning the Tories got that second seat.

MSPs Correct; 15 of 16

Highlands Real

A rare success for Labour in the Highlands and Islands region, as they just managed to hold on to both of their list MSPs where my model was inclined to suggest they’d drop below 10% of the vote. Likewise, the model significantly overestimated the potential for Tory success in the region, with the Lib Dems bumping up a fair bit and the Greens also performing significantly better.

MSPs Correct; 14 of 15

Lothian Real

Lothian is always one of Scotland’s most interesting regions. My model would have expected the Tories to reclaim the Edinburgh Pentlands seat they held from 2003-2011 in addition to winning Edinburgh Central. In actual fact the SNP held it rather comfortably, and wins for Labour in Edinburgh Southern and the Lib Dems in Edinburgh Western relieved enough pressure on the lists for Andy Wightman to win a second Green seat.

MSPs Correct; 11 of 16

MF Real

A shock win for Willie Rennie in North East Fife as well as a solid increase in vote share gives the Lib Dems a far better result in Mid Scotland & Fife than my model would anticipate. Meanwhile, slightly worse results in terms of vote share for the Tories and Greens, who nonetheless won the number of seats the calculator expected.

MSPs Correct; 15 of 16

NE Real

The big inaccuracy in the model for North East was between the SNP and the Tories – the SNP suffered a surprisingly large dip in support in the region, whilst the Tories pulled off such a surge that they are two seats better off than modelled, including snatching one of the constituencies. A big flaw in the model here for the Greens, with it expecting Co-convenor Maggie Chapman to have been elected.

MSPs Correct; 14 of 17

South Real

A really mixed bag in South, with the model accurately identifying Tory constituency wins but obviously completely failing to predict Iain Gray would increase his majority in East Lothian. It also over-estimated Green support a little bit, with the absolutely tremendous Sarah Beattie-Smith missing out in reality, and could not foresee a surprise collapse in the Lib Dem vote leaving them without a regional MSP where once they had a stronghold in the Borders. It was the Tories that benefited most from this inaccuracy, with two regional seats, and failing to win East Lothian gave the SNP’s Paul Wheelhouse a lucky return to Holyrood.

MSPs Correct; 8 of 16

West Real

West was always the region my model was most suspect for – I kind of knew that, even if the SNP stayed static nationally, it was likely to be by losing votes in some regions and gaining in places like West and Glasgow, whilst that enormous Tory vote share was just too high. Still, consistent predictions of a Tory win in Eastwood proved quite correct, although even hailing from Dumbarton constituency originally, neither me nor the model could see Jackie Baillie clinging on. The biggest success though was for the Greens, the result on the day being far higher than in my model, successfully electing Ross Greer as the youngest MSP ever.

MSPs Correct; 14 of 17

On the whole then, the accuracy of my model seems a little bit mixed. Summed up across constituency and regional seats, my model correctly identified 106 of the 129 MSPs. I’m not counting constituency winners I’d thought would get in on the list or vice versa as correct, as even though I identified the MSP correctly I got their method of election wrong.

How does that square up with Universal Swing? I’ll fire through this with less comment, as it’s the numbers that matter.

National Universal Swing

National MSPs Correct; 121 of 129.
National Parties Correct; 1 of 5 (parliamentary).

Central US

MSPs Correct; 14 of 16

Glasgow US

MSPs Correct; 15 of 16

Highlands US

MSPs Correct; 14 of 15

Lothian US

MSPs Correct; 11 of 16

Mid US

MSPs Correct; 15 of 16

NE US

MSPs Correct; 14 of 17

South US

MSPs Correct; 9 of 16

West US

MSPs Correct; 15 of 17

Total; 107 of 129 MSPs.

I hereby concede narrow defeat to tradition when it comes to predicting individual MSPs – but claim a narrow victory in predicting national results.

And finally… What about the Scotland Votes calculator, from which we can only get the national overview for regions?

Scotland Votes.PNG

It doesn’t accurately predict any of the national seat shares, coming in at a total of 12 seats at odds with the actual results, for 117 of the 129 national MSPs. Now, that, I’m claiming as a significant victory!

Post Election Round-up; That’s Not A Beamer, I’m Just Sunburnt

Now that the dust has begun to settle on what was in many ways a shocking election result, I thought the best place to start my series of post election round-up posts was with a reflection on my own final prediction. Let’s start with where my predictive wizardry proved most potent, before moving on to where it flopped. (Full results for the election available from the BBC.)

What Did I Get Right?

Impact of Minor Parties – In the constituency vote, I predicted that 99.5% would go to the SNP, Labour, Tories, Lib Dems and Greens. With 0.2% for independents, 0.2% for others, and 0.1% for the Christians amounting to 0.5% overall, I was spot on. Likewise, in the regional, I called a 98% share for the above parties plus UKIP and RISE. Solidarity managed 0.6%, Christians 0.5%, Women’s Equality 0.3%, Independents 0.2% and Others 0.4%, for a total of 2%.

Few SNP List Seats – I only predicted list seats for the SNP in two regions. I said Central rather than South, but nonetheless, it proved quite accurate to state that in the vast majority of regions across the country, the SNP were unlikely to pick up enough list votes to elect any MSPs.

No Seats for UKIP and RISE – As expected, neither of the newcomers (yes, yes, SSP aren’t, RISE are) had much of an impact on the result. For UKIP, their best result of 2.6% in Highlands & Islands fell far short of what Coburn needed to enter Holyrood. Meanwhile, 1% for RISE in Glasgow was pitiful.

Greens in Fourth – Didn’t quite get the numbers right, but the Greens did indeed overtake the Lib Dems to become the fourth largest party, with 6 seats in total.

Lib Dems Static – I expected the Lib Dems to win 5 seats, just as they had in 2011. Although I wasn’t quite right with their vote share, I did state that I wasn’t expecting them to have attracted many new voters. With 7.8% (down 0.1%) in the constituency and 5.2% (same as 2011) in the regional, they really didn’t go anywhere nationally this election.

Labour’s 24 Seats – Although the spread of seats was off, I correctly predicted their overall number. I also came pretty damn close on the regional vote share, my 19.25% barely exceeding the 19.1% they achieved.

Tories Take Eastwood – I was consistent throughout the campaign that the Tories were on track to take Eastwood, and this proved correct.

What Did I Get Wrong?

SNP Majority and Constituency Sweep – With 63 seats on the day compared to my prediction of 71, the SNP not only didn’t increase the majority, they actually lost it. As part of that over-estimation, I expected them to win a lot of constituencies they didn’t.

Labour in Second – I refused point blank to believe the Tories were going to come second. I conceded the possibility, but consistently rejected that it would happen on the day. It did.

Labour Constituency Wipeout – Clinging on to East Lothian and Dumbarton, plus gaining Edinburgh Southern, means Labour managed to save a little bit of face in what was otherwise a very poor election for them.

No Lib Dems in Lothian, No Mainland Constituencies – With a win in Edinburgh Western, the Lib Dems did indeed make a return to the Lothian region, and combined with a shock victory in North East Fife, they party has a constituency foothold on the mainland once more.

No Tory Constituencies Outside South and West – Ruth Davidson scored an impressive win in Edinburgh Central, and there was a surprise North East win in Aberdeenshire West that see the Tories expand their base of constituency MSPs for the first time since 2007.

Glasgow and Central Don’t Vote Tory – My regional weightings of party support suggested even on a good result, Glasgow and Central would likely return just one Tory MSP. In fact, Glasgow returned two and Central a whopping three.

Two Greens for Glasgow and Nine Nationally – Unfortunately, despite early confidence from sampling, the Greens did not manage two MSPs in Glasgow, and poor results for a number of excellent women candidates didn’t quite see the party with 9 seats overall. With 6.6% of the vote, they also came in a fair bit below the 7.5% I predicted.

Why Did I Get It So Wrong?

There are three prongs to this. First, the polling just wasn’t accurate enough. The SNP and Greens didn’t live up to theirs, whilst the Tories actually exceeded expectations. Although I used a little bit of gut-feel, at the end of the day, even my final prediction was fuelled largely by polling.

Then we have the unexpected constituency results. I’d heard whispers that the Lib Dems were confident of Edinburgh Western before polling day, but almost everything else was a surprise. I certainly didn’t expect them to win North East Fife, I expected Labour to lose East Lothian and Dumbarton, and the Tories certainly weren’t on my radar outside of South and West.

Finally, there’s the simple fact that my model did indeed prove to be a bit off. It’s always going to be tough to predict how national swings will look at a more local level, so I’m glad I tried something a bit different. I’ll take a more in-depth look at my calculator in another post, and see what happens when I plug the national results into it.

When Polling stops Rolling; The Final Prediction

It’s been a long and tiring campaign, but it’s been a blast. I’ve enjoyed my predictive shenanigans (and so have nearly 13,000 of you!) – and there’s just one prediction left to do. Over the past few months, I’ve exclusively considered the results of polls and of averages of polls. This time, although informed by polls, I’ve held my finger to the wind and listened to my gut for a DIY prediction. As ever, word of caution – this isn’t gospel, it’s basically educated guesswork and wonkery, and I report the output from my calculator regardless of how suspect I might view specific individual results.

What our parliament looks like will determine how our country navigates the uncharted and distinctly choppy waters ahead. To a degree the result vis a vis who will form the government seems like a foregone conclusion. But nothing can be taken for granted in elections. And no majority lasts forever. No government can expect everything to go to plan. There are always opportunities for opposition parties to seize the initiative, to win support and trust, to push public discourse in a different direction. Every vote matters.

Final Prediction

Given the dearth of small parties and independents this time, I’ve assumed that the seven parties my calculator tracks will between them manage 98% of the regional vote – up markedly from the 93.6% in 2011. Dedicated followers of this blog will notice a much tidier distribution of regional totals in this prediction. I’ll explain how that came about at the end, for the real nerds. On the constituency front, I’m expecting 99.5% of the vote – that’s up only marginally on 2011, when it was already a pretty solid 98.9%. I’ve also included, for the first time, the full constituency predictions in the PDF – though these aren’t tidied up to the degree regional votes are.

SNP

It’ll come as no surprise that I’m predicting an SNP majority – and with 71 seats, an increase of 2 over their 2011 tally. This time, I expect that to be made up of a whopping 68 of the 73 constituencies thanks to 52% share of the constituency vote, giving them a comfortable lead of 31.5% over their next closest rival. I’m expecting far less of an increase in their regional vote, coming in at 45% and only 3 regional seats, two in Highlands & Islands and one in Central. In 2011, the SNP’s list performance was massively underestimated – frankly, I’m not expecting pollsters to have made the same mistake twice.

Labour

In spite of the Tories nipping at their heels, the recent bump in Labour polling has me pretty confident – I’d say 2 to 1 – they’ll hold on to second place. Nonetheless, they suffer a significant loss of seats, their 21.5% of the constituency vote returning no MSPs at all, so 19.25% on the regional vote returns all 24 of their MSPs in this prediction. I expect Labour to follow past performance and lose a fair chunk between the list and regional vote, but with their vote already cut to the bone, the gap won’t be as large as it was in 2011.

Tories

Unfortunately for the Tories, I don’t see them quite making it to second place this time, though 20 seats would be their best haul yet. Nonetheless, in one sense, the 18.5% I expect for them in the constituencies would give them a marginally healthier result than Labour, seeing them win 3 constituencies – most notably Eastwood in the West Region. Their other 17 MSPs would be delivered on 17% of the list vote, though I predict both Glasgow and Central will remain tough nuts to crack for the party.

Greens

Although the Greens have been polling well, between 8% and 11% over the last month of the campaign, caution and some last minute slides in the polls leads me to expect them to just fall a bit short of that 8% mark, with 7.5%. Nonetheless, that could deliver not just their best ever vote share for a Scottish Parliament election, but their best haul of seats too, with 9. As an aside, although this particular calculation would see Kirsten Robb miss out on a seat in Central, the margins are all so fine I’d not be surprised to see Kirsten elected ahead of either Zara Kitson or Andy Wightman (second in Glasgow and Lothian.)

Lib Dems

I don’t expect much in the way of recovery from the Lib Dems. I’ve said in the past I reckon a 6%, 6 seat result is where they are headed, but I also find it hard to imagine that with the expected increase in turnout compared to 2011, a great many of those new voters will be going Lib Dem, nor that many of their former voters will have returned, so I’ve opted for 5.5% instead. This just preserves their current batch of MSPs, with a narrow miss for Alex Cole-Hamilton in Lothian, and meaning the Lib Dems would keep their two constituencies out of the SNP’s clutches.

UKIP and RISE

And what of the hopefuls, seeking their first MSPs? (Okay, RISE includes the SSP which won 6 in 2003, but the coalition is new!) Well, UKIP seem to have slid a bit of late, so I’m not even expecting them to make 3%, coming in at 2.75% instead. That’s still a couple of % too low for Coburn to make it into Highlands and Islands – thankfully. Meanwhile, I just don’t think RISE have made any impact in this campaign outside the Indy bubble. Lacking any figures as big as Sheridan was in his heyday, I imagine a measly 1% translating to no seats.

Detailed National Prediction;

CG Final HIL Final MFNE Final SW Final

PDF of results

PDF of constituency results

Final Prediction Methodology; The baseline methodology of the calculator operates as it always has, but this time, I’ve smoothed out all the regions to give sensible totals and avoid exceeding 100%. This smoothing is not, and I cannot emphasise this enough, even vaguely scientifically rigorous. It’s a broad brush assumption to make the overall prediction look sensible. 

Having assumed a national share of 2% for other parties and independents, and with an average of four additional candidacies per region (see this post on small parties), that means an average of 0.5% per additional candidacy. Taking four as the baseline, regions with exactly four additional candidacies are smoothed to a 98% share between the seven headline parties. Regions with less see those parties collectively gain 0.5% for every minor candidacy short of 4. Regions with more see the opposite, the headliners collectively losing 0.5% for each additional candidacy above 4. 

Survation, 1st-2nd of May

(Full tables for this poll)

Survation 2nd May

With just two days to go until the election, the end is just about in sight.  Having contributed a grand total of one third of all polls published over the campaign, Survation have just released their (presumably) last – and it’s a real head scratcher, according to my calculator.

Although the SNP are still on course for an increased majority according to this, it’s the second poll in quick succession – and only the third of the campaign – to have them on less than 50% of the vote. Still, with 27% over their nearest rivals, it would likely deliver them 66 constituencies, topped up by a mere 4 regional seats.

There’s a really perplexing result in the contest for second place. Despite coming in at 1% behind the Tories on the list vote, Labour would narrowly squeak a lead in seats. I can’t even imagine the kind of feeling such a result would provoke in either party, though I can imagine Davidson giving it absolute hell for leather in the media about having been denied their rightful position as the “main” opposition, which would be absolutely delightful from a party that still prefers FPTP to proportional representation and thus can’t really talk about distorted results.

Likewise, no satisfaction to be had for the Greens, who would – although trebling their number of seats – fail to displace the Lib Dems as the fourth party, instead coming in at a dead heat despite that 1% lead in votes. Somewhat surprisingly, after leading the pack throughout the entire campaign, Survation find themselves on the lower end of expected Green votes, with 7% the lowest they’ve recorded for the party this year. Following a similar inexplicable drop with Panelbase, the next 72 hours will be rather tense for both parties.

Worry too for UKIP, as up until this point Survation have been most positive about their chances. A sudden dip down to 2% doesn’t bode well for a party already hardly likely to win any seats – a sign, perhaps, that their absolute meltdown in Scotland has been noticed by their would be voters.

RISE have not.

Detailed National Result;

CG 2nd May HIL 2nd May MFNE 2nd May SW 2nd May

Agency Trend;

Survation 2nd May Trend

PDF of results

Panelbase, 23rd-28th of April

(Full tables for this poll)

Panelbase 28th April

This poll was a bit of a surprise – having kept their gobs firmly shut from their January poll to their April poll, a second Panelbase poll a mere couple of weeks later was rather a bolt from the blue.

Most surprising is that they have the SNP on 49% of the constituency vote. Only one other poll this year had suggested that would be the case. My gut instinct is this is unlikely to  be replicated on the day, especially given the 5% of the constituency vote for “others” is unlikely to find many others. Nonetheless, they still have a very healthy lead indeed over their competitors, and would be returned to parliament with both an increased majority and almost every constituency in the country.

Some much needed relief for Labour here who are not only on the highest regional vote share they’ve managed this year, backing up the recent TNS poll, but also the  highest constituency result. That would be just enough by my calculator’s reckoning for them to – for the first and only time this year – hold onto a constituency, with Elaine Murray in Dumfriesshire narrowly holding back the constituency wipe out.

The Tories would receive their biggest haul of seats since Holyrood was created under this poll, but that late Labour bump would open up a wider gap between the two than most polling has predicted. It’s absolutely possible the Tories could yet come second, but Davidson would be wise not to get too attached to the notion of getting the opening question at First Minister’s Questions just yet.

A disappointing 6% for the Greens, the lowest they’ve been on since February, would nonetheless see a reasonable boost to their parliamentary representation, with 5 MSPs. With the Lib Dems only on 4% and dropping down to 3 seats, this would still see them replaced as the fourth party in Holyrood.

Unsurprisingly, neither UKIP nor RISE are expected to win any seats – but again, there’s a healthier looking 2% for RISE in this poll. I’m still absolutely certain they won’t win anything, and that’s still running close to 0% when you consider the margin of error, but interesting nonetheless.

Detailed National Prediction;

CG 28th April HIL 28th April MFNE 28th April SW 28th April

Agency Trend;

Panelbase 28th April Trend

PDF of results

TNS, 1st-24th of April

(Full tables for this poll)

TNS 24th April

As anticipated, TNS released the data from their monthly poll today, and it makes for quite the contrast with what we saw from Ipsos MORI yesterday.

It’ll come as no surprise that SNP are still on course for an increased majority, and will most likely win almost every constituency, but it’s interesting that TNS have over the course of two polls gone from having by far the highest figures for SNP on the regional vote to being in line with everyone else. TNS themselves tend to report this as a decrease in support compared to previous polls, but I have just the tiniest feeling they might instead have realised that something in their methodology was a bit funny.

Most remarkably, following on from a streak of polls that put the two absolutely neck and neck, Labour not only record a comparatively healthy lead over the Tories, but this is their strongest performance in list vote polls this year. That may offer some comfort, but I’d suggest that no one in Labour HQ breathes too easily just yet. Looking back over the polling this year, TNS have actually been the most favourable to Labour – they’ve been the only agency that’s ever had them above 20% of the vote, finding them on 21% in their last two polls. So although 22% is a fair bit higher than the 17% and 18% they’ve been seeing recently with other pollsters, it’s probably not any kind of late recovery.

For all that the gap between Labour and the Tories is larger than in most recent polls, this is also a TNS high for Ruth Davidson’s team. Second place may yet go to Labour, but breaking into the 20’s in terms of seats would be a pretty good consolation prize.

Still looking likely to become the fourth largest party in parliament, the Greens are holding steady at the 8% we’ve come to expect as standard for them at this stage in the campaign. Thanks to strong Labour and Tory performances, this wouldn’t net them the 10 seats that figure has suggested for them in the past, falling just short of double figures on 9 seats. Nonetheless, that’d still be a record performance for the party.

After a run of polls indicating there is life in the Lib Dems yet, another disappointing result for them here. Although down only marginally on their 2011 vote, the tight competition for list seats from other parties would see them drop to 3 seats. I’m still minded to suggest they’ll win 5 or 6, but that’ll be no great recovery.

As ever, neither UKIP nor RISE are expected to win any seats. Again though, RISE have come ahead of UKIP in voting intention. That’s distinctly odd, seeing as most pollsters have UKIP on 3 or 4% – only yesterday’s Ipsos MORI also expects near invisibility for the ‘kippers. Of course, the usual notes about margin of error apply – for all intents and purposes, UKIP and RISE in this poll are united in their irrelevance.

Detailed National Prediction;

CG 24th April HIL 24th April MFNE 24th April SW 24th April

Agency Trend;

TNS Trend 24th April

PDF of results

Ipsos MORI, 18th-25th of April

(Full tables for this poll)

Ipsos MORI 25th April

Is it a Survation? Is it a TNS? No! It’s an Ipsos MORI! The poll we’ve all – and by “all”, I really just mean “me” – been waiting for. We haven’t heard a peep from the good folks at Ipsos MORI since January, so long that with the release of the BMG poll the other week I’d stopped even including them when I was considering polling averages across the various agencies. With just over a week to go until polling day, it was about time we heard from them again. And what a poll – there’s actually a huge amount of information in there, but that’s beyond the scope of this blog. I’m in the business of seat predictions, so let’s get down to it.

For a change, let’s get the usual news out of the way first. Yep, the SNP are on track for an increased majority, yep, they would win a majority just on constituencies, and no, RISE don’t get any seats. It’s been clear for months, but worth saying explicitly – RISE are not going to win any seats next week. They’ve singularly failed to make their presence known beyond the usual suspects this election, failing even to get their logo on the ballot paper or a freepost out in good time.

More drama, too, in the ongoing scrap over who gets to call themselves the “official opposition” in a parliament which doesn’t designate an official opposition, with another Tory lead over Labour, with 22 seats to 20. I’m going to be a broken record again, but although a 2% lead is slightly wider than previous ones, it’s still within the margin of error.

However, I’m slowly creeping back from my assertion that Labour will come second on the day. I expect two further polls before the election, at which point I’ll re-assess, but I’m currently about 55% sure of Labour being second. That’s a big drop from even a week ago when I’d have said about 75% sure. Whether Labour come second or not though, it seems likely their list vote will indeed drop into the teens – delivering the worst result for the party in Scotland since before women could vote and men still had to own property to do so.

The Greens see another very positive result, with Ipsos MORI joining Survation in suggesting a break into double figures. That’s still the minority position amongst polling agencies, but it does back up the overall trend that shows the party is headed for a record result, comfortably returning an MSP in each region.

Another 7% for the Lib Dems will give the party some cheer, although I still only expect it to lead to 6 MSPs. This is very much the upper range of their polling in recent months though, rather than an upward trend, so it’s not looking entirely rosy. Finally, UKIP are on a low of 1%. The only other agency to have them so low is TNS, so I’d be inclined to suggest that their actual level of support is closer to what the other agencies are predicting, but either way they are still looking likely to miss out on an MSP.

Detailed National Prediction;

CG 25th April HIL 25th April MFNE 25th April SW 25th April

Agency Trend;

Ipsos MORI Trend 25th April

PDF of results