There's a few interesting things about the most recent polls. The first is that only YouGov have shown a big swing to Yes. The TNS poll, whose fieldwork finished around the same time as YouGov shows only +3% Yes among those certain/very likely to vote where YouGov is +12%. For both of them DK is much the same as their usual and TNS' level is still very high. In both cases, you'd expect DK to be falling but it isn't.
And, of course, Panelbase - the pollster most likely to show the highest levels of Yes support throughout has shown no gain at all in this period.
So, something weird is happening. I was pondering this today alongside some analysis by Alan Renwick of Reading University, which I think has been mentioned already. This shows that the percentage in favour of Yes in most referendums falls between the final poll and the actual vote, by an average of 11 points.
It got me wondering - what if it's not that there's a late swing back. What if actually the polls are wrong? Because then instead of explaining why people changed their minds, you'd have explain why referendum polls are so consistently wrong and in the same direction when we don't see that with normal elections. Why is the track record for referendums comparatively poor when with elections it's generally very good? It got me wondering if it's not so much that there's a late swing back to the status quo but that in the run-up to the vote polls become biased by the participation of those with the highest levels of motivation - people who want to make their views known. You'd expect to see the proportion saying they're certain to vote increasing, which I'd need to have a look and see*, although the 91% telling TNS they are certain or very likely to vote is strikingly high. These motivated people would, in turn, be biased in favour of people who want change (compared with the status quo) so you'd see Yes support increasing.
If that were the case, you'd see people signing up to online pollsters and being more willing to complete their surveys. Telephone pollsters would find it easier to get their surveys done. I've been told by a few people (anecdote alert) about calls from Ipsos MORI and their enthusiasm to finally be polled for the referendum. Face-to-face pollsters would see two things - interviewers would get their work done most easily by targeting houses with something in the window (and those would be predominately Yes things) and the people they approach would, because they're interested, be more likely to take part. I've been out with interviewers (not for the referendum) and watched them select houses that are most likely to match their quotas and have someone at home because their main priority is to get their interview target as quickly as possible.
In all cases, the surveys only need to match broad demographic quotas - age, sex, working status - so interviewing stops when the target number and quotas are met. You can see in the TNS tables where they're having difficulty with their quotas. Overall, they're about 60 interviews short but not short at all in Glasgow (usually the toughest place to get interviews). You can see where the unweighted total is smaller than the weighted - men, under 55s, ABC1 social groups, and in the West (apart from Glasgow), Central and Lothians.
And you wouldn't need many extra-motivated people to have an effect. An extra 50 people out of 1000 being motivated for change and keen to take part would get you a 5% swing.
Maybe all it does is suggest what will happen when the turnout inevitably falls below the level suggested by the certain to vote figures - the people who say they are certain but not actually as motivated as they claim to be are less likely to vote and that impacts on Yes disproportionately.
Anyway, that's all very speculative but if Renwick's analysis suggests there will be a gap between the polling and the result and Scotland follows that pattern, the polling will all be wrong, which would be a bit of a disaster for the pollsters since I can't see any way of estimating in advance show big that gap would be and if it's down to motivation and willingness to take part in surveys, it's not a thing you can deal with by weighing.
* between TNS' August poll and this one, the percentage certain / very likely to vote has increased from 80% to 91%
closed
posts
