"Where I disagree with your explanation of YouGov and where I think we disagree more fundamentally is that you suggest that if YouGov did 100 polls using the same method they should get one cluster but if they did 100 by five different methods they should get five different clusters. I would say that if YouGov did that, they should still get all of the results in a single cluster."
I think the evidence says that they wouldn't.
At a UK level, if you do face-to-face street polling you get more Labour voters, because you get disproportionately few people who work 9-5. If you do house calling you get disproportionately more Tories because you don't get people who don't have phones, or who are younger and not in the house in the evening, or who work shifts. So if you did 20 of each of those two, you would get two different clusters for Labour v Tory. The centre of the cluster would be accurate for the method, and at the edges would be outliers.
You couldn't know though which of the two clustrers was more likely to be representative unless you know from historical data that one was generally more accurate than the other.
"If they got five clusters it would suggest they do nothing right (well, maybe one but we wouldn't know which) and we should hire someone else."
That's right and wrong. They are getting five different clusters because they are getting unrepresentative samples, but each is unrepresentative in different ways. That's why we have weighting, and why there's a balance in methods.
But you are absolutely correct that we wouldn't know which is right. Unless you could benchmark it against past election results.
"house effects"
Sorry, maybe my terminology has caused a problem here. Because I was using methodology as a synonym for house effects. I meant the whole method - the sampling, the nature of the questions asked, the weighting, whether you report those saying 10/10 certain to vote or 5/10 and above, all of that stuff.
On the level of certainty to vote, that's what causes the big variations in Ukip support. There are two polls out today, YouGov has them on 25% and ComRes on 34%. Which of those is closest to right? We'll find out a week on Sunday.
"It means I don't need to worry about why they are different, just accept that they are and then work out by how much."
Cool - can you tell me which of these is more accurate than the others (see the first table):
http://ukpollingreport.co.uk/blog/archives/7744
"none of them is individually the truth but collectively, appropriately analysed, they can point us to the truth, or at least a better understanding of what the truth might be. We get better view of the underlying trends that the current methods being used for 'polls of polls' - essentially rolling averages - aren't getting close to."
Couldn't agree more.
closed
posts
