Tuesday, September 29, 2020

Leading Questions in Polls

So a Leger poll came out today that had estimates of the Liberals at 40% and the Conservatives at 30%.  If that was the outcome of an election the Liberals would be elected to comfortable majority government.

Interestingly there was one other question for the poll regarding the Throne Speech which did not have the most neutral wording.  I would bet a great deal of money that the Throne Speech question came before the party support question, which would explain the sudden spike in the Liberal estimates.

We have seen this in the past from Leger and from other pollsters as well.  Putting a question in front of the party support question that we could reasonably assume would impact the party support question.  The lowest estimate that the Liberals had in recent Leger polls came when they put a WE question before the party support question.

This may seem unfair but putting these kinds of questions before the party support question is considered to be a valid practice by the industry.  I can see their point to some extent but I have always preferred the party support question to be asked up front.

I believe one reason why it is accepted practice is public opinion and elections result do not take place in a vacuum.  There is a reason why election campaigns matter.  For the whole of the campaign the different parties are trying to establish the dominant ballot question for election day.  So, by putting the issue question before the party support question the pollster is giving the respondent something resembling a ballot question.  

When Leger made WE a simulated ballot question the estimates for the Liberals went down (but the estimates for the Conservatives did not go up it should be noted). Although, the reduction was still not enough for the Liberals to lose an election if that would have been the outcome of one.  Conversely, when Leger made the Throne Speech a simulated ballot question the estimates for the Liberals increased substantially.  

What does this prove?  Not much.  It is a single poll so trying to discern a broader message from it is a waste of time.  

A little off topic.  I have seen some people say that Nanos and Ipsos have different estimates even though they were in the field at the same time as Leger.  So which one is correct?  None of them are.  Why are they different?  Different polling companies with different frames, sample designs, questionnaires, collection methods, data processing strategies, estimation methods and so on.  All of these things impact the final estimates from polls and very few people really understand how.  Although, that really does not matter because the polling companies do not share this information anyway.

1 comment:

Jackie Blue said...

Many of these are self-selecting Internet panels that weed people out to achieve a certain outcome. That seems to be part of the "house effects" however else those come about, and I believe Leger favors the Bloc, which is why their numbers tend to be higher in Quebec than pollsters from English Canada. Angus Reid, DART and Ipsos overtly favor the Cons and I'm not sure about the others, although I think Nanos has started to put his thumb on the scale for the Cons ever since he signed on with CTV.

Which is another aspect of "push polling" that I believe is fundamentally corrupt, and a far worse conflict of interest than vague hand-waving about PMJT and his family: the media frames the issues and then runs a call-and-response poll with their contracted pollsters about the coverage they just did, who then provide "punditry" that often toes the line of whatever network or paper they're signed on with.

So if a bunch of stories run that the Governor General is doing a bad job, then the public is going to respond unfavorably to a poll asking about her job performance, never bothering to educate the public on what exactly is the job she is supposed to do. The headlines say the deficit is bad, so people get worried about the deficit. There's no substantive or fair informing of the public on issues, just "punditry" running their mouths off and then doing polls to see if it resonated the way they want it to.

I get panicky about these things because of how effective it is but I keep trying to remind myself that temperature checks outside the writ period aren't predictive. It nevertheless infuriates me that there's never any coverage of the bad things the Cons do. Only the Liberal government, because the media and their pet pollsters are the Cons' pro bono PR.