Wednesday, September 25, 2019

Why I discount polls.

I have mentioned many times, here and on other on-line blogs, that I do not put much stock in polls.  The reasons are many but the biggest reason is because of their inability to adequately account for people who say "Don't know" or "Undecided" during polling interviews.

I will demonstrate.  Let us say for the sake of argument that I conducted a poll of 1000 respondents.  The results that I found:


  • 300 people who support the Conservatives
  • 290 people who support the Liberals
  • 150 undecided
  • and the rest divided roughly equally amongst the other parties.
Simply put the result of my poll would be:


  • 300/1000x100 = 30%
  • 290/1000x100 = 29%
  • 150/1000x100 = 15%
When polling companies publish their estimates in the newspapers and online these are probably the raw numbers they have.  Unfortunately, nothing can really be inferred from these estimates except that no one as a clear advantage.  No one can say that any party will form a strong stable government with just 30% support. 

The problem is the undecideds.  They have always been a problem for pollsters who want to publish for news organizations because the news organizations want something that will pop.  They want something that will provide their reporters and columnists with something to write about.  Unfortunately, those numbers above do not do it.  It used to be the pollsters would just distribute the undecideds amongst the decideds using some kind of formula and each polling company would have its own formula for doing so.  They discovered that it did not work very well.  So then they decided that they would just remove all of the undecideds which is why you will always see polls indicating that their estimates are of decided and leaning respondents.  When you remove the undecided you have:

  • 300/850x100 = 35.3%
  • 290/850x100 = 34.1%
There you go.  There are some estimates that you can sink your teeth into.  Those estimates can be used by the media to advance whatever narrative they choose.  They can be fed into a seat projection model to show how many seats each party will receive and the regional estimates can be used to "refine" the projections.  Unfortunately, they are false because you cannot remove 150 respondents from a survey of 1000, who respond with a valid response to a survey question, and produce reliable statistics.

The funny thing is the first example that I gave above probably reflects reality.  Both the Liberals and the Conservatives have a base of support that is about 30% of the electorate and there are probably about 15% of the electorate who could be termed non-partisan swing voters. The Conservatives have a bit of an advantage because their base is pretty solid while the Liberal base is a bit more wishy-washy but in both cases neither has enough to form a strong government with just their base.  Therefore they need the swing voters to put them over the top.  That is the job of the leader's campaign.  The local campaigns attempt to identify and pull as many of their partisans as possible within each riding and hope the national campaign can successfully swing a sufficient number of swing voters over to them to take the seat.

Of course, the problem for all campaigns is there is no way to really know how successful they are until the votes are counted.  Most of the swing voters will not make up their minds until much later in the campaign, with a significant number of them doing so only when they step behind the cardboard divider to mark their ballot.  So, there is no way to predict which way they will jump.  However, it is interesting that they tend to jump in one direction when the finally do make a decision.  It is really quite remarkable.  For some reason the swing voters collectively decide to vote one way.  That is, the majority decide to vote for one party and the minority divide their votes between the rest, giving the one who earned the majority of their votes government.  No one has been able to figure out why.

So that is where we stand.  The two main parties are hanging on to their bases and waiting for the swing voters to make up their minds.  Neither know which way they will jump but they are working very hard to convince them to jump to their side.

As I have stated before I believe they will jump to the Liberals.  The main reason is the Liberals have three different historical voting patterns on their side.  First term governments tend to be re-elected in this country.  Since the introduction of the CCF/NDP to Canadian politics the Conservatives have only won when the NDP does well and the NDP is not running an aggressive campaign this time so they probably will not do as well as the Conservatives need them to.  Voters in Ontario seem to like governments of opposite stripes in Toronto and Ottawa.  With a Conservative government at Queens Park it is very possible that Ontario voters will vote Liberal at the federal level, and that is before taking into account any possible Doug Ford effect on their voting intentions.

So there you have it.  The public polls are telling a misleading story because they cannot account for undecided voters.  As long as that is true the polls and the different seat projection models that are fed by them are interesting to look at and debate but they are not a reliable indicator of how the campaign is unfolding and they definitely cannot be used to predict the outcome of the election.

1 comment:

Jackie Blue said...

The immature all-around pissing match involving pollsters and aggregators isn't helping their credibility much either. Some of the things they've said could theoretically be legally actionable. The defensive reaction that has caused a number of them to adopt a freemium model for access to their data ends up clouding the picture even further.

The Canadian polling industry isn't very well-regarded to begin with simply due to their poor record of accuracy, but the unprofessional behavior that some pollsters have been exhibiting on social media as of late only digs them further into a hole. Bunch of middle-aged high school stats nerds getting into virtual fisticuffs over math. And people say the drama teacher needs to grow up.