Sunday, August 29, 2021

The Role of Data in Modern Election Campaigns

In 1997 I worked for a Member of Parliament of an Ottawa riding (The Boss) and I would go to work each day to do my job as a Member's Assistant and then, since my job was as much on the line as The Boss's, I would go volunteer at the campaign office in the evening.

Whenever we went canvassing there was always one member of the team who was responsible for recording whether a person we spoke to would support The Boss or not. They recorded this on a piece of paper on a clipboard. After each canvass that paper would be handed off to the data entry team at campaign HQ so that they could enter it into the computer. That was our database for pulling the vote on E-day. A few days ago when the Conservative candidate visited my neighbourhood I noticed someone with a tablet filling this role.

I provide this little anecdote to demonstrate that data and its analysis has been an important part of election campaigns for decades. With the increase in computer power and capabilities it has become one of the keys to winning a modern election.

With modern computing power, some of the extremely powerful data analytics software available on the market and well trained data scientists a political party can determine the probabilities of how an election will turn out at the riding level. Forget about the national, provincial or regional levels. Think about what all of the seat aggregators produce but on steroids. 

So before this election was called the Liberal data team would have had reliable probabilities for every riding in the country. Since 170 is the magic number they would have identified at least 180 ridings where the Liberals had at least a 75% chance of winning (A large chunk of them would have been in the 85%+ range). To allow for some slippage they would then have identified another 20+ ridings where they had at least a 60% chance of winning. If they were unable to meet these objectives the PM would not have called an election. Being in power with a minority is better than risking an election unless you believe you have it essentially locked up. 

The data team developed these probabilities using a whole host of data collection methods. Quantitative methods such as polling, but at the riding level and qualitative methods such as focus groups just to name two. They then would have crunched the numbers to come up with the probabilities and data analysts would have provided the "picture" to the party decision makers.

In addition these same teams would have tested all of the different policy proposals that have been released by the party and they would have tested the potential reaction to the early election call. 

Once the election is called the data team has to switch to monitoring progress. For the ridings where they identified lower probabilities of winning they would continue to collect data, from polling and other data sources to look for changes in those probabilities. For the safe ridings they would probably not regularly poll them. They would use that money on less safe ridings. However, remember the tablet I mentioned in my anecdote? All of the data collected during canvassing would be uploaded to party HQ where the data analysts would see if there were any serious signs of trouble in those 180 seats with the 75% probability of winning. If they did that would be flagged. In addition they would be testing the reaction of voters to the daily announcements and actions and words of each of the party leaders.

All of this would be done and a daily dashboard would be produced to be presented to the people running the campaign, which would include the PM. Now here is the rub. This data analysis would not just tell the decision makers the probabilities of victory for their party but it would also provide the probabilities for the other parties. So the Liberals know where they stand each day but they also know where the other parties stand. The Conservatives, who have a similar operation, have the exact same information.

I do not know all of the intricacies because I am not part of the team but this is a general description of the importance of data to modern elections. 

Of course, there are limits to what data can tell you and it is not uncommon for people to misinterpret what the data is telling you.  As well, although these data can provide probabilities that does not constitute a guarantee. There is always a chance the probabilities could be wrong. We can never discount that possibility.

However, I will continue to point out that one of the reason why the Conservatives were so hysterical about the early election call was their data teams were showing the same things the Liberal team was showing. The Liberals had at least a 60% probability of winning more than 180 seats. As well, I will also continue to point out that although the public polls are saying the Liberals are losing support neither the Liberals nor the Conservatives are showing signs that their own data analysis teams are in agreement with those polls. 

8 comments:

Jackie Blue said...

On a different type of research, I hope the Liberals have some big oppo bombs to drop on the CPC next week. People need to wake up and realize that this is the same ugly blue monster it's always been, just with a fake affable shit-eating grin.

ottlib said...

If the Liberals are smart, and I think they are, they will make the protection of our health care system the election issue for the next three weeks. Erin O'Toole admitted (their are multiple recordings) he would allow more privatization and "choice" in the system and that is anathema to the majority of Canadians. They need to spend the next three weeks hammering the message home. The message needs to be delivered on many fronts and they need to be relentless in delivering it. They might also be able to ensnare Mr. Singh in that by asking why he is not speaking up more to protect our health care system.

As well, I hope they push the vaccine mandate and vaccine passport angle and indicating that Erin O'Toole has come out against them. This is completely contrary to what the protesters are saying and will bring the unreasonableness of the protesters into focus for the 85% of us who are fully vaxxed or on our way there. It will put Erin O'Toole on the spot of being seen to agree with them. It will energize the fringe right, who support the PPC, so that they will vote and cause all sorts of problems where the Conservatives are in close races with their opponents, particularly in Ontario.

This needs to be delivered as differences in policy positions. Do not mention anybody by name. As well, they will need to be resolute in doing it because we all know that when they do it the media will accuse them of "going negative" while completely forgetting about the Willy Wonka video and all of the personal attacks on the PM by numerous Conservative leaders and Mr. Singh. Then again, the Liberals know that the media is hostile towards them so I would imagine they will do what they have done for the last two election. Go over their heads and speak directly to Canadians.

Jackie Blue said...

Do you still think their data teams are seeing what that anonymous alleged staffer said? Rob Silver, who is married to Chief of Staff Katie Telford, just sent out a pep-talk tweet to the LPC faithful. It reads like "copium" but at the same time I wonder if they aren't planning a more aggressive reset/course for the next three weeks to, as you said, drive the most important messages home. Or it could just be copium... 🤷

ottlib said...

Katie Telford is one of the insiders so she would know the score as much as anybody. She would probably communicate that to her husband.

The only people who really know how this campaign is going are the people on the inside of the political parties and none of them are going to be forthcoming about what they are seeing. We can only speculate based on what the campaigns are doing. I have maintained for some time that there is no sense of panic in the Liberal campaign. Indeed, I would say it was the opposite, which has been a little surprising and rather frustrating for some. Yes Liberals on social media are going crazy but they do not count. Jean Chretien used to call them Nervous Nellies and he was right.

I guess the question is whether what he is saying is genuine or not? If it is genuine then I would say the anonymous staffer may be onto something. If it is not genuine then it could just be a "pep-talk".

Jackie Blue said...

Well, we may have a clue as to the whole "not responding to telephone polls"/"Muskoka cottage" theory that the anonymous Liberal staffer advanced along with Quito Maggi. David Coletto of Abacus Research is due out with a new online-panel survey this week. He suggests that the CPC right now are the most motivated to answer the phone and tell the pollster (even if it's a computer) how angry they are at Trudeau.

He follows with a score of partisan interest level in the election campaign from his latest survey, which shows that the CPC is the only party that isn't "meh" on the election call right now. (Obviously because their sole motivation is getting rid of Trudeau.)

That could affect response rates to online surveys as well, but CPC voters tend to skew older and more likely to pick up for unfamiliar phone numbers. It remains to be seen what the online panels show later this week, but it does seem to corroborate the thesis that the highest level of attention won't pick up until after Labour Day.

I think you're right nevertheless that nobody knows the real state of the race besides the parties' data teams themselves.

ottlib said...

Yep, I am seeing a few pollsters indicating that they do not really trust what the rolling polls are saying, including one of the pollsters that is conducting one of those polls. Something just does not seem right. And really, it is just two of the rolling polls. The other rolling poll, from a pollster that has done this kind of poll for at least the last five elections is not showing such gruesome estimates for the Liberals. They are not great but they are not really bad either.

In three weeks the most important poll of the campaign will take place. When presented with a real decision that will have real consequences we will see how the electorate really feels. I still say that when the time comes enough of the electorate will remember why they were able to go away this summer, and not be home to answer election polls, and enough will decide that they do not want to change the government to give the Liberals the victory, probably a majority.

But we will see.

Jackie Blue said...

Well, here's the data you were waiting to see. Although I guess you could call it "anecdata". An article from the Hill Times about what some candidates are hearing on doorsteps, and how that ties in with the response to the public polls.

https://archive.is/SNh2e

Evan Scrimshaw may very well be correct that the Liberals are able to hang on due to hardcoded structural advantages in the election map that the Tories are unable to overcome. The anti-vax mob and the debates may change things for them too. It may even be possible that the door-knockers themselves may not be reaching people who are away for the summer and not tuned in until Labour Day. But, I don't think anyone could argue that this election so far has gone the way the Liberals hoped it would.

ottlib said...

The Hill Times is the definitive publication for the Ottawa Bubble. It never looks outside of the precincts of Parliament Hill to look at the real world so you always need to keep that in mind when reading it.

The story you link to contains alot of unnamed Liberals sources. I am always suspicious of publications when they quote unnamed sources because you never know what sources they are actually talking to. I would remind you that considering his history Warren Kinsella could be called a Liberal source. So who are these sources? What are their motivations? Not all former Liberals are enamoured with Justin Trudeau and many of them are willing to express that if asked by a media publication. And those media publications are more than willing to ask them the questions and represent them as "Liberal sources". We see that often on the TV political shows.

As for the quotes from the MPs you have to ask how many did they talk to and which ones did they decide to quote? They might have talked to a couple of dozen MPs and found one or two that responded as indicated in the story. It stands to reason that some MPs may be having a harder time than others. The others could have said everything is going well. The media can and does cherry pick quotes from people to advance the narrative of the story and we all know that the current media narrative is the Liberals are in trouble. It is an old media trick, which I have actually lived, and it is THE reason why I will not pay for news anymore.

Some of Mr. Bricker's polling colleagues would seem to disagree with him. I will be writing a new post on this today so I will expand on this idea there.