On poll dances

Shortly after I posted yesterday's roundup featuring some discussing of Praxis Analytics' Saskatchewan NDP leadership polling, Jordon Cooper chimed in with the results of an internal poll distributed by Cam Broten - which has been treated as somewhere between worthless and gospel depending on the leadership camp commenting on it. So let's take a quick look at the actual significance of internal polls in past NDP leadership races in determining what weight any new ones should carry.

The most recent examples to be considered would come from the 2012 federal race - where virtually all polling confirmed Thomas Mulcair to be the front-runner, but a string of polls (both candidate-based and independent) placed Brian Topp a distant fourth or fifth. Topp's camp declined to make his full numbers public, but said its polling suggested a radically different shape to the race - and it was the candidate who played his cards close to the vest who proved to be correct, as Topp finished a strong second.

But an even better previous example can be found in the 2009 provincial leadership campaign. At that time, this was my attempt to make sense of a poll showing Yens Pedersen running a strong second among a sample of 20% of the NDP's membership at a point when virtually no other data placed him in that position. And I tend to see the theory as largely holding up: an internal poll is likely to be highly inaccurate in telling us about vote shares (as Pedersen's was), but it may also help to set a floor for the polling candidate's raw vote level - and in retrospect I should have placed greater emphasis on that point in ranking the candidates through the balance of the 2009 campaign.

In Broten's case, I don't think there was much doubt that he would hold at least as much first-ballot support as we can plausibly infer from the poll released so far - though more information as to how many votes were actually included would offer a bit more perspective on that point. But for other factors including the relative positioning of the candidates, the safest starting point is to view the noise included in candidate-based polling as far outweighing any signal - particularly in light of the requirement that campaigns identify themselves (thus planting a seed for respondents) before carrying out the poll.

That said, I'll shift gears to a somewhat broader point. The major difficulty in trying to evaluate the state of any leadership race is the fact that all polling faces at least one of two inherent flaws: truly independent pollsters don't have access to the membership list necessary to take a viable sample of voters, while campaigns have an obvious bias in conducting and releasing polls based on an accurate voter list.

But there's another party involved in any leadership campaign with both the necessary information to reach all members (or a fair sample thereof), and at least an arguable incentive to ensure members have better information about the state of the leadership race. So with that in mind, I'll close this post with questions for discussion: should the NDP and other political parties consider commissioning leadership polls for themselves under agreed rules and timelines? Or does the risk of being perceived to interfere in a leadership campaign outweigh the value of what would surely be seen as the authoritative source of polling data?