Now, I don't want to sound like a Liberal apolgist. First I trashed the Bristol poll and I am about to trash the CRA poll. The first poll was so-so for the Liberals and the second was relatively bad but, honestly, this is not the reason why I dislike these polls.
The reasons I didn't like the Bristol poll were laregly due to Bristol's massive inaccuracy in 2003 and their illogical region results (owing largely to teeny sample sizes and therefore high margins of error).
I will get into the reasons I don't like the CRA poll below but I am going to admit that I like the Bristol poll better. (No it is not because the Bristol poll had the Liberals in a better position)
The Bristol poll had a small sample size ~640 as I recall. The CRA poll has an even smaller sample (500). Any statistician will tell you that you need a sample in the range of 1000 to have solid results.
Second the CRA poll was, as they always are, conducted over a long period (13 days in this instance). That is fine for their ordinary quarterly polls which I respect and I think are accurate measures of the pulse of New Brunswickers.
However, in an election campaign, peoples opinions are volatile. It is not really uncommon in a federal election to see the two main parties change leads several times in a two week period when an election is on. This is due to gaffes, policy announcements, rebutts of policy announcements, chosing of candidates, etc, etc.
When there is no election, two weeks is a relatively static period. When an election is on, some people you survey may have changed their minds 2 or 3 or more times as you were questioning others. Thus, this poll is not, in my mind, a very good measure of where people stand.
It looks as though I will likely be forced to use the Bristol polls as the only reasonable measure of voter intentions over the course of this campaign unless ATV springs for a poll (as they have in past campaigns though, in those instances they've used Bristol which has already been spoken for).
Anyway, I know the old expression live by the poll, die by the poll and it is a fair adage. However, for political junkies polls are a lot of fun and, if done properly, they are a pretty good snapshot of what is going on. That said, if we continue with the camera analogy, if you were to leave the shutter on your camera open for 7 times longer than normal when there was a lot of hussle and bussle going around, all you would get would be a blur. I think that that is true in the case of this poll.
PS for those of you that didn't see it, the poll was conducted Aug 17 to Aug 30 and had essentially the same results as the May CRA poll: I don't have the paper but it was PC 45, Lib 37, NDP 9 give or take. Undecideds were at 37.
Posts
15 comments:
Polls are volatile especially during election. Only poll which matters is on election day.
People have not forgotten how Bernard Lord butchered the healthcare.
Don Mills at CRA predicted the 2003 election almost to a tee. Their methodology, I think, is very sound.
Did they not predict Liberal win in 1999?
CRA is know for their Liberal work. When they release a poll like this, I'm tempted to believe things are even worse for the Liberals than the numbers suggest.
Monctonite, Is it possible that they want PCs to become complacent.
Here's a thought, for 500 people we think we know how they'll vote. More than that, if you think asking 500 people will tell you how 500,000 or more people will vote, sucker.
Apart from that, what do those numbers mean? They aren't percentages, and they aren't ridings, so what are they?
I tend to agree monctonite. The Libs have had a tough go so far. In a tight election, they have to convince the voters that it's worth changing governments. Maybe there's something in their (the libs) back pocket they're waiting to bring out but I'm getting the sneaking suspicion I'm going to be waiting a long time.
I don't claim to know polling methodology, but if Nik Nanos and SES research used a rolling sample of 1,200 a day for the entire nation, then how is 500+ not enough for a small province with a sparse population. Wouldn't it adjust itself to population density?
As well, if they were to use large samples (1,200) from end to end in this campaign, that would essentially mean that 32, 000 people would be polled or better yet 5 per cent of the total population. For anybody who has worked in a polling company, they would tell you that the above scenerio is just not feasible.
I'm with David on this one, CRA's methodology must be sound as they predicted the 2003 election successfully. As well, federal and provincial samples traditionally differ in size, that I know as I was once employed by Decima research. LOL Sorry about the fib.
That's like saying that because I picked the lottery numbers once I must be able to win the lottery each time (and of course thats without verifying the data, it would be nice if the poll information were posted so it can be debated). Asking 500 people MIGHT let you predict who win and by how much. Saying that asking very few canadians federally who they will vote for means a methodology is sound is a logical fallacy, it's not sound in one case, therefore its not sound in another.
Anybody working in statistics can tell you that. There's a reason you'll almost never find one profession commenting on polls-and that's an actuary.
That's because its bollocks for so many reasons, only a few mentioned by our blogging friend. And in the fact that virtually nobody under 60 answers the phone to talk to strangers, the number of undecideds, the type of lead in comments, the tone of the person asking the question, the time of day, and on and on and you can see why liberals wouldn't exactly be sweating about the polls.
Of course there is a chance they MIGHT be right. There is also a chance I can write down the percentages and be right as well, or ask ten friends who they might for and be right.
And of course percentages don't mean a thing, Frank got barely 60% of the votes, yet got every seat. It's 'almost' completely irrelevant what the percentages are.
That few people quite literally means 10 people per riding were asked who they voted for. So walk into work and ask ten co-workers who they will vote for and post it, you're data is just as 'sound'. But if people don't get too excited by what you've found don't be surprised.
The main interest in polls I suspect is the simple fact that voters are next to irrelevant to the political process and media needs SOMETHING to talk about. There are huge issues that New Brunswick is facing, again, just go to www.davidwcampbell.com to read about just one area that 'nobody is talking about'.
If you look at the media coverage of "Bernard Lord takes Robichaud's name in vain" you can see how desperate people are to think of a non essential issue they can talk about. Because canadians play such a minor role in their own government polls have become the only fodder available during a campaign. I mean, just two months ago Atlantica people held a conference in Saint John and discussed tons of policy issues, and how often have you even heard one of them mentioned?
http://en.wikipedia.org/wiki/Margin_of_error check that out. It explains that population is not actually a factor in the equation for the accuracy of a poll. If you are surveying a city, province or country you still need about 1000 respondants to make it meaningful. I know it is counterintuitive but it is a fact.
I agree that CRA did well in 03 but they did not use the same methodology. Their sample was 800 and they sampled over 4 (I think) says. It was also later in the campaign. This poll started the day before the writ was dropped.
Keep in mind too that if they sampled 500 and 37% are undecided then the actual measure of intentions was of a sample of 315 people.
Thx for the Wiki link.
You make an excellent point NB politico, but while watching CPAC one afternoon (yes, I'm a geek), a very intelligent man made the clever statement that when polling companies tell you that statistical theory in a random poll of a 1000 people, with a 100% response rate, are accurate within 3 % 19 times out of 20, they are not being completely upfront. Why? Well, even though the 19 times out of 20 sounds like it is considerably accurate, we have to remember that 1 percentage point, in reality, is really inaccurate by 5 per cent. A much higher margin! In other words, there is a considerable margin for error...something polling companies would like you not to think about.
Polls are fun to analyze, I'm kinda a numbers person myself ... but the old addage holds true, "The only poll that matters is the one on election day." -- that seems to be especially true for New Brunswick. In the 1999 and 2003 elections most predictions were proven inaccurate - not to mention COR's unexpected 1991 showing!
Greetings,
I caught your comment to me on Brent's blog -- here's the passage I was referring to in your update from Fredericton:
"BUT I was surprised that the only private property signs were NDP.
Dennis Atchison had one sign I saw and Allison Brewer had lots on private property. Maybe I and others are underestimating her?"
Maybe I was inferring too much from this statement though.
Amen to the above. Personally, I'd far rather see blogs popping up like David Campbell's and Charles Leblancs that try to get discussion and notoriety onto actual issues, particularly during an election which is the only time frame that politicians are remotely interested in what the public is saying (but thats just a personal comment not a swipe at the blogger who posts good stuff). Talking about polls is fine for media, their job is to avoid issues and they do it remarkably well. The only poll that matters is on election day, and anybody telling you about 'margins of error' is working for a polling company (and of course we don't know WHO our blogging friend actually is:)
Post a Comment