Alex, I don't think that's a sufficient explanation. The things that you're describing are perfectly true, and they bias the sample, but they shouldn't increase its variance, unless those are factors that can actually change substantially from week to week. The things you're describing might make the polled repub ID significantly higher than the actual number, but how would they cause the party ID figure to change so dramatically from week to week? The sharp week-to-week variation in party ID is what's at issue here, not the level. So I'm still not convinced that we've found the culprit.
But Jonweasel (as I play the Devil's Advocate), it's hard for me to believe that sample variance is that large when you're asking 600-1000 people. The MoE tells us that 95% of the time, the sample variance will be less than 3% (or 4%, or 3.5%, or whatever it is for each individual poll). Yet we're seeing 5 or 7% swings in party ID. That seems unlikely to be due to just sample variance, not for so many polls at the same time.
First let me say that I've admired and enjoyed your analysis of poll results for quite some time now. You're the place that I go to first for discussions of polls.
You build a good argument for weighting polls by party ID. I'm left with one lingering, but possibly important question. Why does party ID fluctuate so much in these presidential polls in the first place? I'm persuaded that party ID nationally doesn't normally fluctuate that much from year to year, but we now have poll results showing party ID fluctuating by 5% or more from one week to the next. What's the explanation?
Is it as simple as just getting a bad sample one week? It shouldn't be, because that's what the MoE is designed to take account of.
Or is it something deeper, like independents who aren't really strongly identified with either part, but who tell pollsters that they ID Repub when they're leaning toward GWB, and tell them that they ID Dem when they're leaning toward JFK?
If it's the latter, then simply weighting by party ID doesn't seem to fix the problem, I think.
jeromearmstrong Our Polarized and Money-Driven Congress: Created Over 25 Years By Republicans (and Quickly Imitated by Democrats http://bit.ly/ewXlXI #bblue
"Being certain is one thing. But you can be certain and wrong."
I also like a slight variant on that idea:
Bush says he's worried about sending mixed messages. He should worry more about sending the wrong message.
First let me say that I've admired and enjoyed your analysis of poll results for quite some time now. You're the place that I go to first for discussions of polls.
You build a good argument for weighting polls by party ID. I'm left with one lingering, but possibly important question. Why does party ID fluctuate so much in these presidential polls in the first place? I'm persuaded that party ID nationally doesn't normally fluctuate that much from year to year, but we now have poll results showing party ID fluctuating by 5% or more from one week to the next. What's the explanation?
Is it as simple as just getting a bad sample one week? It shouldn't be, because that's what the MoE is designed to take account of.
Or is it something deeper, like independents who aren't really strongly identified with either part, but who tell pollsters that they ID Repub when they're leaning toward GWB, and tell them that they ID Dem when they're leaning toward JFK?
If it's the latter, then simply weighting by party ID doesn't seem to fix the problem, I think.
Curious about your thoughts.
Kash