Do National Polls Really, Really Inflate Clinton's Lead? Really?
by Chris Bowers, Sat Apr 14, 2007 at 10:22:27 AM EDT
Back on Thursday morning, I posted an article claiming that most national primary polls inflate Hillary Clinton's lead. My argument had two basic points. First, national primary polls are sampling all voters who self-identify or lean Democratic, which is about 40-50% of the registered voting population, rather than the 15-20% of registered voters who actually comprise the Democratic primary electorate. Second, I argued that the 15-20% of registered voters who actually comprise the Democratic primary electorate are significantly less pro-Clinton than the 40-50% of registered voters who comprise all Democratic self-identifiers and leaners. I based my second point on evidence showing Obama performing stronger among high-income, high education, and high news consumption voters, as well on polls showing both Edwards and Obama performing better among "likely" primary / caucus voters than among "registered" Democrats and Democratic leaners.
Mystery Pollster, while agreeing with the first point,that national primary polls are sampling far too broad a spectrum of Democrats, takes issue with my second point, that this inflates Clinton's advantage, which he ultimately thinks lacks conclusive evidence. He writes:
Mystery Pollster, while agreeing with the first point,that national primary polls are sampling far too broad a spectrum of Democrats, takes issue with my second point, that this inflates Clinton's advantage, which he ultimately thinks lacks conclusive evidence. He writes:
I am not ready, however, to agree that this practice inflates Clinton's lead in national polling. What evidence I see is sketchy and contradictory. For example, Tom Riehle of RT Strategies recently shared some tabulations combining data from two recent surveys conducted by his company (in February and late March) for the Cook Political Report. Clinton wins 41% of the vote against Obama, Edwards and the other candidates on these two surveys. However, she wins more support from pure Democratic identifiers (44%) than among the independents that lean Democratic (33%). That difference is statistically significant despite the small sample sizes (n=558 Democrats, n=164 Democratic leaners). So if sampling too many voters means too many independents, it will tend to depress Clinton's vote rather than exaggerate it.Fair enough. I even looked at the Rasmussen reports polls I cited as evidence for my argument, and it turns out their primary polling universe is drawn from all Democratic self-identifiers (774 likely voters) taken from a four day sample (2,000 likely voters) of their presidential approval tracking poll. In other words, even Rasmussen is drawing from 38-39% of the entire likely voter population for their primary voter sample. This might be too high, it might not be. It all depends on how Rasmussen determines "likely voters." Since, I do not know how Rasmussen determines likely voters, and since polling companies are loathe to give out information on their likely voter screens, doubt is even cast on that element of my evidence.
On the other hand, we did a quick comparison of national surveys fielded this year that asked the primary question of all adult Democrats (and leaners) versus those that asked the question of only registered voters that identify or lean Democratic. We looked only vote questions testing the whole field, but excluding Al Gore. Clinton received an average of 44% on four surveys conducted by Gallup and ABC/Washington Post that included all adult Democrats, and 38% on eleven surveys** that included only Democrats registered to vote (or "likely" to vote in the general election). So we have sketchy support for the notion that surveys with slightly tighter screens show Clinton with slightly less support.
Admittedly, there are many potential pitfalls with such comparisons (commenters, have at it), and none of the available data allows for a direct test of Bowers' contention. In other words, we have no survey that allows us to compare the vote among all Democratic identifiers to a theoretically "true" likely primary electorate. So I think the jury is still out.
Chris cites some data from recent LA Times/Bloomberg [PDF] and Pew Research Center surveys showing Obama running much closer among well educated Democrats. And better educated adults tend to vote at higher rates than less well educated voters. That much is true, although education is just one predictor of turnout. Another is age, and the Pew survey shows Clinton with more support among older voters who also tend to turn out at higher rates.
However, one aspect of my argument cannot be doubted. By sampling between 40-50% of all registered voters, the vast majority of national Democratic primary polls are not specifically sampling the Democratic primary electorate. As such, these polls should not only be taken with a grain of salt, but should almost be dismissed entirely as useful indicators of the current state of play in the Democratic primary / caucus season. If they are not polling the Democratic primary / caucus electorate, or even coming close to doing so, then they should not be used as indicators of opinion in the Democratic primary / caucus electorate. To use an analogous example, one does not poll the entire nation as a means of determining public opinion in California. One does not poll all Democratic self-identifiers and leaners in order to determine the opinion of Democratic primary and caucus goers.
News organizations commissioning national primary polls should either pony up the cash necessary to accurately sample the national primary electorate, or they should stop pimping their own polls as accurate reflections of the national primary and caucus electorate. Further, the American Association for Public Opinion Research should really lay into new organizations that don't do this, since it is blatant false advertising and manipulation of data by the national media. Finally, there may not be enough evidence to conclusively demonstrate my thesis at this time, but I still believe that when national polls sample an accurately narrow universe of the Democratic primary and caucus electorate, Clinton's lead will come in at 12% or less in every single survey. I just hope some organization will begin conducting the necessary (and expensive) surveys necessary to either prove or disprove my claim.
Tags: Hillary Clinton, inflated Clinton poll theory, polls, President 2008, Primary Elections (all tags)
-----------
myDD - skin
-----------
23 Comments