Inflated Clinton Poll Theory: Prudence Sets In
by Chris Bowers, Fri May 18, 2007 at 06:18:14 PM EDT
Pursuant to my latest Inflated Clinton poll theory post below, it needs to be pointed out that Internet polling conducted by YouGov (formerly Polimetrix), has a perfectly fine track record. I think Zogby's poor performance in 2004 has scared people away from Internet polling in general. However, if one compares the final Polimetrix polls in the thirteen closest Senate races in 2006 (PDF) with the IVR poll averages and live interview poll averages in the final week of those thirteen states (source), you can see that Polimetrix did just fine.
If you don't believe me, just check the numbers in the extended entry. For those not interested in looking at the numbers right now, the basic lesson is this: the three methods (IVR, Polimetrix, and live interview) performed identically well in the thirteen Senate races decided by less than 20%. In terms of overall accuracy, there were four states where Polimetrix beat the live interview average by more than 1%, four states where the live interview average beat Polimetrix by more than 1%, and five states where they were within 1% of each other. This is pretty impressive for Polimetrix, since they are a single polling firm, and they were up against poll averages in every state listed below. Clearly, Polimetrix didn't do that bad. Internet polling is coming along nicely, and Polimetrix (YouGov) is leading the way.
It seems that automated IVR telephone polling (Survey USA, Rasmussen), properly conducted Internet polling (YouGov, maybe Harris and Zogby), and live interview telephone polling (virtually everyone else) are all about as accurate as one another at this point. It makes sense, since the percentage of Americans who have email and the percentage of Americans who have landlines are almost identical right now (about 80% each). However, by the 2008 election, the number of people who have email will exceed the number of people who have landlines, and by 2012, it could be a significant difference. In order to compensate for this, Internet polling needs to by done correctly. So, we should be grateful that YouGov is making such large strides in the field, and not be so quick to dismiss Internet polls.
But what is to be done when IVR polls (Rasmussen), Internet polls (YouGov) and live interview polls (everyone else) contradict each other on individual campaigns, as they appear to be doing in the national Democratic primary? The best answer is probably to just average out the polls, and look for the causes of their difference in other locations. For example, it is entirely possible that Rasmussen, as is the case with many IVR polls, is simply sampling a higher percentage of younger voters, and younger voters tend to be more pro-Obama. In the case of YouGov, the difference is probably connected to the prominent "Undecided" option on their questionnaire, something that most live interview polls lack. This actually works quite well with the original Inflated Clinton Poll Theory. There are probably a lot of people right now who are not paying much attention, and as such don't know much about Obama or other less well known candidates. Thus, these people probably lean Clinton, but will only choose Clinton when pushed. Makes sense to me. Clinton still has the advantage among these voters, although that is the sort of lead that could quickly evaporate once more people start paying attention. I have actually written in the past about how polls that push undecideds favor Clinton more than other polls. Until that changes, her lead can be considered somewhat soft (though certainly not Lieberman-like soft).
I spent most of the last four hours thinking about my post from earlier today, and about the Inflated Clinton Poll Theory in general. Basically, I have concluded that averaging all polls is the best way to go. With so many polls, it just seems unlikely to me that one extreme Clinton-Obama margin or the other is absolutely correct, or that one methodology or the other is absolutely correct. In a heavily polled race, when has there ever been a large, hidden vote out that that most pollsters were missing? Outside of the Iowa caucuses and post-Katrina New Orleans, the answer over the last thirty years has been "basically never." These days, the worst-case scenario is for poll averages to be about six points off the final margin, which isn't that bad and can be accounted for in margin of error and turnout programs. As such, I just don't feel comfortable throwing my lot with one extreme or another, or with treating some polls as more accurate than others. I just don't want to go out on a limb like that right now, considering how I was burned by going with one extreme theory in my 2004 projections (the incumbent rule theory) and vindicated for going with poll averages in 2006 (thus opposing the wave theory).
Anyone who discounts Rasmussen and YouGov polls because they are one extreme in current national Democratic polling, or who discounts Gallup and ABC-WaPo because they are another extreme, is probably making a mistake. At this point, with so many different polls floating around, with so many different methodologies, with about half of the primary and caucus electorate not even paying "somewhat" close attention, and with an ever-changing and developing campaign, the simple fact is that widely varying results among polls is unavoidable. From time to time, I can become obsessed with trying to solve a problem in a way that will "scoop the world" that I can forget the problem probably has no clear answer. In many ways, I still live as though I am in graduate school, and as such I can still suffer from Smartest Kid In The Class syndrome, where showing up everyone else is more important than even being right. When it came to the Inflated Clinton Poll Theory, I think I just let that a case of that syndrome get out of hand.
Average the polls--all of the polls--and don't dismiss any of them just because they seem odd or you don't like the results for your candidate. Right now, that would indicate that Clinton is probably up by 10-12 points. And so she probably is. However, as the differences between the varying polls shows, there is still a lot of movement left in this electorate. It ain't over until February 6th.
If you don't believe me, just check the numbers in the extended entry. For those not interested in looking at the numbers right now, the basic lesson is this: the three methods (IVR, Polimetrix, and live interview) performed identically well in the thirteen Senate races decided by less than 20%. In terms of overall accuracy, there were four states where Polimetrix beat the live interview average by more than 1%, four states where the live interview average beat Polimetrix by more than 1%, and five states where they were within 1% of each other. This is pretty impressive for Polimetrix, since they are a single polling firm, and they were up against poll averages in every state listed below. Clearly, Polimetrix didn't do that bad. Internet polling is coming along nicely, and Polimetrix (YouGov) is leading the way.
It seems that automated IVR telephone polling (Survey USA, Rasmussen), properly conducted Internet polling (YouGov, maybe Harris and Zogby), and live interview telephone polling (virtually everyone else) are all about as accurate as one another at this point. It makes sense, since the percentage of Americans who have email and the percentage of Americans who have landlines are almost identical right now (about 80% each). However, by the 2008 election, the number of people who have email will exceed the number of people who have landlines, and by 2012, it could be a significant difference. In order to compensate for this, Internet polling needs to by done correctly. So, we should be grateful that YouGov is making such large strides in the field, and not be so quick to dismiss Internet polls.
But what is to be done when IVR polls (Rasmussen), Internet polls (YouGov) and live interview polls (everyone else) contradict each other on individual campaigns, as they appear to be doing in the national Democratic primary? The best answer is probably to just average out the polls, and look for the causes of their difference in other locations. For example, it is entirely possible that Rasmussen, as is the case with many IVR polls, is simply sampling a higher percentage of younger voters, and younger voters tend to be more pro-Obama. In the case of YouGov, the difference is probably connected to the prominent "Undecided" option on their questionnaire, something that most live interview polls lack. This actually works quite well with the original Inflated Clinton Poll Theory. There are probably a lot of people right now who are not paying much attention, and as such don't know much about Obama or other less well known candidates. Thus, these people probably lean Clinton, but will only choose Clinton when pushed. Makes sense to me. Clinton still has the advantage among these voters, although that is the sort of lead that could quickly evaporate once more people start paying attention. I have actually written in the past about how polls that push undecideds favor Clinton more than other polls. Until that changes, her lead can be considered somewhat soft (though certainly not Lieberman-like soft).
I spent most of the last four hours thinking about my post from earlier today, and about the Inflated Clinton Poll Theory in general. Basically, I have concluded that averaging all polls is the best way to go. With so many polls, it just seems unlikely to me that one extreme Clinton-Obama margin or the other is absolutely correct, or that one methodology or the other is absolutely correct. In a heavily polled race, when has there ever been a large, hidden vote out that that most pollsters were missing? Outside of the Iowa caucuses and post-Katrina New Orleans, the answer over the last thirty years has been "basically never." These days, the worst-case scenario is for poll averages to be about six points off the final margin, which isn't that bad and can be accounted for in margin of error and turnout programs. As such, I just don't feel comfortable throwing my lot with one extreme or another, or with treating some polls as more accurate than others. I just don't want to go out on a limb like that right now, considering how I was burned by going with one extreme theory in my 2004 projections (the incumbent rule theory) and vindicated for going with poll averages in 2006 (thus opposing the wave theory).
Anyone who discounts Rasmussen and YouGov polls because they are one extreme in current national Democratic polling, or who discounts Gallup and ABC-WaPo because they are another extreme, is probably making a mistake. At this point, with so many different polls floating around, with so many different methodologies, with about half of the primary and caucus electorate not even paying "somewhat" close attention, and with an ever-changing and developing campaign, the simple fact is that widely varying results among polls is unavoidable. From time to time, I can become obsessed with trying to solve a problem in a way that will "scoop the world" that I can forget the problem probably has no clear answer. In many ways, I still live as though I am in graduate school, and as such I can still suffer from Smartest Kid In The Class syndrome, where showing up everyone else is more important than even being right. When it came to the Inflated Clinton Poll Theory, I think I just let that a case of that syndrome get out of hand.
Average the polls--all of the polls--and don't dismiss any of them just because they seem odd or you don't like the results for your candidate. Right now, that would indicate that Clinton is probably up by 10-12 points. And so she probably is. However, as the differences between the varying polls shows, there is still a lot of movement left in this electorate. It ain't over until February 6th.
-----------
myDD - skin
-----------