Research 2000 polling questioned

Markos Moulitsas fired Research 2000 as the pollster retained by Daily Kos a few weeks ago after R2K fared poorly in "pollster ratings" compiled by FiveThirtyEight's Nate Silver. At the time I wondered whether Markos reacted a bit harshly, since Silver himself admitted, "The absolute difference in the pollster ratings is not very great." In addition, some polling experts had raised questions about Silver's rating system (see also here).

Today Markos published a remarkable analysis of "problems in plain sight" with Research 2000's polling. Three researchers uncovered "extreme anomalies" in certain results and concluded, "We do not know exactly how the weekly R2K results were created, but we are confident they could not accurately describe random polls." You should click over and read the whole thing, but here are the anomalies in question:

1. A large set of number pairs which should be independent of each other in detail, yet almost always are either both even or both odd.

2. A set of polls on separate groups which track each other far too closely, given the statistical uncertainties.

3. The collection of week-to-week changes, in which one particular small change (zero) occurs far too rarely. This test is particularly valuable because the reports exhibit a property known to show up when people try to make up random sequences.

Markos has renounced "any post we've written based exclusively on Research 2000 polling" and asked polling sites to "remove any Research 2000 polls commissioned by us from their databases."

Based on the report of the statisticians, it's clear that we did not get what we paid for. We were defrauded by Research 2000, and while we don't know if some or all of the data was fabricated or manipulated beyond recognition, we know we can't trust it. Meanwhile, Research 2000 has refused to offer any explanation.

Last year the Strategic Vision polling firm was brought down by convincing allegations that at least some of its polling results had been fabricated. Research 2000 had a much better reputation than Strategic Vision, though. Markos listed some of the news organizations that have commissioned R2K polls. I am seeking comment from KCCI-TV (the CBS affiliate in Des Moines), which has used R2K in the past. I will update this post if I hear back about their future plans regarding commissioned polls.

Share any relevant thoughts in this thread.

Research 2000 gets the "Zogby Award" 2008

There's a post by Markos made while I was on vacation-- basically a chest-thumping post of his about how right Research2000 was for the GE match-up in his tracking poll.

Markos points out that I was one of the detractors of the Research 2000 poll, but unfortunately for his pointed finger, everything I pointed out that was flawed in the poll turned out to be the case:

To believe the Research2000 prediction for 2008, you have to believe that latinos & blacks will vote in equal numbers to their populations as whites do, that Democrats will outnumber Republicans by 9% and Independents/Refused will be nearly 40% of the voting day population, and to assume that older people (60+) are going to be less of the voting population (even though the trend says they will be more).

*R2K said voters above 60 years of age would comprise 22 percent, I said higher, and it was at least 24 percent.

*R2K poll said blacks and Latinos would be 27 percent of the electorate, I said lower, and it turned out to be 22 percent.

*R2K projected that Democrats outnumbered Republicans by 9 percent, I said lower, and it turned out to be 7 percent.

*R2K projected that Independents/refused would numbered 39 percent, I said lower, and it turned out to be 29 percent.

But all that said, Research2000, amidst their weeks of showing it a 11-12% race, put in two days of McCain only having a 4% lead in the final four tracking days, for a 51-46 final projection, which wasn't too bad, being off by a little less than 2 percent. But even Markos didn't believe his own poll, predicting a Obama win by 9 percent. So, I will give R2K the "Zogby Award" for 2008. This is the award that goes to the pollster that, given the final result, is consistently off until the final days before the election, when they poll to make it more in line with the other poll projections. So congratulations, Markos, on your R2K success.

There's more...

Indiana: Obama 48% - Clinton 47%

I'm surprised this hasn't been brought up here, seeing as we're all dying of thirst for new poll numbers. It's a timely poll, conducted 4/23 - 4/24, and the numbers couldn't be closer:

Clinton 47%
Obama 48%

The demographics released show a familiar refrain:

Voters over 60:
Clinton 59, Obama 35

Women:
Clinton 53, Obama 45

The Young (18-29):
Obama 66, Clinton 33

The General Election:
McCain 51, Obama 43
McCain 52, Clinton 41

Update [2008-4-24 21:43:33 by VAAlex]:: Don't know what I was thinking not including the source ... mea culpa! http://www.southbendtribune.com/apps/pbcs.dll/article?AID=/20080424/NEWS07/284272668/1129/News

There's more...

Two New Iowa Polls, Two Different Results

As we've seen so many times before, two new polls released out of Iowa today tell two entirely different stories. The Research 2000 poll (500 LVs, 12/10-13, MOE 4.5%) has Obama up 9% over Clinton while the latest Diageo/Hotline poll (569 LVs, 12/7-12, MOE 4%) shows Obama and Clinton tied for first.

Diageo/ HotlineResearch 2000 (Nov.)RCP 5-poll Ave.
Obama2733 (25)29.8
Clinton2724 (27)26.8
Edwards2224 (21)21.8
Richardson89 (10)7.7
Biden53 (4)4.3
Dodd11 (1)
Kucinich11 (1)
Undecided85 (11)

One thing both polls agree on is that, technically, Iowa is still a statistical tie. One aspect of both polls that demonstrates the momentum Obama has generated in the state though is on the crucial second choice question. Obama is clearly eating into Edwards's traditional lead on this measure, even besting Edwards in the R2000 poll.

Diageo/ HotlineResearch 2000
Edwards2525
Obama2329
Clinton1821

Both polls also asked about experience versus new ideas/direction and the results are surprising. In the Diageo/Hotline poll, voters say they value "new direction" over "experience" 50% to 25% yet Clinton, the "experienced" candidate, ties Obama, the "new direction" candidate, overall. In the Research 2000 poll on the other hand, voters say they value "experience" over "new ideas" 43% to 35% yet here Obama is 9% up. The reason seems to be that Obama is eroding concerns about his lack of experience as 36% of those who consider experience the most important issue when voting support him (vs. 46% for Clinton.) Both polls still find Clinton with credibility as a candidate who would bring a "new direction" or "new ideas;" her advantage over Obama on experience is still much greater than his advantage over her on new direction yet he is either tied with her or beating her among all voters. It's not that people's minds are changing about who has the most experience, it's just that Obama's lack of experience is becoming less and less of a problem for him.

There's more...

Democrats VERY strong in new Missouri and Minnesota polls

  • Minnesota stays blue
SUSA released a new poll from Minnesota today -- and the results are very similar to yesterday's survey from Wisconsin: The state clearly trends Democratic, Hillary Clinton is stronger than Barack Obama, and John McCain is the only Republican who wins a match-up.

Full poll numbers and analysis here, on Campaign Diaries.

There's more...

Diaries

Advertise Blogads