R2K fail

by Jerome Armstrong, Tue Jun 29, 2010 at 10:59:11 PM EDT

I thought it was a sham of a poll from the first time I saw the internals in 2008, and I was pretty open in that criticism too with Markos. It reminded me of Rasmussen's first attempt, with Portrait of America iirc, in 2000 (huger fail). In fact, there's a real funny, in hindsight, back and forth between us over Research 2000 if you want to read it, from November 2008:

markos: Research 2000 tracking poll accuracy

jerome: Research 2000 gets the "Zogby Award" 2008

markos: Cooking the polls

jerome: The most accurate pollsters

I should probably shut up, lest I get dragged into this mess like Nate Silver with a cease and decist email, but seriously, was it ever a doubt that these poll numbers by R2K were catered for the crowd?  Not to me anyway. Cooked too now? What a shocker.

Tags: (all tags)

  • 22 Comments
  • 1 recs
  • Share
  • Permalink

Comments

22 Comments

Shadow_head_small
Serious people knew these polls were cooked

Kos always featured their "Research 2000" poll results prominently, and it would be kind to even call them an outlier. On any given issue, they were so far separated from the results of the major polls---Ras, Gallup, NBC/WSJ---that an impartial observer knew something was fishy.

Yet if one dared to question these fictitious numbers in a comment on Kos, it was like a Catholic questioning the Vatican....you were a heretic to the liberal orthodoxy, and risked excommunication. It's somewhat gratifying that others will now see that the liberal emperor Markos has no clothes.

 

by BJJ Fighter 2010-06-30 12:07AM | 0 recs
Shadow_head_small
RE: Serious people knew these polls were cooked

Uh, what?  Markos isn't a statistician or a polling expert.  When problems with the polls were pointed out to him by people who had no axe to grind, he confronted R2K and did further research on the topic.  When their response was unsatisfying, he took appropriate remedial action, pulling and disavowing the polls, and suing R2K. 

DKos is the biggest lib/prog/Dem blog on the Net.  Getting flamed on a major lib/prog/Dem blog is roughly akin to taking in oxygen and expelling carbon dioxide.  It happens so often it's not even noteworthy. 

by megaplayboy 2010-06-30 09:10AM | 0 recs
Shadow_head_small
Even Millikan

cooked his numbers.  It is human (and not always a bad thing to do)

by Ravi Verma 2010-06-30 12:13AM | 0 recs
Marine sunglasses
RE: Even Millikan

When is lying ever a good thing?

by Nathan Empsall 2010-06-30 01:10AM | 0 recs
Shadow_head_small
When Millikan does it..

Here is some interesting reading on that

http://www1.umn.edu/ships/ethics/millikan.htm

Millikan had, it appears, been selectively highlighting some of his results... while rejecting others.  The global average of all his measurements would have yielded a value for e (charge of an electron) somewhat closer to what is the real value.  Instead, he selectively highlighted some of his results (cooked the books), and came up with a value that was slightly less right (but still got him a Nobel).

Some things that detract from his legacy.... include his claim that "It is to be remarked, too, that this is not a selected group of drops but represents all of the drops experimented on during 60 consecutive days, during which time the apparatus was taken down several times and set up anew"

But, before I criticize, I should remember that I will be happy to achieve 0.01% of what he accomplished before I die.

 

by Ravi Verma 2010-06-30 01:39AM | 0 recs
Marine sunglasses
RE: When Millikan does it..

That strikes me more like ignoring or not publishing some polls than manipulating them.

by Nathan Empsall 2010-06-30 01:42AM | 0 recs
Shadow_head_small
RE: When Millikan does it..

And you are probably correct.  

 

However, the experiment is taught (or should be taught) to all budding engineers/scientists for the ethics lesson it offers.  He was manipulating data, but where he had good reason to do so (he thought some of his experiments were bad).

The more important ethics lesson is about what happened next (and the reason it should be taught to all budding engineers/scientists).  Over the years, other scientists emulated Millikan's experiment, and skewed their results to match his.  That was an example of data manipulation, simply to make your data more consistent with a preconceived notion of what the final answer should be.  

Sounds like that is what r2k did.

by Ravi Verma 2010-06-30 01:56AM | 0 recs
Marine sunglasses
RE: When Millikan does it..

"make your data more consistent with a preconceived notion of what the final answer should be."

Sounds like Fox News. ;)

by Nathan Empsall 2010-06-30 02:16AM | 0 recs
Shadow_head_small
RE: When Millikan does it..

I don't have any problem with leaning a poll. It can help drive a narrative. The other side certainly uses it and with great effect.

The issue here is one of fraud, not lean. If R2k was leaning further than they did, but they were doing so legitimately (e.g., using a less realistic sample pool), then I don't think there would be an issue.

by NoFortunateSon 2010-06-30 10:25AM | 0 recs
4a2fab88-ed3b-47a4-8f14-3ec8a24504ac_normal
RE: When Millikan does it..

No, the issue was making shit up, and then folding in line as the election neared. Did you even read the posts in question; really, don't let your hatred get in the way of the facts.

by Jerome Armstrong 2010-06-30 11:46AM | 0 recs
Shadow_head_small
RE: When Millikan does it..

How is that any different than what Ras does?

by NoFortunateSon 2010-06-30 12:00PM | 0 recs
4a2fab88-ed3b-47a4-8f14-3ec8a24504ac_normal
RE: When Millikan does it..

"It reminded me of Rasmussen's first attempt, with Portrait of America iirc, in 2000 (huger fail)."

He's just gotten better at it over time.

 

by Jerome Armstrong 2010-06-30 01:08PM | 0 recs
Shadow_head_small
RE: When Millikan does it..

Sadly, the same can probably be said of MSNBC, CNN, NYtimes etc.

 

Or, have we forgotten Judith Miller so quickly ?

by Ravi Verma 2010-06-30 08:14PM | 0 recs
Shadow_head_small
Impressive analysis

I waded through the statistical analysis of the r2k data.  It is fairly impressive, but not conclusive of fraud...until you see the plots that the researchers make with point #3.  

For instance, point #1 does not prove fraud.  It just means that something very unlikely happened (which is happenstance), or was made to happen (fraud).  If kos had based his article based on point #1 alone, he likely would have lost easily.

However, it is hard to dispute point #3; with a large enough sample size, the distribution had better be centered on one value.  The binomial distribution cannot happen without fraud....if the sample size is large enough.  One can quibble that a sample size of 60 is too small to make definite conclusions (rough rule of thumb is you need 100 in the sample (wish they had waited another year!!), before the number becomes "large enough" to satisfy the "fluctuation dissipation theorem", and make Gaussian analysis valid.)  But, they had 3 examples of 60 samples each...for a total of 180 in the lot.  180 is definitely enough.

Under the circumstances, it would be nearly impossible to conclude that someone didn't cook the books. 

by Ravi Verma 2010-06-30 01:10AM | 2 recs
Shadow_head_small
Everybody is a genius

after the fact. And gloating is a very unattractive trait.

I give Markos credit for being very open and up front about this screw up, but in the end  . . . so what. 

by jsfox 2010-06-30 08:41AM | 0 recs
4a2fab88-ed3b-47a4-8f14-3ec8a24504ac_normal
RE: Everybody is a genius

2008 is after the fact?

by Jerome Armstrong 2010-06-30 08:57AM | 1 recs
Shadow_head_small
Correlation and causation

Well, as I noted earlier, your allegation against R2k was that it leaned left.

But Nate Silver pointed out that the margin of error for R2k was not that great. So your argument against R2k because they were providing friendly results doesn't hold much merit -- the results just weren't that friendly. Plus R2k was a reputable pollster used by many other media outlets besides dKos.

Markos' allegation is that fraud was behind the results regardless of lean, which required intense statistical analyses to prove, and only on point #3 of 3, as Ravi notes above.

As such, Markos, Nate, and those who did the statistical analyses receive the credit. You may have been right, but for the wrong reason, which on the Internet, amounts to nothing special.

by NoFortunateSon 2010-06-30 10:22AM | 0 recs
4a2fab88-ed3b-47a4-8f14-3ec8a24504ac_normal
RE: Correlation and causation

roflmao, you are such a sourpuss! That the numbers were cooked is pretty in line with their being numbers catered to the crowd-- I'm sure you licked up the bowl too.

by Jerome Armstrong 2010-06-30 11:44AM | 0 recs
Shadow_head_small
RE: Correlation and causation

I would disagree with your take somewhat.  It may be hard to prove fraud from statistical analysis; but that does not mean you cannot determine fraud from other methods.

 

Jerome claims that he knew r2k was fraud back in 2008, based on other factors.  (and if I recall, Charles Lemos had once made a similar suggestion).  There is no reason to dispute that, simply because statistica evidence was hard to come by.

In fact, his argument sorta makes sense...if a pollster is systematically leaning left (presumanly to satisfy left leaning customers), and lurching back to the center just before the elections (presumably to avoid being caught), then that pattern looks mighty suspicious.

by Ravi Verma 2010-06-30 08:19PM | 0 recs
Shadow_head_small
RE: Correlation and causation

Yes, I agree.

But leaning and late correcting is what Ras has been accused of doing. It can be done legitimately by "updating" likely voter screens, etc., at the last moment. In this regard, R2k is no different than Ras.

The allegation of fraud by Markos implies that data was forged, and there seems to be some evidence of that.

Leaning/skewing and forging are two very different practices.

My criticism of Jerome's argument is one of the stopped clock being right twice a day: if every poll that leaned was fraudulent, we wouldn't have many polls. All those internal polls? Out the window. I can stand here and accuse Ras of fraud, but that doesn't mean they are forging data. They are choosing their LV model, and then adjusting it based on whatever empirical or subjective rationale.

by NoFortunateSon 2010-06-30 08:49PM | 0 recs
4a2fab88-ed3b-47a4-8f14-3ec8a24504ac_normal
RE: Correlation and causation

Your making a convenient  strawman argument, with the "if every poll that leaned was fraudulent we wouldn't have many polls." Really, just admit your only point here is one of petty detraction, simple spite, and we'll all be better off not wasting our time.

Markos was the one making the argument that I was laying out an accusation of fraud toward R2K polls. My assertion was much simpler: that R2K was making up a voter model to fit their client and the crowd of looker-ons. And it turns out that they were, despite the strong denials by Markos and others that what I was saying wasn't true. And in spite of the facts that I laid out in the posts above. 

That they turned out to really be faking it just validates the point of my argument. I made the same argument against Rasmussen's POA polls in 2000, and after the election, he admitted that he was using a faulty voter turnout model. Being fraudulent and being wrong are not exclusive or inclusive of each-other. 

I don't see the same being the case with Rasmussen currently. For the most part, its just whiners that don't like the polls looking a certain way. Nitpicking some cross-tab or another (as if they have a MoE with only hundreds being polled for the topline), but in substance, not doing much of anything in terms of showing his polls to be out of whack with the final result.

by Jerome Armstrong 2010-06-30 09:20PM | 0 recs
Shadow_head_small
Just excellent

Nice crystal ball there, sir.

The emperor is frantically recalling his tailor....

Too damn funny.

by eroded47095 2010-06-30 12:29PM | 0 recs

Diaries

Advertise Blogads


----------- myDD - skin -----------