Rasmussen Polling Irregularities - A first post

It's already well established that Rasmussen polls are consistently to the right of other polls, and this is often explained in terms of legitimate differences in methodological minutiae. However, looking at the a large database of Rasmussen polls, it seems that their pro-republican bias, or House Effect, is not monolithic. There seems to be evidence that Rasmussen's house effect is much larger when Republicans are behind, and that it appears and disappears quickly at different points in the election cycle.

 

See graphs and more below the fold or at StochasticDemocracy.com

 

 

***Cross-posted at StochasticDemocracy***

I originally had this on DailyKos, but DesMoinesDem advised that I post it here as well.

 

Rasmussen polls are consistently to the right of other polls, and this is often explained in terms of legitimate differences in methodological minutiae. But there seems to be evidence that Rasmussen's house effect is much larger when Republicans are behind, and that it appears and disappears quickly at different points in the election cycle.

In order to estimate "instantaneous" house effect, we looked at all the Rasmussen 2008 polls(Senate, House, Governor, etc), and compared them with the filtered best estimate for public opinion on that day using pollsters excluding Rasmussen.

I'll hold off commentary for now, because these findings are preliminary, other then to say that House Effects seem to be more subtle then previously assumed, and that it warrants more study.

Note: All graphs are in terms of Republican Two-Way vote, defined as RepublicanVote/(Democratic Vote+RepublicanVote), as is standard in political science. "Margins", or RepublicanVote-DemocraticVote, are about twice TwoWay.

Estimated Rasmussen House-Effect vs "Closeness of race", defined as Abs(50-TwoPartyvote), estimated from all available pollsters excluding Rasmussen. This tests the idea that Rasmussen polls are reliable for close races, but that there is more ambiguity in non-competitive races. There seems to be a trend here, but the absolute value masks something...

 

Estimated Rasmussen House-Effect vs Estimated Republican vote from all non-Rasmussen Pollsters. The graph seems to show that Rasmussen House effects are much larger in races where democrats show large leads, making democratic-leading races seem closer then they really are. The effect is not symmetrical.

 

Estimated House Effect vs Days before the Election.

 

P-Value for "Rasmussen House Effect is greater than 0" over time. The graph seems to show Rasmussen polls have a Statistically significant Pro-Republican house-effect that appears during primary season in the beginning of the year, disappears during the summer, and then very rapidly appears right before the Republican National Convention

The findings are preliminary, my next post will expand the dataset to other elections and run the same procedure on Dailykos/R2K and SUSA polls as a control. In the meantime, I'm looking for explanations or mechanisms that could explain the patterns above, because I don't see how they can be explained in terms of different voter screens...

 

 

Tags: math, statistics, Rasmussen, house effects, polls, Elections, Bias (all tags)

Comments

2 Comments

why not?

<i>because I don't see how they can be explained in terms of different voter screens...</i>

I assume you're using loess (or similar) smoothing because you assume that the relationship, or the Rasmussen shift is only piecewise continuous. You've identified three regimes: early cycle, intermediate and post-convention. Rasmussen has explained that his LV screen is <a href="http://www.pollster.com/blogs/so_whats_a_likely_voter_answer.php"&gt;tighter when approaching the election:</a>

<blockquote>There's a challenge to defining a likely voter. The process is a little different than in the week before an election for us than it is in two months before an election than it is in a year before an election.</blockquote>

What I would probably due to get on the right track is as follows -- take the limiting cases: early in the cycle and right before the election.

Group the other polls according to their survey samples -- adults/registered voters/LV. Try the scatterplots again for Ras vs survey type, esp Ras vs surveys using adults:

<blockquote>What does it mean in practical terms? Rasmussen Reports and Gallup are the only two polls out there with a daily tracking poll of the President's job approval. If you go back from January 20th on, most of the time you will see that Gallup's reported number is about three or four or five points higher than ours, because these are surveys and there is statistical noise. Sometimes the gap is bigger, sometimes its smaller. In fact there are some days when our number is a little bit higher than Gallup's. But typically, the gap between the adults and the likely voter sample is in the four or five point range.</blockquote>

Essentially, you're looking for this effect in the context of last year's election polling. With respect to the random noise, I would probably double-check that it's truly random --> if not, this should reveal some clues.

When you look at the weeks before the election remember that some of the other pollsters are transitioning to LV models, generally speaking.

 

 

by dblhelix 2010-02-25 09:10PM | 0 recs
yikes

I do my own html coding on the fly -- guess things have changed here.

by dblhelix 2010-02-25 09:11PM | 0 recs

Diaries

Advertise Blogads