A number of people have wondered why the numbers from Rasmussen Reports look different from polling conducted by other outlets, specifically why Barack Obama's approval rating in their daily tracking poll comes in lower than most other surveys. Central to the answer of this question is the fact that Rasmussen surveys only those it believes to be "likely voters" -- a value judgment, particularly this far away from the 2010 and 2012 elections -- and, additionally, that Rasmussen is using recorded interviews rather than live interviews. But that's not all.

As Pollster.com noted last week, and as I have been meaning to write about for some time (but have been waylaid with preparing for finals), the question being asked in the survey matter, too. Rasmussen was good enough to run three simultaneous polls last month asking three variations of approval rating questions: (1) Asking respondents whether they approve or disapprove; (2) Giving respondents four choices rather than two, asking them if they strongly approve, somewhat approve, somewhat disapprove or strongly disapprove; and (3) Giving respondents four choices, but four unbalanced ones, asking respondents if they think the President is doing an excellent, good, fair or poor job (unweighted because it lumps the ambivalent answer of "fair" into the disapprove category). For reference, Rasmussen traditionally uses question (2). Here were the numbers they found:

The differences are quite apparent: While the traditional question yields numbers that look a lot like the national trend, the question normally used by Rasmussen yields numbers that are flipped on their head. The third question, used by some pollsters like Harris, yield even more divergent numbers, data that look nothing like other polling.

Why does this matter? Many aggregating the polls are lumping together every survey regardless of the question being asked. While this is effective in gauging the overall trend -- which, don't get me wrong, is extremely important -- it obscures the picture of where the President currently stands by including with the traditional approval/disapproval question survey questions that tend to yield much lower "approval" ratings. In other words, it lumps in three different things as if they were the same. It would be like putting in a basket oranges, mandarins and pummelos but simply calling them all "oranges" when it used to be that you only placed oranges (but not mandarins or pummelos) in the basket.

So it's something to consider when looking at Rasmussen surveys, as well as trend estimates of the President's standing among the American people.