Social Pressure and Inaccurate Polling

The latest national survey from Opinion Dynamics (PDF) contains the following unusual question:
18. Have you already or do you plan to contribute money to the candidate you are currently supporting?

1. I have already contributed money to his or her campaign
2. I plan to contribute money to his or her campaign in the future
3. (I have already contributed and plan to contribute more in the future)
4. I do not plan to contribute
5. Don't know / refused
This question was asked in a survey of registered voters, a population that includes about 170,000,000 Americans nationwide. In the poll, 6% indicated that they had indeed already contributed money to the candidate they are supporting. If this number is accurate, that would mean that 10,200,000 people had already contributed money to the campaign they are supporting. However, as we learned this week, only about 350,000 Americans have actually contributed money to a presidential campaign. That is, um, a bit of a gap between claims of financial support and the reality of financial support in this campaign. Another 14% of registered voters, or 23,800,000 Americans, indicated that they have plans to contribute money to the campaign they are supporting. All in all, that would make for 34,000,000 donors to presidential campaigns in the 2008 cycle, which would mean that in 2008 six or seven times as many people will contribute to a presidential campaign as contributed in 2004. I had no idea Americans had become so much more engaged in political activism over the last two years than they were in the already high turnout election of 2004.

The truth is that most of the people who claim they have already given a donation are lying. Many Americans feel a sense of social pressure to exaggerate their civic participation, and this is a clear case of where they are caught in the act of doing so. As Mystery Pollster wrote yesterday:
We know from at least 40 years of validation studies that many respondents will say they voted when they did not, due to the same sort of "social discomfort" mentioned above. Voting is something we are supposed to do, and a small portion of adults is reluctant to admit to non-voting to a stranger on the telephone. In theory, an automated survey would reduce such false reports.
Many people want to say they are engaged in our political process, even though they are not. This also represents another problem with basing national Democratic and Republican primary preference polls on all registered voters. Even though turnout for presidential primaries and caucuses rarely surpasses 25% of registered voters, and usually never surpasses 20%, most national polls are projecting a combined Democratic and Republican primary turnout of around 85% nationwide. Now, turnout on that scale would crush all records for presidential general elections. Even in 2004, turnout among registered voters was 72.9% in the general election, and that was among the highest in the last four decades. Primary turnout will be far lower. For example, in 2000, with a hotly contested primary for both Democrats and Republicans in New Hampshire, turnout was 46% among all registered voters in the Granite State. That was the highest overall turnout for a presidential primary in quite a few cycles. Everywhere else, it was far lower.

In order to develop an accurate picture of the current national state of the Democratic and Republican nominations, the large percentage of the population that exaggerates its civic engagement needs to be removed from polls. Until they are, I remain unconvinced that Clinton holds any significant lead in national polls for the Democratic nomination. Some will say that the latest ABC / WaPo poll as well as the Opinion Dynamics poll discussed here show that I was too hasty in arguing that Obama has nearly caught up to Clinton nationwide. However, I would simply retort by asking where Clinton's roughly 4,200,000 donors are, as that is how many the Fox poll indicates she has. Granted, the poll also claims that Obama has about 1,800,000 donors, but I think we all know which of those figures is closer to reality. Clinton's donors are exaggerated 70-fold, while Obama donors are only exaggerated 15-fold. Now, given those numbers, you tell me which campaign's support is being more inflated as a result of current national primary polling methodologies. The answer is obvious.

Tags: Activism, Culture, inflated Clinton poll theory, polls, President 2008 (all tags)

Comments

39 Comments

Re: Social Pressure and Inaccurate Polling

I don't think that 96% of people are lying when they say they already donated. I think there's a high degree of correlation, however, between response rate to a poll and whether you contributed or not. I.e., probably 100% of people who don't bother responding have also not given any money; donors would be overrepresented among people who bother responding to the poll in the first place.

by jforshaw 2007-04-19 11:46AM | 0 recs
Re: Social Pressure and Inaccurate Polling
Even if you are right, a huge, huge percentage are still lying.
by Chris Bowers 2007-04-19 11:54AM | 0 recs
Re: Social Pressure and Inaccurate Polling

One way this phenomenon is documented (it's quite easy, really) is to correlate the reponse of the interviewee to likely voter screening questions with their actually voter history (which is a public record in my state). That is, if your polling universe is generated from the registered voter file and you ask the individual if they voted in the last election, you can check that against the public record. Sure enough - people lie about their voting history. Some literature I've read (long ago) states that it's because they don't want to admit to someone else that they don't do something (usually for no good reason) that society says every good citizen should do.

by Michael Bersin 2007-04-19 02:18PM | 0 recs
Same Applies To Race

I recall Alan Wheat (African-American Congressman) here in Missouri going into his U.S. Senate race against John Ashcroft in 1994 with the polls of both major daily papers showing the contest too close to call but, as we know from our sad history, the white beat the black handily.  In our recent race for Mayor, two candidate polls showed the black candidate about to get 41% of the white vote and win the race.  He didn't and lost narrowly.

Conformance to social pressure is substantial when a pollee (is there such a word?) knows that the poller got his name from a public list and that someone can identify him.

Michael, perhaps the polls could shame a respondent into more truthful answers, by preferencing the voting history question by something such as: Would you confirm the election board's records on your voting history?  How often have you voted in the last two years?

by Arthurkc 2007-04-19 02:52PM | 0 recs
Re: Same Applies To Race

Would you confirm the election board's records on your voting history?  How often have you voted in the last two years?

In the psychology (as it were) of poll questions you ask the interviewee the demographic questions at the end of the interview - things like ideology, party self identification - income - education - race - religion  because there is a  tendency for them to answer those questions once the believe they've already invested a lot of time in the poll itself. If you asked demographic questions at the beginning they'd tend to bolt (or "refuse" - in opinion research parlance). When refusals happen in significant numbers you can't just add to your sample - that injects significant error.

A practice of opinion research is that the interviewer wants the interviewee to think that the opinion researchers don't really knows who they are. That assumption on the part of the interviewee - the belief that they have anonymity - helps the interviewer complete the whole device.

Letting the interviewee know upfront that you know who they are (by "confirming" their voter history) is the surest way I know to destroy the illusion of anonymity and thus substantially (and I believe, disastrously) increase the refusal rate. That interviewee is going to bolt. If you ask the question at the end of the interview it defeats the purpose of a likely voter screen - and you just wasted the time and effort on a useless interview.

One plus dialing to select a sample does have a measure of anonymity. Getting a sample from the voter rolls doesn't.

by Michael Bersin 2007-04-19 05:46PM | 0 recs
Re: Social Pressure and Inaccurate Polling

you nailed this one.

by nevadadem 2007-04-19 12:00PM | 0 recs
I do not plan on donating

Why should I give up some of my hard earned money to rich campaigns? My first choice is Gore, but evne if he runs, I will not donate a single dollar as he doesn't need it. I will however consider donating to Wes Clark if he runs since he won't be inundated with big money the same way I donated to Howard Dean.

by Pravin 2007-04-19 12:02PM | 0 recs
sure, go for Congress

your dollar matters more there.

by John DE 2007-04-19 01:45PM | 0 recs
Re: Social Pressure and Inaccurate Polling

I agree that the Democratic primary voters are going to be less than what the national polls are showing. However, it seems like this primary will have a record percentage of participation due to the excitement about the Democratic field.

by Cols714 2007-04-19 12:03PM | 0 recs
Re: Social Pressure and Inaccurate Polling

I think that the "attention" the race is getting in early states though mitigates the above biasis thats why Hillary is not doing nearly as well in those places.

by nevadadem 2007-04-19 12:06PM | 0 recs
Re: Social Pressure and Inaccurate Polling

Nice work Mr. Bowers. Your coverage on this topic is impressive and only more impressive in light of the evidence.

by joejoejoe 2007-04-19 12:21PM | 0 recs
More Clinton bashing

I mean if it comes to actually banning me or others over this issue, fine, not seeing 5 snide things being said about Clinton a day may be a good thing I can live with in the long run.

Despite the other good info I find here.

I don't know if this is even partial or impartial or not.

All I know is that you're saying that hillary supporters are more likely to lie and not be engaged in the political process.

great.  edwards and obama supporters are more likely to be hypocrites and condescending towards the rest of the country.  and, you're right.  they may turn out to turn out in droves for their candidate.  and all those people responding for clinton in those polls will just stay at home watching american idol.

who's being elite here, chris?

by Stewieeeee 2007-04-19 12:40PM | 0 recs
Re: More Clinton bashing
Look man, if you want to bring this out in public, fine. I gave you a warning because you accused me of biasing my polling analysis to manufacture pre-determined, desired results. I just won't stand for bullshit claims like that. If that is what you think of my polling analysis, then go somewhere else. And yes, I will ban you if you keep making that accusation.

You might have noticed that just yesterday I posted a piece of pro-Clinton polling analysis. You might have also noticed that my theories on Clinton's inflated poll numbers are being taken seriously by several polling analysts. Or, you might not even care about either, and just like to accuse people of bias when they arrive at conclusions you don't like.

Either engage in the debate over polling with actual analysis, or stay out of the debate. Continue to make your only contribution to the discussion regular claims that I am intentionally skewing results, and you will be banned, and fast.
by Chris Bowers 2007-04-19 12:57PM | 0 recs
You're not skewing results

You're interpretting them.

The results are that hillary is ahead in the polls (for the time being) and your argument seeks to render those results invalid.  or at least faulty cause they use a sample set -- or includes data -- that you would not choose.

it's no mystery that CHOOSING different sample sets gets different results.  

the goal in any poll is to take that CHOICE and render it inert.  to be random.

the polls you are questioning don't make that choice.   there is a goal of some randomness.  i'm no expert but i understand the goal is to take a random sample set.  not a CHOSEN sample set.

and if you've noticed, i won't say if this is impartial or partial or not.   i'll say it's completely impartial, and just say that it's based on a cynical misreading of the american public.  you are underestimating people who respond "hillary" on the polls in question.

i can't tell if that's engaging the debate or not?

it might eventually just get to the point where any disagreement at all is going to be viewed as a refusal to participate in debate.

which would be wierd.

so.  can i at least say i disagree without inpugning your integrity?

lastly.  trend lines are going to put obama on top in the next few months anyway.  the trending is enough on this issue.

and then we'll have to wonder if obama's support is over-inflated because 30% of THOSE respondents want to believe their participation in the political process is more extensive than it is..

you see.  that's what clues me in to some partiality (IF it indeed does exist!).  the assumption that what you point out above about the american people and their involvement in american politics doesn't apply to any of the other dem candidates.

it might, you know!

by Stewieeeee 2007-04-19 01:31PM | 0 recs
Re: You're not skewing results

Uh, polling for a primary or an "off" year election is difficult precisely because it's harder to get a representative sample - due to lower voter turnout. by comparison, polling six weeks out from a presidential election (on any subject) is a piece of cake.

Polling in a small and discreet geographic area with a fairly large sample can get you good information. Especially with a good understanding of historical precinct level turnout. A national poll with a moderate sized sample this far out from a presidential election (and the various primaries) ain't worth the electrons it's passed around on when it comes to predictive value. Mostly because a significant percentage of any candidate's supporters at this stage are "fair weather" supporters. If someone else's campaign takes off in their view or in the conventional wisdom's view you'll see the bottom drop out. This goes for any of the candidates.

by Michael Bersin 2007-04-19 02:27PM | 0 recs
Yes

randomness doesn't mean you poll the entire country to figure out what the temp is for voters in illinois or amongst democrats.

you can define data sets and also maintain strict standards of randomness within the confines of that dataset.

which is i guess another question for mr. bowers, how does he intend to isolate the dataset he's looking for and then maintain randomness within that dataset?

he merely suggests a way to magically erase the data that puts hillary (for the time being) ahead in the existing polls.

by Stewieeeee 2007-04-19 02:44PM | 0 recs
Re: Yes
Jesus, you really are ignorant about polls. You have never heard of likely voter screens? Just screen out those poll participants who are not very likely to vote in a Democratic primary. It isn't fucking hard. Every pollster knows how to do this. If you didn't do this as a campaign pollster, you would be fired.
by Chris Bowers 2007-04-19 02:53PM | 0 recs
Which would be great

but people are still going to lie based on the argument you make above.

see below.  i'm not being as facetious as you think.

you're the polling expert.  what do you do about people who don't answer truthfully?

but.  more crucial to this discussion is NOT what you do about people who don't answer in complete truthness about their participation level, but do you apply those same considerations to people who respond "obama" or "edwards."?

that's my main question.  and i think that's a very serious question.  i don't ask it lightly.

by Stewieeeee 2007-04-19 03:03PM | 0 recs
Re: Which would be great
"ore crucial to this discussion is NOT what you do about people who don't answer in complete truthness about their participation level, but do you apply those same considerations to people who respond "obama" or "edwards."?

that's my main question. and i think that's a very serious question. i don't ask it lightly."

And it is exactly what I told you would result in you being banned. You entire argument is that I am biased. Goodbye.
by Chris Bowers 2007-04-19 03:06PM | 0 recs
Re: You're not skewing results
The results are that Hillary is ahead in the polls (for the time being) and your argument seeks to render those results invalid. or at least faulty cause they use a sample set -- or includes data -- that you would not choose.

it's no mystery that CHOOSING different sample sets gets different results.

the goal in any poll is to take that CHOICE and render it inert. to be random.

the polls you are questioning don't make that choice. there is a goal of some randomness. i'm no expert but i understand the goal is to take a random sample set. not a CHOSEN sample set."
Look genius, the goal in a poll isn't just to be random. It is to be random within a defined universe. If you are trying to measure public opinion of Kentucky voters, then you work to develop a means of randomly sampling Kentucky voters. My argument is simple: polls that are seeking to measure public opinion among the Democratic primary electorate should randomly sample the Democratic primary electorate, rather than all self-identifying and self-leaning Democrats.

I guess your argument is that any poll, no matter what is it trying to measure, is invalid unless it simply measures a random group of people. That is, it would be wrong for a polling firm to try and only randomly sample Kentucky voters in order to determine what Kentucky voters think, because that would be an example of "choice" within a system that should be random. Instead, in order to find out what Kentucky voters think, we should just randomly sample people no matter where they live or how likely they are to vote.

All polls have targeted universes. Polls to determine the current national state of the Democratic primary and caucus electorate should poll Democratic primary voters and caucus goers. Right now, they don't. They just poll all self-identified and leaning Democrats. And yes, as I have made clear in several posts along these lines, I do think that inflates Clinton's numbers more than those of others.

But I don't think your argument has anything to do with polls. From what you have written here, and by your own admission, this is not a topic you know much about. Thus, instead your argument is just trying to prove that I am biased.
by Chris Bowers 2007-04-19 02:50PM | 0 recs
See above

Yes.  You define a dataset.  And then you maintain randomness within that dataset.

So.... genius.... how are you going to isolate the dataset you're looking for?

what do you do?  What's the script?!

"Hi.  I'm conducting a poll and would like to ask a few questions."

"Ok.  Shoot."

"Did you give money to a candidate this year?"

"yes."

"Ok.  Were you lying when you just said 'yes.'?"

"yes."

"Good.  Thanks!"

by Stewieeeee 2007-04-19 02:56PM | 0 recs
Re: More Clinton bashing

not fair I completely disagree with Cris on his theory that Hillary's general election number don't matter, but what he say's he thinks is right.Can't we disagree on the data means?

by nevadadem 2007-04-19 01:06PM | 0 recs
BIG NEWS
Joe Trippi just got hired by John Edwards
This is just another sign that John Edwards cares about the progressive movement.
by Edwards Supporters United 2007-04-19 12:42PM | 0 recs
Re: BIG NEWS

I'm so sick of shills.

by jallen 2007-04-19 12:47PM | 0 recs
Re: BIG NEWS
I'm sick of shills too. Please, if your purpose here is to shill for Edwards, go somewhere else. Off-topic posting like this to promote your candidate to just horseshit.
by Chris Bowers 2007-04-19 01:05PM | 0 recs
Opinion Dynamics is RDD (random digit dial)

and just asks if the respondent is a registered voter. It seems likely that even polls of 'registered voters' have some self-reporting error.

by IVR Polls 2007-04-19 01:04PM | 0 recs
Re: Opinion Dynamics is RDD (random digit dial)

What they get when they do that is the registered voters with "unlisted" phone numbers. They probably have formulas (based on experience) which gives them  an idea what their failure rate will be in any geographic area. They do have to rely on likely voter screen questions (and that has problems), but they do get a portion of the population with those unlisted numbers which are missed in samples which start from the voter file and subsequent phone matching.

by Michael Bersin 2007-04-19 02:31PM | 0 recs
Sure

I'm just arguing that missing unlisted phone numbers is a much smaller distortion than you get by assuming that respondents will be accurate with their voting history.

by IVR Polls 2007-04-19 06:08PM | 0 recs
Extrapolation

I think you're just extrapolating a bit more than you can support.  If we examined Dean vs Kerry, or any other campaign's number of donors compared to actual votes, how much of a correlation is there?

Yes people lie, that's a problem with any poll.  But extrapolating as to who is lying more to make predictions, with scant evidence, seems even less sound of a methodology than just acknowledging, "This what people said, even though they may be lying."  

by MassEyesandEars 2007-04-19 01:20PM | 0 recs
Re: Social Pressure and Inaccurate Polling
The same methodoligy that produces larger Hillary leads also produces huge Guliani leads.
Does anyone really think Rudy is up that much among the people who are really going to show up for the GOP primary. Most political anylasts like Cook and Rothenberg don't even think he had a shot at the nomination but the way these polls are being done Hillary and Rudy are going to win them easily, that is fact, it is opinion (based on facts) what it actually means to the overall state of the race.
by nevadadem 2007-04-19 01:25PM | 0 recs
Re: Social Pressure and Inaccurate Polling

I think you have made a great case for blogging just a little less about meaningless bull** polls.  I look at 'em too, but that doesn't mean it means anything, sort of like porn.

By the way, the endless shilling for presidential candidates (I'm not talking about any of the front pagers) is getting tiresome.  How many of these people are paid shills?  If it is almost imposible to discuss anything without someone chiming in about how great XXXX is or crying about Matt or Chris being mean to XXXXX than what is it going to be like next December?

by howardpark 2007-04-19 01:43PM | 0 recs
Re: Social Pressure and Inaccurate Polling

One minor complaint: I could do without the insinuation that people are being paid by the campaigns.  Accusations like that are less common, but almost as annoying, as the stupid fights themselves.

People are more than capable of being jerks without getting paid to do so.

But on the broader point, I heartily agree.  

This is great campaign, with great candidates, and it's understandable that people will want to debate the merits, and will occasionally take it a bit farther than they ought to - no foul there.  But the constant, neverending, obviously partisan bickering is a major turn-off.  If I wanted to read people with no objectivity, I would go start fights on Manchester United boards or something.  I'd like to think that people can be a little more self-reflexive on MyDD.

Though, to be fair, it's a pretty small number of people on all sides.  30-40 people are constantly creating comment-threads I don't dare wade into, while hundreds or thousands of others are calm, reasonable, and have all kinds of interesting things to say.

by Baldrick 2007-04-19 03:41PM | 0 recs
Re: Social Pressure and Inaccurate Polling

I asked how many are being paid.  It's a fair question.  I think there are more paid shills on here and other blogs than anyone wants to admit.  Maybe I'm wrong but people should disclose voluntarily.

by howardpark 2007-04-19 05:50PM | 0 recs
Re: Social Pressure and Inaccurate Polling

It is also an insinuation of a lack of integrity. You can't be surprised to find that those you accuse refuse to answer or engage with you on the subject.

by awgupta 2007-04-19 06:33PM | 0 recs
Identifying Registered Voters Who Tend to Not Vote

Here in Solano County, California, my local Democratic club has a method to identify by name which registered voters have a history of low turnout.  

We take the voting rolls (with voter history), available from the county registrar of voters for a nominal charge, and compute the percentage of time each voter has voted in the last twenty elections.  (County data only goes back that far.)

It takes a fair amount of time to import the data into spreadsheets, enter the coding to do the math, and make the computations.  But once it is done and printed out by precinct, it is very easy for a precinct captain to spot who is likely to vote and who isn't, based on their history of voting or not voting in each of the twenty most recent elections.  Our precinct captains can then concentrate their efforts on those with a low turnout propensity.  

I highly recommend this method as a way to get our folks to the polls.  

by Airpower 2007-04-19 02:05PM | 0 recs
Re: Social Pressure and Inaccurate Polling

This is a well-known effect when conducting studies. Sometimes the effect can be minimized when the poll is filled out anonymously as opposed to answering questions from a pollster.

Where this shows up most is in polls of public policy issues. A perfect example is religious affiliation. The number of people who report going to church each week is about 40% while checking attendance records comes up with a figure closer to 20%.

Bad polling is frequently used to justify a policy that some group is advocating. Even the terms used can affect the result. Asking if a person supports welfare scores low while rephrasing as helping those in need scores much higher.

People also express opinions when they don't know anything about the issue just not to seem uninformed.

by rdf 2007-04-19 02:30PM | 0 recs
Re: Social Pressure and Inaccurate Polling

If the respondent is called at random and asked if they are a registered voter, a fair number probably lie in response to that question as well.

That being said, voter turnout in the primaries will be very high in 2008, not just because of the wide open field, but because many voters in many states (especially large states like CA, IL, FL, NJ, TX, NY, etc.) will feel that their primary votes matter this time around, thanks to the compressed schedule many of us are complaining about....

by Lex 2007-04-19 03:20PM | 0 recs
Demographics and Socio Economic Correlations

Having had personal experiences with conducting public polls for a very prestigious polling organization - and conducting Political polls for candidates, one has to question whether the people who participate are truly representative of the population as a whole in terms of their personalities and mindset.

Many polls are still conducted by phones and  are unexpected and can be quite long.  

There is an attempt to correlate by chosing demographics and including socio economic questions - but that can only go so far.

by Security 2007-04-19 09:04PM | 0 recs
Re: Social Pressure and Inaccurate Polling

I believe that analyzing polls is very hard to do, so I commend Chris Bowers for trying.  I for one don't believe that Clinton's numbers are inflated, I actually believe in the opposite, that her numbers are somewhat higher than what an averaging of all polls shows today.    

All primary polls generally capture Independents who state that they lean Democratic and intend to vote in the Democratic primaries.   Those poll results are lumped in with the results of Democrats making their preferences known, which then combine to give polling figures.    However, Independents, despite their best intentions leading up to primary day, are least likely to go out and actually vote.    Thus, they are giving full weighting in polls (and we all know that Independents are less enthused about Clinton than rank-and-file Democrats) when their poll preferences should be viewed as less important, as they simply show up in much lesser numbers at the primaries than partisans.  

The Clinton poll underreporting was discussed in a recent Quinnipiac poll article in which a well-known pollster, who has polled Clinton for years for the New York Senate, has pointed the phenomenon out.  It is largely due to the stark difference in support Clinton gets from Democrats vs. Independents, but also in part due to Clinton supporters not "admitting" to their support because of the small, but very rabid and vocal anti-Hillary contingent voices in the Democratic party.    They then show up in the polling booth to pull the Clinton lever to the consternation and surprise of the anti-Clintoners, who were actually convinced that the Hillary numbers are inflated instead.  

Can the underpolling phenomenon that was apparent in both NY Senate races be transferred to the presidential?  I believe it is the same poll discussion we are looking at here.  Others believe in the opposite, even going as far as citing her 5th place ranking in a recent Moveon.org poll, as if that poll was representative of Democrats at large, which is obviously not the case if one considers that she leads every national poll, has not placed second in a single poll as of yet, let alone fifth.  

by georgep 2007-04-20 08:29AM | 0 recs

Diaries

Advertise Blogads