On the AP Race Study

Like Nate Silver, I’m hesitant to comment on the AP/Yahoo/Stanford poll’s estimate that Obama’s race is costing him 6% of the vote until I see the full underlying methodology. From what the AP has released, though it’s clear the study relies on self-reporting, which, when the subject is socially unacceptable prejudice, is obviously problematic. Discerning an exact figure like 6% from inherently flawed research methods is pretty absurd.

For a much better treatment of the question of race in presidential elections, see Jennifer Heerwig and Brian McCabe’s study, “Social Desirability Bias in Estimated Support for a Black Presidential Candidate.” Heerwig and McCabe conducted an experiment in which the control group was read three political statements unrelated to race and was asked to say how many of the statements they agreed with. The treatment group is shown these three statements as well as one about a black presidential candidate, and was also asked to say how many of the statements they agreed with. The mean number of statements supported was calculated for each group, with the difference between the two being the level of true support for a black presidential candidate. By comparing the results of this experiment with the results of a poll that overtly asks about support for a black presidential candidate. Both the experimental surveys and the traditional poll were conducted by phone nationally with a nationally representative sample, making them applicable to the national electorate.

Heerwig and McCabe estimated that the level of true support for a black candidate nationally is around 70% and the level of stated support is 84%, leading to an estimated social desirability bias of 14%. Now, there are many caveats here. As Mark Blumenthal noted at the time of the study’s release, the data was collected when Hillary was the frontrunner for the Democratic nomination, and the difference in the method of survey between the experiment and the traditional poll could account for some of the difference. Also, as Nate says, there’s very good evidence that the Wilder effect is gone. This is suggested both by the 2008 Democratic primaries and by the 2006 midterms.

But I’d be interested in seeing Heerwig and McCabe’s study replicated some time soon with an Obama-specific question like “Does Senator Obama’s race make you less likely to vote for him?” Theirs seems a much more reliable methodology than the AP’s, and if we want a reliable estimate of racial bias that’d be a better way of getting it.

One thought on “On the AP Race Study

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s