Thursday, February 14, 2008

Polling Analysis: Looking Ahead to Wisconsin and Ohio

One of the most frequently replayed arguments at TPM Cafe goes something like this:

ObamaLover27: Wow! New polls show a huge Obama lead!
Clinton4Ever: You can't
trust polls. They've been wrong before. Particularly polls from that
firm/state/day of the week.
I have to confess that these exchanges drive me nuts. Devoid of context, the topline polling figures (Obama X%, Clinton Y%) don't mean very much at all. The most useful feature of polling, particularly with substantial time still to elapse before an election, is the demographic trends and breakdowns. Pollsters, after all, are using a series of assumptions about the composition of the electorate in order to produce the horse-race numbers. Detailing those assumptions, and then discussing whether or not they're plausible, is a far better way to assess the validity or predictive utility of a given poll than: I dislike the result, therefore I dislike the poll.

So, as part of my continuing effort to explore the polling of Election 2008, I've decided to devote today's post to a detailed discussion of a recent spate of polls from a pair of key battleground states.

Wisconsin:

First, let's pay a visit to America's Dairyland. There have been three polls from Wisconsin during the month of February, and they don't quite add up. Let's take them apart, one at a time.

The first, from ARG, was taken on the two nights following Super Tuesday, and found Clinton leading 50-41%. I wish I knew what to make of that, but alas, ARG releases only the two questions it asks and the sample size, and not the composition of the sample or how it was selected, let alone its non-existent crosstabs. It's a datapoint, to be sure, but it's a week old, and from about the least reliable polling operation in America. So let's set it aside for the moment.

The headline of the second poll, from the Democratic firm PPP, trumpets the reverse result: Obama up, 50-39%. But not so fast. The write-up of the poll continues:

“Our poll shows that if there was standard turnout for the primary, Obama would
lead Clinton 46-42...But since we believe record numbers of young voters and
African Americans will turn out to support him, the weighted result gives Obama
a greater 50-39 advantage.”
Excuse me? In layman's terms, they're saying: "We polled people, and found Obama ahead 46-42. But since Obama did better than certain polls predicted in some other states, we tinkered with the results to inflate his margin." Fortunately, PPP gives us crosstabs, and some back-of-the-envelope math allows us to roughly quantify what they're saying. Their adjusted sample presumes that 12% of primary voters will be black. For context, in the 2004 primary, black voters were 6% of the Democratic electorate, roughly their proportion of the total population (and just 3% in 1992). To double their share of the primary electorate in a year when overall turnout may itself double, black voters would have to turn out at four times the rate of 2004. That, despite the fact that as a reliable segment of the Democratic base, their share of the vote has actually decreased in some states this year as less-reliable voters flood the polls. Am I the only one who sees a problem here? I suspect the youth voter numbers are similarly nuts, but can't prove it, because PPP doesn't provide enough information to allow for the calculation.

Fortunately for Obama-philes, there's yet another problem with the poll: it calculates that Republicans and independents will comprise just 23.5% of the voters. That may sound high, but in 2004, they accounted for fully 38% of the ballots cast. (In 1992, they were 47%, but let's not go there.) I'd hazzard a guess that this is what comes of gerrymandering your samples - artificially inflating the number of black and young voters may also inflate the percentage of Democratic voters, and thus depress independent and Republican turnout estimates. Adjusting partisan turnout to match 2004 gives Obama a 53-38% lead in the poll. So in conclusion, according to PPP, Obama is either locked in a tight race or poised for a blowout win. Great, huh?

Two more polls for you. One was taken by the Republican firm Strategic Vision from Feb 8-10, and it found Obama leading 45-41%. Alas, they tell us nothing about their sample, so we should assign that result roughly the same weight as the first two. The other was released this morning by Rasmussen Reports, and shows Obama leading 47-43%. Rasmussen is stingy with its demographic breakdowns - it charges a hefty premium to see them - so I don't know what to make of that result, either.

So is the Copper State a total cypher? Not quite. There is some value in the polling we've seen. The PPP survey suggests that Obama is running extremely strongly among independents and Republicans who are likely to vote in the Democratic primary, garnering 63% of the support from the members of each group. That's consistent with his extremely strong performance with crossover voters around the country. I suspect that Wisconsin may not be quite as close as these polls make out, but since they don't (or won't) tell me how they're projecting partisan turnout, I can't say for sure. It makes sense to wait for some more detailed polling before we assume that the cheeseheads are going to back Obama. For now, mark this one surprisingly close.

Ohio:

On to the Buckeye State. The Columbus Dispatch conducted a meticulous poll from Jan 23-31, which would be fascinating if it hadn't included John Edwards. Since it did, about all we can take away from the 42-19-18 Clinton-Obama-Edwards split is that this didn't start off as friendly territory for the midwestern senator.

That leaves us with three recent polls that show very similar results. The first, by SurveyUSA, was taken from Feb. 10-11, and found Clinton up 56-39%. The second was released today by the more-reliable Quinnipiac Polling Institute, and put Clinton's lead at a commanding 55-34%. The third, just out from Rasmussen Reports, puts Clinton ahead 51-37%. The polls share some common ground - two show Clinton leading almost 2-1 among white voters, and all three confirm that yet again, Obama is running best among men and voters under the age of 45, although in Ohio he's still losing both categories.

Let me throw out two reasons for caution concerning these early polling results. The first is my old hobby-horse, the Reverse Bradley/Wilder Effect. In a nutshell, it's the observation that Obama has invariably done better on election day among black voters than virtually any pre-election polls have predicted. In this case, the Q-Poll puts his margin at 64-17%, and SurveyUSA at 73-24% (Rasmussen didn't publicly announce its racial crosstabs). Those may sound like impressive percentages, but Obama has lately been taking roughly 9 out of 10 votes from the black community. In 2004, black voters were 14% of the Ohio Democratic primary electorate, so give the man a 3-point bump right there.

I'm also skeptical of some of the other projections. Consider that SurveyUSA finds the gender breakdown in Ohio likely to be 59-41; the Q-Poll doesn't give its breakdown, but we can infer from its numbers that a shocking two-thirds of its respondents were women. For a little context, that divide was just 52-48 in the 2004 Democratic primary. SurveyUSA projects the partisan breakdown between Democrats and Republicans/Independents will stand at 81-17 (compared to 72-28 in 2004). Now, these pollsters are in a bit of a bind. Earlier on, I ripped PPP for tinkering with its results to meet expectations; now I'm criticizing two other firms for releasing results that fail to gibe with expectations. But I'm not entirely inconsistent. I don't mind Qunnipiac and SurveyUSA telling us that they're projecting a huge surge among women voters, and an enormous decline in the percentage of independents. If that's what their polls show, maybe it will happen. I take issue with their trumpeting the top line results (Clinton ahead in Ohio!) without noting the extent to which they're an artifact of the demographic splits (Clinton ahead! Women to set turnout record! Independents vanish!) Let's recalculate those polls, assuming another 52-48 gender split: Q-Poll, 54-36%; SurveyUSA, 54-40%. Now, let's give Obama 3 points for his undercounted black vote, subtracting 1 from Clinton: Quinnipiac, 53-39%, SurveyUSA, 51-43%. Those numbers look about right to me. They give Clinton a robust, statistically-significant lead in Ohio, but not the twenty-point margin she's been hoping for. They're also roughly in line with Rasmussen, for whatever that's worth.
(A note of caution: here, and elsewhere, I've done some rough recalculation of polling figures for illustrative purposes. This is a useful tool for assessing the impact of the demographic composition of the sample, but it does not produce robust or predictive results. I don't have access to the raw data, and possess neither the training nor the experience necessary to re-weight samples, even if I did.)

Take Home Lessons:

So it looks like Wisconsin remains quite close, despite Obama's national momentum. The key for his campaign in that state will be drawing independents to the polls; if he can turn them out in percentages that rival prior primaries, he should win easily. For Clinton, the key to Wisconsin is white working-class voters, as it is pretty much everywhere else. Ohio is an interesting case. Here, Clinton needs to accomplish what she's failed to achieve almost anywhere else - she needs women voters to comprise an unusually high percentage of the electorate. Most states where that's happened, Clinton's been crushed; that's because it's generally been a straight function of exceptionally high black turnout, since black women vote at far higher rates than black men. In Ohio, Clinton needs to draw white women to the polls, and that will be challenging. Quinnipiac and SurveyUSA suggest she may be succeeding; the margin of her success will ride on that effort. If Obama wants to close the gap, he needs to turn out black voters, and continue to win virtually all of their votes. He also needs to make further inroads among white men; he hasn't done nearly as well with that constituency in Ohio as he has in recent states that he's won. That's an uphill struggle. If there's going to be an upset on March 4, these early polls suggest it won't be in Ohio.

If you've enjoyed this, please share it with other readers by clicking the 'recommend this' link. You can find more analysis on my blog. As always, I welcome comments and corrections. And thanks to all who have contributed to the remarkably civil and well-informed recent conversations - I've enjoyed the dialogue immensely, and learned a great deal.

No comments: