Tuesday, October 2, 2012

Should we believe the polls?

Maybe this was last week's topic, meaning that I'm late to the fray. Nonetheless, I'll share my thoughts on whether the polls for this presidential election season are believable.

My first thought was that the polls are unreliable because the polling companies don't know who is going to show up to vote. Will the surge of Republican-leaners be very large? Will the counter-surge of Dem supporters be even larger? NO ONE knows this in advance, so no one can accurately poll. The polls are guesswork at this point--you guess who is going to turn out, and try to poll on those demographics.

It turns out that this impression (my impression of how polls are conducted) isn't correct. According to this detailed article, pollsters poll a random sample of everybody, not a chosen sample based on their hunches of who is going to vote. Pollsters learn who intends to vote only after running the random poll. Duh, that makes sense.

So perhaps the supposed skewed polls, the ones that "are in the tank" for Obama, well, maybe they aren't skewed. But don't tell that to diehard conservative paranoiacs. Their argument is that the MSM is terribly skewed, and it's not much of a step to posit that polling companies are skewed too. Thus, the need to unskew polls, which this site helpfully does by recalculating the sample to fit the demographics that are expected at the voting booth. Yes, those demographics that NO ONE KNOWS IN ADVANCE.

My conclusions: Doubts about polls are reasonable. Unskewing polls is ludicrous.

Image: davidmixner.com

1 comment:

Anonymous said...

Some pollsters, like Rasmussen, DO weight their sample based on expected party identification turnout. The "expected" part is where they can tilt their results, along with they basis for deciding if someone is a "likely" voter or not.

I don't pay much attention to the "likely voter" models for any of the polls, any more than I care what the self-reported party affiliation percentages say. A random sample should be a random sample. Pollsters should report their results -- with at least one decimal place if the difference is 10 points or less -- and leave all the rest of their predictive stuff out of it.

One thing they can do, and most do, is weight the results based on non-conditional factors against a) the population as a whole, b) standard exit-poll election year voter demographics. That way, if they oversample parties that are more likely to answer the phone from a random sample (e.g., older folks), the can adjust the results to better match the true population they are supposed to be representing. This is tricky, too, but also scientific in a way. The pollsters rely on U.S. census statistics, and widely available voting statistics to apply those weights. Hence, a 20-year-old's answers might not be counted as 1, but maybe 1.02 of so since that age cohort may be under-sampled per census statistics.

There is some guesswork here, as well, but the very good pollsters do a lot of science to reach those weights, eliminating any wishful-thinking or bias in them.

In the end, a poll of polls will do a decent job balancing the various factors and give you a decent snapshot of what the results would be if the votes were taken and counted today.

Early voting complicates this math, as does the opposing campaign's ground game. But a highly-contested presidential race in hot times, and with lots of opportunity, will tend to increase turnout across the board. If the result is pre-determined, many people will skip voting. But I'll bet in Florida, with early voting, a close election, and lots of issues, "likely" voters should be closer to "registered" voters.