This guy (who really really doesn’t need my help, link-wise, but etiquette demands) makes a good point about a new poll with a small sample size, whose built-in inaccuracy is, as they say, a feature rather than a bug. Remember this when you’re poring over each morning’s numbers.
More important than that, though, is this: when you read a headline like McCain, Obama Tied you should snort with derisive laughter. Leave aside the question of why we’re following national opinion polls in the first place, when the race isn’t settled by a national vote. There’s still the rather shifty question of what relation these numbers have with the actual facts that they’re attempting to describe.
Look, it’s complicated, okay, and I’m sure a room full of real statistics wonks could argue about it all day (and school me quite thoroughly in the process). But at any moment the world is in a hopelessly complex state, with a hundred-some-million voters all in varying states of voting-for-X-ness and voting-for-Y-ness and don’t-know-ness and won’t-vote-anyway-ness. This is what we’re trying to gauge with these polls — a very very rough sketch that hopefully resembles the underlying reality in a meaningful way. (Is there an underlying reality, really? What form does this resemblance take? Are you a frequentist or a Bayesian? I don’t even know!)
The folksy, homespun analogy is this: what CBS is saying is that they stuck the thermometer in the turkey and the dial went to 42. Per cent, I suppose; it’s a strange thermometer. Leave that out of it for a second. Does that mean the turkey “is” 42 degrees, or whatever? What about the thigh? What about the breast?
Yeah, this is what “margin of error” is for. But still. If you stuck a meat thermometer in a turkey’s leg and it reads 160º, and then five minutes later George Gallup told you his fancy infrared thermometer reported a surface temperature of 205º, would this really help you draw any kind of inference about the state of the turkey?
If you’re addicted to the polling numbers, by all means, knock yourself out. At the very least it might be more informative to follow the trends in the tracking polls, rather than just the one-day numbers — at least they’re trying to repeatedly stick the same thermometer in the same spot each time. Do remember, of course, that the turkey is a complicated thing, and there’s more going on in there than a thermometer can hope to measure. And it doesn’t help that thanks to the electoral college, certain parts of the turkey matter a whole lot more than others. Or that some thermometers might not be quite up front about their racial views, for example.
Best yet, ignore the polls entirely, and take a look at some numbers that really matter.