Victory probability mapObama lead over time

Sunday, August 03, 2008

Wisdom about interpreting polls

This DailyKos post by "DemFromCT" has a lot of wisdom about reading polls. Some of the points there (Look at all the polls, Polls are a snapshot in time) are fundamental to the work I'm trying to do here. I think my only small quibble is that I believe that a snapshot in time today can be boundedly predictive of future results, since public opinion has a measurable amount of volatility.

The statement that state polling is more difficult than national polling is surprising to me, and I'm curious why DemFromCT thinks that, or what the evidence is. Anecdotally, from things I've been involved in, I can remember both the Star Tribune's poll on the 1998 MN gubernatorial race and Terry Madonna's poll on the 2006 PA-08 congressional race being way off the mark (speaking of Terry Madonna, who died and made him Grand Guru of Pennsylvania Politics?), but the dual of "anecdote" is not "data." I actually would have guessed that a national poll or a poll in a large heterogeneous state would be more difficult than a poll in a small homogeneous state. In a place like CA, a pollster has to account for different populations of people having different access to phones, speaking different languages, having different propensities to answer surveys, etc.; while in a place like ND, there's basically just one undifferentiated population.

One thing that DemFromCT emphasizes, but that I've never paid much attention to, is the difference between LV and RV (and, god forbid, A) polls. Now that I've worked through parameter-setting, something I might try to figure out is how to measure the uncertainty implied by the difference between LV and RV results. For now, my methodology is simplistic: If a pollster only gives one result, I use it; if a pollster gives both LV and RV results, I use RV for polls before the conventions end, and LV for polls after the conventions end.

2 Comments:

Anonymous DemFromCT said...

the problem with state polls is getting a good representative sample.

When pollsters with no experience in a state come in, they don't always get it right, might not weight the different regions correctly, etc. And you might not know their track record.

So an average of 5 polls when 2 are garbage is worse than one good poll, but you need to know which the 'good one' is.

As an example, while it's a caucus scenario, Selzer in IA (and no other poll) is the one to watch before IA decides. And Field is preternaturally correct in CA. Averaging others in just makes it less accurate.

So, for each of 50 states, you need to know this 'local' info.

August 4, 2008 9:49 AM  
Anonymous myspace polls said...

I think polls can be very decieving. I actually learned this in one of my math classes. You have to be very careful in interpreting polls carefully because a simple mistake could lead to a drastically different result! This of course depends on the type of poll. There can be regular polls where you are looking for numbers of some people creating myspace polls asking friends for answers to everyday questions. Those types of polls are alright to interpret almost however you like!

August 4, 2008 5:31 PM  

Post a Comment

<< Home