Be Skeptical

Note to political consumers: view all polls released in next 72 hours with great skepticism. Very hard to poll now.

— mike murphy (@murphymike) October 31, 2014

Over the past two months, my home phone has been bombarded. Outreach contacts. Robo-call messages. Pollsters, so frequently as to defy the odds; no way could we be sampled so frequently. The closer we get to election day, the more frequent the phone rings. This past week, perhaps 5 calls a night. Last weekend, maybe 10 per day. I see no reason to think the trend will slow today or tomorrow.

It is always fun (and educational) to extrapolate out from a sample size of one, but I know that my wife refuses to answer the phone now and tells me I should let our answering machine screen things for us. It is easy for me to believe that the cacophony can influence response rates in a distorting fashion, and that the closer to election day it is, the more this is true.

I also find it easy to imagine that, as the election looms near, fewer people want to admit that they have tuned it out, or don't really care. The responses showing such disinterest, which can be used to screen out unlikely voters, may not be as frequent.

It does not take a huge change in either the response rate, nor in the answers to screening questions, to fundamentally change the quality of a poll sample as compared to those gleaned earlier in the cycle.

Throw on top of it that last night was Halloween, which is a night where who is home and who is not is just different than it normally is on a comparable day of the week.

Lastly, for the final surveys, there often is pressure to minimize the undecideds; to either put more effort into pushing leaners in the script, or to use some sort of modeling to allocate the coy.

I don't know which of these effects Mr. Murphy had in mind (or if there are others), but experience tells me he is right. The last election I did comprehensive, day-by-day analysis of the races was 2004, and the very last set of polls, all conducted in the last few days before the election, provided distorting information, not clarifying information. Experience had taught me before to view those with skepticism. I may not know the precise reason why he is, but Mr. Murphy is right. Bake in your perception of the races with the poll information you have on-hand right now, and just sit back until the results come in.

Edited to add: My warning above does not apply to polls that were previously in the field but concluded before the weekend (and just have not yet been published). Selzer, I am looking at you.