We’ve always known that polls had problems. You can get very different answers depending on how you ask the question, as Yes, Prime Minister effectively dramatized. Sampling problems arise when people who don’t get chosen for the poll, or refuse to respond, are systematically different from the rest of the population. (This is how the infamous “Dewey Defeats Truman” headline happened.) Even with problems, however, polls remain useful — as long as you keep those problems in mind.
I’ve seen a lot written today about how this shows the need to fix polling. I’ve seen few people asking what seems like the more pertinent question: What if polls can’t be fixed?
The second half of the 20th century was the golden age of survey data. But toward the end of the century, changing technology began to threaten the accuracy of polls. We now have caller ID, voicemail, and millennials who regard talking on the phone as a barbarism akin to the chamber pot. The modern American workday also compresses housekeeping and socializing into a few narrow hours, during which people are less likely to humor an unsolicited caller. In part because we’ve also seen the proliferation of robocalls in the survey industry and beyond.
Pollsters say that by carefully calibrating for the missing responders, they can still get accurate results. But what if it’s getting too hard?
In some sense, that doesn’t matter all that much for elections. For one thing, there are still betting markets, which, as my colleague Leonid Bershidsky points out, were doing an excellent job of predicting elections long before we had representative national surveys. For another, in the realm of politics we always get the answer we need eventually — on election day.
But there are broader issues where we don’t know the answers, and would very much like to, about the lives of the people who live within our borders, and what they want from their governments, businesses and civic institutions. Without good survey data, all those institutions will be operating blindly, groping toward answers they used to be able to get just by picking up the phone.
Megan McArdle, “Pollsters Are Worse Than Ever”, Bloomberg View, 2015-05-08.
December 20, 2016
QotD: The problem for pollsters
Comments Off on QotD: The problem for pollsters
No Comments
No comments yet.
RSS feed for comments on this post.
Sorry, the comment form is closed at this time.
