{"id":17524,"date":"2012-10-29T10:41:52","date_gmt":"2012-10-29T15:41:52","guid":{"rendered":"http:\/\/quotulatiousness.ca\/blog\/?p=17524"},"modified":"2012-10-29T10:41:52","modified_gmt":"2012-10-29T15:41:52","slug":"polls-are-less-accurate-thanks-to-a-9-response-rate-and-falling","status":"publish","type":"post","link":"https:\/\/quotulatiousness.ca\/blog\/2012\/10\/29\/polls-are-less-accurate-thanks-to-a-9-response-rate-and-falling\/","title":{"rendered":"Polls are less accurate thanks to a 9% response rate (and falling)"},"content":{"rendered":"<p>Iowahawk has been posting a daily Twitter update reminding people that the reliability of political polls is much lower than ever before:<\/p>\n<blockquote class=\"twitter-tweet tw-align-center\">\n<p>Your daily reminder: all political polls are still garbage. <a href=\"https:\/\/twitter.com\/search\/%239percentresponserate\">#9percentresponserate<\/a><\/p>\n<p>&mdash; David Burge (@iowahawkblog) <a href=\"https:\/\/twitter.com\/iowahawkblog\/status\/262910838367723520\" data-datetime=\"2012-10-29T13:36:43+00:00\">October 29, 2012<\/a><\/p><\/blockquote>\n<p><script src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<p>At <em>Macleans<\/em>, <a href=\"http:\/\/www2.macleans.ca\/2012\/10\/29\/mind-the-credibility-gap\/\" target=\"_blank\">Colby Cosh<\/a> digs a bit deeper to find out how polling organizations are responding to their approaching-flatline response rate:<\/p>\n<blockquote><p>The boffins are becoming increasingly reliant on \u201cnon-probability samples\u201d like internet panel groups, which give only narrow pictures of biased subsets of the overall population. The good news is that they can take many such pictures and use modern computational techniques to combine them and make pretty decent population inferences. \u201cObama is at 90 per cent with black voters in Shelbyville; 54 per cent among auto workers; 48 per cent among California epileptics; 62 per cent with people whose surnames start with the letter Z\u2026\u201d Pile up enough subsets of this sort, combined with knowledge of their relative sizes and other characteristics, and you can build models which let you guess at the characteristics of the entire electorate (or, if you\u2019re doing market research, the consumerate).<\/p>\n<p>As a matter of truth in advertising, however, pollsters have concluded that they shouldn\u2019t report the uncertainty of these guesses by using the traditional term \u201cmargin of error.\u201d There is an extra layer of inference involved in the new techniques: they offer what one might call a \u201cmargin of error, given that the modelling assumptions are correct.\u201d And there\u2019s a philosophical problem, too. The new techniques are founded on what is called a \u201cBayesian\u201d basis, meaning that sample data must be combined explicitly with a prior state of knowledge to derive both estimates of particular quantities and the uncertainty surrounding them.<\/p>\n<p>[. . .]<\/p>\n<p>Pollsters are trying very hard to appear as transparent and up-front about their methods as they were in the landline era. When it comes to communicating with journalists, who are by and large a gang of rampaging innumerates, I don\u2019t really see much hope for this; polling firms may not want their methods to be some sort of mysterious \u201cblack box,\u201d but the nuances of Bayesian multilevel modelling, even to fairly intense stat hobbyists, might as well be buried in about a mile of cognitive concrete. Our best hope is likely to be the advent of meta-analysts like (he said through tightly gritted teeth) Nate Silver, who are watching and evaluating polling agencies according to their past performance. That is, pretty much exactly as if they were \u201cblack boxes.\u201d In the meantime, you will want to be on the lookout for that phrase \u201ccredibility interval.\u201d As the American Association for Public Opinion Research says, it is, in effect, a \u201c[news] consumer beware\u201d reminder.<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>Iowahawk has been posting a daily Twitter update reminding people that the reliability of political polls is much lower than ever before: Your daily reminder: all political polls are still garbage. #9percentresponserate &mdash; David Burge (@iowahawkblog) October 29, 2012 At Macleans, Colby Cosh digs a bit deeper to find out how polling organizations are responding [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[28,53],"tags":[188,289,547,290],"class_list":["post-17524","post","type-post","status-publish","format-standard","hentry","category-media","category-politics","tag-electionwatch","tag-polls","tag-smartphones","tag-statistics"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p2hpV6-4yE","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/17524","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/comments?post=17524"}],"version-history":[{"count":7,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/17524\/revisions"}],"predecessor-version":[{"id":17531,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/17524\/revisions\/17531"}],"wp:attachment":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/media?parent=17524"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/categories?post=17524"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/tags?post=17524"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}