{"id":16818,"date":"2012-09-07T08:20:03","date_gmt":"2012-09-07T13:20:03","guid":{"rendered":"http:\/\/quotulatiousness.ca\/blog\/?p=16818"},"modified":"2012-09-07T08:20:03","modified_gmt":"2012-09-07T13:20:03","slug":"when-i-discover-something-surprising-in-data-the-most-common-explanation-is-that-i-made-a-mistake","status":"publish","type":"post","link":"https:\/\/quotulatiousness.ca\/blog\/2012\/09\/07\/when-i-discover-something-surprising-in-data-the-most-common-explanation-is-that-i-made-a-mistake\/","title":{"rendered":"&#8220;When I discover something surprising in data, the most common explanation is that I made a mistake.&#8221;"},"content":{"rendered":"<p><a href=\"http:\/\/www.johnkay.com\/2011\/08\/24\/sex-lies-and-pitfalls-of-overblown-statistics?utm_source=dlvr.it&#038;utm_medium=twitter\" target=\"_blank\">John Kay<\/a> suggests you always ask how a statistic was created before you consider what the presenter wants you to think:<\/p>\n<blockquote><p>Always ask yourself the question: \u201cwhere does that data come from?\u201d. \u201cLong distance rail travel in Britain is expected to increase by 96 per cent by 2043.\u201d Note how the passive voice \u201cis expected\u201d avoids personal responsibility for this statement. Who expects this? And what is the basis of their expectation? For all I know, we might be using flying platforms in 2043, or be stranded at home by oil shortages: where did the authors of the prediction acquire their insight?<\/p>\n<p>\u201cOn average, men think about sex every seven seconds.\u201d How did the researchers find this out? Did they ask men how often they thought about sex, or when they last thought about sex (3\u00bd seconds ago, on average)? Did they give their subjects a buzzer to press every time they thought about sex? How did they confirm the validity of the responses? Is it possible that someone just made this statement up, and that it has been repeated frequently and without attribution ever since? Many of the numbers I hear at business conferences have that provenance.<\/p>\n<p>[. . .]<\/p>\n<p>Be careful of data defined by reference to other documents that you are expected not to have read. \u201cThese accounts have been compiled in accordance with generally accepted accounting principles\u201d, or \u201cthese estimates are prepared in line with guidance given by HM Treasury and the Department of Transport\u201d. Such statements are intended to give a false impression of authoritative endorsement. A data set compiled by a national statistics organisation or a respected international institution such as the Organisation for Economic Co-operation and Development or Eurostat will have been compiled conscientiously. That does not, however, imply that the numbers mean what the person using them thinks or asserts they mean.<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>John Kay suggests you always ask how a statistic was created before you consider what the presenter wants you to think: Always ask yourself the question: \u201cwhere does that data come from?\u201d. \u201cLong distance rail travel in Britain is expected to increase by 96 per cent by 2043.\u201d Note how the passive voice \u201cis expected\u201d [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[831,25,84,28],"tags":[213,51,290,101],"class_list":["post-16818","post","type-post","status-publish","format-standard","hentry","category-business","category-economics","category-government","category-media","tag-newspapers","tag-pr","tag-statistics","tag-tv"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p2hpV6-4ng","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/16818","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/comments?post=16818"}],"version-history":[{"count":1,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/16818\/revisions"}],"predecessor-version":[{"id":16819,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/16818\/revisions\/16819"}],"wp:attachment":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/media?parent=16818"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/categories?post=16818"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/tags?post=16818"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}