{"id":27773,"date":"2015-08-14T01:00:38","date_gmt":"2015-08-14T05:00:38","guid":{"rendered":"http:\/\/quotulatiousness.ca\/blog\/?p=27773"},"modified":"2018-09-17T22:27:26","modified_gmt":"2018-09-18T02:27:26","slug":"qotd-when-the-science-shows-what-you-want-it-to-show","status":"publish","type":"post","link":"https:\/\/quotulatiousness.ca\/blog\/2015\/08\/14\/qotd-when-the-science-shows-what-you-want-it-to-show\/","title":{"rendered":"QotD: When “the science” shows what you want it to show"},"content":{"rendered":"

To see what I mean, consider the recent tradition of psychology articles showing that conservatives are authoritarian while liberals are not. Jeremy Frimer, who runs the Moral Psychology Lab at the University of Winnipeg, realized that who you asked those questions about might matter — did conservatives defer to the military because they were authoritarians or because the military is considered a \u201cconservative\u201d institution? And, lo and behold, when he asked similar questions about, say, environmentalists, the liberals were the authoritarians<\/a>.<\/p>\n

It also matters because social psychology, and social science more generally, has a replication problem, which was recently covered in a very good article at Slate<\/em><\/a>. Take the infamous \u201cparadox of choice\u201d study that found that offering a few kinds of jam samples at a supermarket was more likely to result in a purchase than offering dozens of samples. A team of researchers that tried to replicate this — and other famous experiments — completely failed<\/a>. When they did a survey of the literature, they found that the array of choices generally had no important effect either way. The replication problem is bad enough in one subfield of social psychology that Nobel laureate Daniel Kahneman wrote an open letter<\/a> to its practitioners, urging them to institute tougher replication protocols before their field implodes. A recent issue of Social Psychology<\/em><\/a> was devoted to trying to replicate famous studies in the discipline; more than a third failed replication.<\/p>\n

Let me pause here to say something important: Though I mentioned bias above, I\u2019m not suggesting in any way that the replication problems mostly happen because social scientists are in on a conspiracy against conservatives to do bad research or to make stuff up. The replication problems mostly happen because, as the Slate<\/em> article notes, journals are biased toward publishing positive and novel results, not \u201cthere was no relationship, which is exactly what you\u2019d expect.\u201d So readers see the one paper showing that something interesting happened, not the (possibly many more) teams that got muddy data showing no particular effect. If you do enough studies on enough small groups, you will occasionally get an effect just by random chance. But because those are the only studies that get published, it seems like \u201cscience has proved …\u201d whatever those papers are about. <\/p>\n

Megan McArdle, “The Truth About Truthiness”, Bloomberg View<\/em><\/a>, 2014-09-08.<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"

To see what I mean, consider the recent tradition of psychology articles showing that conservatives are authoritarian while liberals are not. Jeremy Frimer, who runs the Moral Psychology Lab at the University of Winnipeg, realized that who you asked those questions about might matter — did conservatives defer to the military because they were authoritarians […]<\/p>\n","protected":false},"author":1,"featured_media":35193,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[28,41,16],"tags":[39,477,213,139,1233,101],"class_list":["post-27773","post","type-post","status-publish","format-standard","hentry","category-media","category-quotations","category-science","tag-junkscience","tag-magazines","tag-newspapers","tag-psychology","tag-replicationcrisis","tag-tv"],"jetpack_featured_media_url":"https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2016\/06\/favicon.png","jetpack_shortlink":"https:\/\/wp.me\/p2hpV6-7dX","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/27773","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/comments?post=27773"}],"version-history":[{"count":1,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/27773\/revisions"}],"predecessor-version":[{"id":27774,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/27773\/revisions\/27774"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/media\/35193"}],"wp:attachment":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/media?parent=27773"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/categories?post=27773"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/tags?post=27773"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}