{"id":36205,"date":"2016-10-31T01:00:38","date_gmt":"2016-10-31T05:00:38","guid":{"rendered":"http:\/\/quotulatiousness.ca\/blog\/?p=36205"},"modified":"2016-10-28T09:57:35","modified_gmt":"2016-10-28T13:57:35","slug":"is-the-gold-standard-of-peer-review-actually-just-fools-gold","status":"publish","type":"post","link":"https:\/\/quotulatiousness.ca\/blog\/2016\/10\/31\/is-the-gold-standard-of-peer-review-actually-just-fools-gold\/","title":{"rendered":"Is the “Gold Standard” of peer review actually just Fool’s Gold?"},"content":{"rendered":"

Donna Laframboise<\/a> points out that it’s difficult to govern based on scientific evidence if that evidence isn’t true:<\/p>\n

We\u2019re continually assured that government policies are grounded in evidence, whether it\u2019s an anti-bullying programme in Finland, an alcohol awareness initiative in Texas or climate change responses around the globe. Science itself, we\u2019re told, is guiding our footsteps.<\/p>\n

There\u2019s just one problem: science is in deep trouble. Last year, Richard Horton, editor of the Lancet<\/em>, referred to fears that \u2018much of the scientific literature, perhaps half, may simply be untrue\u2019 and that \u2018science has taken a turn toward darkness.\u2019<\/p>\n

It\u2019s a worrying thought. Government policies can\u2019t be considered evidence-based if the evidence on which they depend hasn\u2019t been independently verified, yet the vast majority of academic research is never put to this test. Instead, something called peer review takes place. When a research paper is submitted, journals invite a couple of people to evaluate it. Known as referees, these individuals recommend that the paper be published, modified, or rejected.<\/p>\n

If it\u2019s true that one gets what one pays for, let me point out that referees typically work for no payment. They lack both the time and the resources to perform anything other than a cursory overview. Nothing like an audit occurs. No one examines the raw data for accuracy or the computer code for errors. Peer review doesn\u2019t guarantee that proper statistical analyses were employed, or that lab equipment was used properly. The peer review process itself is full of serious flaws, yet is treated as if it\u2019s the handmaiden of objective truth.<\/p>\n

And it shows. Referees at the most prestigious of journals have given the green light to research that was later found to be wholly fraudulent. Conversely, they\u2019ve scoffed at work that went on to win Nobel prizes. Richard Smith, a former editor of the British Medical Journal<\/em>, describes peer review as a roulette wheel, a lottery and a black box. He points out that an extensive body of research finds scant evidence that this vetting process accomplishes much at all. On the other hand, a mountain of scholarship has identified profound deficiencies.<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"

Donna Laframboise points out that it’s difficult to govern based on scientific evidence if that evidence isn’t true: We\u2019re continually assured that government policies are grounded in evidence, whether it\u2019s an anti-bullying programme in Finland, an alcohol awareness initiative in Texas or climate change responses around the globe. Science itself, we\u2019re told, is guiding our […]<\/p>\n","protected":false},"author":1,"featured_media":35193,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[65,84,66,16],"tags":[434,513,290],"class_list":["post-36205","post","type-post","status-publish","format-standard","hentry","category-environment","category-government","category-health-science","category-science","tag-peerreview","tag-research","tag-statistics"],"jetpack_featured_media_url":"https:\/\/quotulatiousness.ca\/blog\/wp-content\/uploads\/2016\/06\/favicon.png","jetpack_shortlink":"https:\/\/wp.me\/p2hpV6-9pX","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/36205","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/comments?post=36205"}],"version-history":[{"count":1,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/36205\/revisions"}],"predecessor-version":[{"id":36206,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/posts\/36205\/revisions\/36206"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/media\/35193"}],"wp:attachment":[{"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/media?parent=36205"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/categories?post=36205"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/quotulatiousness.ca\/blog\/wp-json\/wp\/v2\/tags?post=36205"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}