Quotulatiousness

July 23, 2014

In statistical studies, the size of the data sample matters

Filed under: Food, Health, Science, USA — Tags: , , , , — Nicholas @ 08:49

In the ongoing investigation into why Westerners — especially North Americans — became obese, some of the early studies are being reconsidered. For example, I’ve mentioned the name of Dr. Ancel Keys a couple of times recently: he was the champion of the low-fat diet and his work was highly influential in persuading government health authorities to demonize fat in pursuit of better health outcomes. He was so successful as an advocate for this idea that his study became one of the most frequently cited in medical science. A brilliant success … that unfortunately flew far ahead of its statistical evidence:

So Keys had food records, although that coding and summarizing part sounds a little fishy. Then he followed the health of 13,000 men so he could find associations between diet and heart disease. So we can assume he had dietary records for all 13,000 of them, right?

Uh … no. That wouldn’t be the case.

The poster-boys for his hypothesis about dietary fat and heart disease were the men from the Greek island of Crete. They supposedly ate the diet Keys recommended: low-fat, olive oil instead of saturated animal fats and all that, you see. Keys tracked more than 300 middle-aged men from Crete as part of his study population, and lo and behold, few of them suffered heart attacks. Hypothesis supported, case closed.

So guess how many of those 300-plus men were actually surveyed about their eating habits? Go on, guess. I’ll wait …

And the answer is: 31.

Yup, 31. And that’s about the size of the dataset from each of the seven countries: somewhere between 25 and 50 men. It’s right there in the paper’s data tables. That’s a ridiculously small number of men to survey if the goal is to accurately compare diets and heart disease in seven countries.

[…]

Getting the picture? Keys followed the health of more than 300 men from Crete. But he only surveyed 31 of them, with one of those surveys taken during the meat-abstinence month of Lent. Oh, and the original seven-day food-recall records weren’t available later, so he swapped in data from an earlier paper. Then to determine fruit and vegetable intake, he used data sheets about food availability in Greece during a four-year period.

And from this mess, he concluded that high-fat diets cause heart attacks and low-fat diets prevent them.

Keep in mind, this is one of the most-cited studies in all of medical science. It’s one of the pillars of the Diet-Heart hypothesis. It helped to convince the USDA, the AHA, doctors, nutritionists, media health writers, your parents, etc., that saturated fat clogs our arteries and kills us, so we all need to be on low-fat diets – even kids.

Yup, Ancel Keys had a tiny one … but he sure managed to screw a lot of people with it.

H/T to Amy Alkon for the link.

No Comments

No comments yet.

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress