We’ve all seen many examples of health news stories where the headline promised much more than the article delivered: this is why stories have headlines in the first place — to get you to read the rest of the article. This sometimes means the headline writer (except on blogs, the person writing the headline isn’t the person who wrote the story), knowing less of what went into writing the story, grabs a few key statements to come up with an appealing (or appalling) headline.
This is especially true with science and health reporting, where the writer may not be as fully informed on the subject and the headline writer almost certainly doesn’t have a scientific background. The correct way to read any kind of health report in the mainstream media is to read skeptically — and knowing a few things about how scientific research is (or should be) conducted will help you to determine whether a reported finding is worth paying attention to:
Does the article support its claims with scientific research?
Your first concern should be the research behind the news article. If an article touts a treatment or some aspect of your lifestyle that is supposed to prevent or cause a disease, but doesn’t give any information about the scientific research behind it, then treat it with a lot of caution. The same applies to research that has yet to be published.
Is the article based on a conference abstract?
Another area for caution is if the news article is based on a conference abstract. Research presented at conferences is often at a preliminary stage and usually hasn’t been scrutinised by experts in the field. Also, conference abstracts rarely provide full details about methods, making it difficult to judge how well the research was conducted. For these reasons, articles based on conference abstracts should be no cause for alarm. Don’t panic or rush off to your GP.
Was the research in humans?
Quite often, the ‘miracle cure’ in the headline turns out to have only been tested on cells in the laboratory or on animals. These stories are regularly accompanied by pictures of humans, which creates the illusion that the miracle cure came from human studies. Studies in cells and animals are crucial first steps and should not be undervalued. However, many drugs that show promising results in cells in laboratories don’t work in animals, and many drugs that show promising results in animals don’t work in humans. If you read a headline about a drug or food ‘curing’ rats, there is a chance it might cure humans in the future, but unfortunately a larger chance that it won’t. So there is no need to start eating large amounts of the ‘wonder food’ featured in the article.
How many people did the research study include?
In general, the larger a study the more you can trust its results. Small studies may miss important differences because they lack statistical “power”, and are also more susceptible to finding things (including things that are wrong) purely by chance.
[…]
Did the study have a control group?
There are many different types of studies appropriate for answering different types of questions. If the question being asked is about whether a treatment or exposure has an effect or not, then the study needs to have a control group. A control group allows the researchers to compare what happens to people who have the treatment/exposure with what happens to people who don’t. If the study doesn’t have a control group, then it’s difficult to attribute results to the treatment or exposure with any level of certainty.
Also, it’s important that the control group is as similar to the treated/exposed group as possible. The best way to achieve this is to randomly assign some people to be in the treated/exposed group and some people to be in the control group. This is what happens in a randomised controlled trial (RCT) and is why RCTs are considered the ‘gold standard’ for testing the effects of treatments and exposures. So when reading about a drug, food or treatment that is supposed to have an effect, you want to look for evidence of a control group and, ideally, evidence that the study was an RCT. Without either, retain some healthy scepticism.
