As someone interested in health and wellness, you’ve probably noticed how often the drugs that are prescribed for us by doctors and the food supplements and vitamins that we buy in health food stores advertise their benefits based on “scientific evidence.” The manufacturers of these drugs and health products trot out the results of innumerable studies that “prove” how beneficial they are. But are these studies really true? Do they really constitute “proof?”
The POM Wonderful lawsuit decision
A recent legal decision casts doubt on the assumption that they are true. The manufacturers of POM Wonderful pure pomegranate juice advertised its benefits heavily, and rather effectively – they managed to sell $248 million dollars of it. In the ads, they depended heavily on scientific evidence – nearly 100 studies, 70 of them published in peer-reviewed journals. Based on these studies, they claimed that their juice could improve or even cure diseases such as cancer, heart disease, and erectile dysfunction.
Then the US Federal Trade Commission (FTC) sued them. And in a landmark decision, the judge trying the case ruled that there was not enough evidence to prove POM’s claims. His reasoning was based on one simple fact – POM paid for every one of the 100 studies, therefore increasing the possibility of bias. If the people performing the studies depended for their livelihood on the company paying for their research, would they be able to keep bias from creeping into their studies, bias that caused them to make their “findings” appear more positive?
The fallout or implications of this decision
If this judge’s ruling were applied to studies “proving” the safety or effectiveness of drugs approved by the Federal Drug Administration (FDA), and thus then prescribed by doctors to millions of people, 90% of these drugs would be found to have similarly insufficient evidence proving their worth. And for the same reason – the drug companies paid for every single one of the preliminary studies, and there have been no follow-up studies replicating the original research.
A remedial primer on the scientific method, and how it works
Many people are under the impression that the first study on a subject actually “proves” something. That’s not true. The scientific method depends on the assumption that this early research be considered preliminary, and that it only becomes “proof” when the research is replicated by other researchers, who find the same results that the first researchers did.
Got that? Now, some hard facts
• Over 90% of all cancer research cannot be replicated. Attempts to do so either fail to support the preliminary studies, or result in completely contradictory findings.
• In an exhaustive review of PubMed research conducted by the Center for Pharmacoeconomic Research, they found 742 published studies that later had to be retracted by the publishing journals because of scientific misconduct (data falsification, data fabrication, questions of data veracity, unethical author conduct, or plagiarism) or error (scientific mistakes or journal/peer review failure).
• Retractions don’t get read by the doctors prescribing the medicines. In a famous reversal, the respected British publication The Lancet retracted an earlier study in which it was claimed that two popular drugs to treat high blood pressure worked better in combination than they did on their own. Years later, over 10,000 patients are still taking the drug combo that has now been proven ineffective, because their doctors never read the retraction.
According to research published in Pharmacotherapy, the authors found that while the number of drug studies published during the period they studied increased by 44%, the number of retractions increased ten-fold. They also stated, “We found that the proportion of retractions for scientific misconduct was much higher in drug therapy studies (72%) than in the broader biomedical literature as reported in the 1998 study (37%) and in another study in 2011 (27%).”
So what do we DO about this? What can we believe?
Unfortunately, there is no easy solution. Part of the reason is economic. We know that the drug companies and the manufacturers of vitamins and food supplement pay for the studies that (surprise!) find them to be beneficial. But who pays for the follow-up studies to replicate them?
The benefits attributed to these drugs and health care products are not facts until the preliminary research has been replicated. But how do we – as consumers and as people concerned about our own health and that of our families – know that it has been replicated? The sad answer is that today, given the way that the publication/peer review process works, we don’t.
We have to fall back on common sense, and whatever research we can perform over the Internet. So when you see the next article that claims that such-and-such has been “proven” to cure such-and-such, don’t automatically believe it. Don’t accept what the writer of the article says – read the original research. What you’ll often find is that the study authors say outright that theirs are preliminary findings, and that they only suggest benefits. Whereas the articles in the popular press and on the Internet leave that part out, and publish the benefits as if they’re facts.
They’re not. They’re preliminary findings, that only hint at or suggest something. They won’t become actual facts until the original research has been replicated – hopefully several times, and not by researchers paid by the same company that paid for the original research.