How to Handle Conflicting Health Advice
"Don't eat red meat? No! Don't eat no green meat!" –Chris Rock
As memes go, conflicting health advice has been around for a while. I can remember, back before the internet existed, how people would complain that every month a new study contradicts what we thought we knew.
In rare cases, the problem is simply corrupt science.
For instance, after sugar was linked to heart disease and obesity in the 1950s and 60s, the Sugar Research Foundation and other industry groups began paying researchers to conduct biased studies and reviews that shifted the blame to cholesterol and saturated fats. More recently, Coca-Cola used a similar strategy to help defend sugary beverages.
These industry-driven scientific activities influenced public perception. Up until the 1950s, sugar was often considered good for you. Then it was bad for a decade or two. Then, thanks to industry funding, it became not so bad, so long as you brushed your teeth. Nowadays it's bad again, and there's growing awareness of indirect effects, such as links to cancer. One of the proposed mechanisms is that excessive sugar consumption creates metabolic changes that promote obesity, and it's obesity that increases the risk of certain cancers. In other words, sugar per se isn't the essential culprit.
Although corporations sponsor research in all areas of health, I like to think that sugar is an exceptional case, and that most health studies are conducted by well-meaning scientists who may have personal biases but aren't corrupt. Why then are their findings so often contradictory?
This week and next, I'll be doing something a little different from what you usually find here. Instead of taking a deep dive into a single study, I'll be touching lightly on several that made the news recently. These studies help illustrate why health advice often seems contradictory – and how you can make the best of the available data.
I know this sounds like clickbait ("keep reading, and you'll discover the secret to healthier living"), but I'm not supported by advertisers, and I won't be revealing any secrets. I just want to share a statistics-oriented perspective on a topic many people care about.
This week I'll focus on a case study of conflicting advice about berberine, a weight-loss supplement that's all the rage on social media. Why do views on berberine differ, who should we believe, and what are the broader implications? Next week I'll create a broader, "consumer guide" to navigating conflicting health advice that will draw on recent studies pertaining to sleep, alcohol consumption, keto diets, cold showers, and even the "health" of the nation.
Nature's Ozempic
Semaglutide, sold under brand names such as Ozempic and Wegovy, was developed as an anti-diabetic medication but is known to genuinely promote weight loss, even if statistics on its effectiveness are misleading (see here for details).
We're in the midst of a semaglutide frenzy, but you can't buy one of the drugs without a prescription, and they can only be prescribed for weight loss "off label", meaning that insurance won't cover them. So, people who can't afford around a thousand dollars per month (i.e., most people) have been seeking cheaper, more accessible alternatives. Enter berberine, often called "nature's Ozempic", currently priced at $12 to $50 for a one-month supply.
Berberine, a plant extract sold as a supplement, has been used for centuries in Asian countries. Colleagues in the pharmacy school of the Chinese university where I taught for a year tell me that it's still used in China, mostly in conjunction with other drugs, for gut pain, bacterial infections, etc., though not for weight management.
In the U.S, berberine is touted by some as a miracle weight-loss drug. Others question its effectiveness and safety. Who should you believe?
You're probably expecting me to say that it's the TikTokkers and other social media communities who gush over berberine, whereas more credible sources – the ones you should believe – warn that existing research is too limited to draw conclusions.
Well, yes and no.
Yes, there's a ton of uncritical praise for berberine, and yes, the evidence that it promotes weight loss is slim.
But no, that evidence is not as limited as people claim. Recent articles in the Washington Post, Wall Street Journal, and Rolling Stone, for instance, have overstated the inconclusiveness of recent research.
In short, there are three positions on berberine and weight loss:
Berberine is an astonishingly effective weight-loss drug. If you want to slim down, take it! (The social media community.)
We have no idea yet how berberine affects weight. If you want to slim down, don’t take it yet! (Some experts, many journalists.)
Berberine probably promotes weight loss. If you want to slim down, consider taking it. (Some experts, and me.)
So, who should you believe, why should you believe them, and what does this tell us about the origins of conflicting health advice?
The evidence on berberine
Several articles published this month warn that there are too few studies on berberine, the samples are tiny, the methods are weak, and in some cases the focus is on animals. Additional concerns were aired in a recent Washington Post article:
"[M]uch of the research on berberine is limited, of low quality and ultimately inconclusive, said Carol Haggans, a registered dietitian and consultant for the NIH’s Office of Dietary Supplements. Many studies include small sample sizes and short study periods or have analyzed berberine in combination with other dietary supplements."
There's some truth to this statement, but it doesn't work well as a generalization.
For one thing, although the article hyperlinked at the end of the quote reviewed studies on the benefits of berberine in combination with other supplements, it also reviewed studies on berberine in isolation. The main finding was that berberine, all by itself, can reduce low-density lipoproteins (LDLs – the so-called "bad" cholesterols) as well as triglycerides (the most common type of fat in one's body).
Another thing that's misleading about that hyperlink is that the article was published in 2017. There have been dozens of peer-reviewed studies since then.
Some of the more recent studies have acceptable sample sizes, at least statistically speaking, and they do consider clinically meaningful periods of time. For example, a 2021 experiment followed 100 people with non-alcoholic steatohepatitis (fatty liver disease) and type 2 diabetes for a period of 18 weeks and found that, compared to a placebo, berberine increased insulin levels and decreased cell resistance to insulin, thereby lowering blood sugar.
If berberine lowers LDL cholesterol, triglycerides, and blood sugar, it is indeed a useful drug, but does it promote weight loss? The aforementioned changes are associated with weight loss among people with obesity, but that's not direct evidence of causality.
Such evidence does exist, though it's not as strong as one might like. Animal studies, including one published this April, and a 2020 meta-analysis focusing on 12 studies with humans have all shown that berberine leads to moderate weight loss and metabolic gains. Although the 12 studies point to the same conclusions, they're extremely heterogenous in terms of people studied, methods used, and overall reliability, and the meta-analysis itself was not very skillfully done.
Bottom line on berberine
Dozens of experimental studies have now looked at the impact of berberine on metabolic changes associated with weight loss (reductions in blood pressure, blood sugar, LDL cholesterol, triglycerides, etc.) as well as weight loss itself.
You might say then that we have a sample of studies. It's a noisy sample – not a very strong one on the whole – but the findings all point to a similar conclusion: Berberine positively impacts metabolism and weight. What you rarely see in this literature are studies showing that berberine is ineffective.
Wait, you might say. What about publication bias? Journals tend to only publish significant findings. Researchers consequently tend to discard non-significant findings and only attempt to publish significant ones. Maybe berberine isn't consistently effective; we merely see the experiments where it worked.
Publication bias is a widespread problem that leads to overestimates of treatment effectiveness. But it's not clear why this would be more problematic for berberine research than for any other type of study. And, in some of the studies I saw, researchers with no apparent financial conflicts of interest compared berberine to metformin (Glucophage), the main first-line treatment for diabetes. In studies like those, any finding would be publishable. It would be news, for example, if berberine turned out to be less effective than metformin, or not effective at all.
Clearly we need better research and better reviews. But the evidence is more than merely suggestive. My conclusion is that berberine probably does have metabolic benefits and promote weight loss. The effects are bubbling up in different labs, in spite of at least some of the studies relying on small samples and weak methods.
(By analogy, some of the early studies linking smoking to lung disease were weak, methodologically speaking, but the effects of smoking were strong enough that they could still be observed anyway. The signal was louder than the noise, you might say.)
So, if you want to lose weight, should you try berberine? Ultimately, that's the question on most peoples' minds. From what I've written here, it seems like the supplement does deserve a chance.
Reasons for caution
Before actually trying berberine, I would consider the following:
—As with Ozempic, the benefits of berberine are demonstrated by aggregate data such as group means. Berberine does not work for everyone, just as Ozempic does not (see here). Most of the berberine study samples consisted of people with health problems related to metabolism and weight.
—If berberine is effective, it may need to be taken for as long as one wishes to maintain a lower weight. As with Ozempic, it's not clear yet that once people go off the drug, the benefits will endure.
—Like Ozempic, berberine can have side effects, but it's less clear what the prevalence and severity of those side effects might be in the long run.
—As with other supplements, which are not FDA-approved, the actual berberine content of different brands varies widely. One study found that the average berberine content across 15 brands was only 75% of the advertised amount, with a range of 33% to 100%. So, anyone who does plan on taking the supplement should choose the brand carefully (data for the 15 brands can be found here.)
A quick digression
This is a deeply personal topic, but one might question the need to lose weight in the first place.
I realize that it's a million degrees outside right now – I'm thinking of my friends in Texas, but summer has become relatively brutal everywhere – and so it would be great to be able to wear as little as possible outside and still look hot (not just temperature-wise). But it's all too easy for Americans to succumb to unrealistic standards for our bodies. After all, the demand for Ozempic and other semaglutides is what drove interest in berberine in the first place.
The body positivity movement, including campaigns against "fat shaming", has experienced some pushback, but studies show that not everyone labeled "overweight" experiences undesirable metabolic conditions (elevated blood sugar, triglycerides, etc.) or greater risk for later health issues. Being a larger person doesn't necessarily make one an unhealthy person.
Part of the confusion comes from reliance on the Body Mass Index, or BMI, to label people as "overweight". As it turns out, the BMI is particularly bad at characterizing people in that middle category between average and obese (see here for more details).
The broader issues
Berberine helps illustrate why evidence-based health advice is so often contradictory, and how we might begin to sort out the inconsistencies.
Sometimes inconsistencies can be traced to the studies themselves. That will be my focus next week. In other cases, as with berberine, inconsistencies arise from the way study findings are presented and discussed by non-scientists. Specifically, here are three reasons why public discussion of health research creates inconsistencies:
1. Sources differ in reliability.
In the case of berberine, TikTokkers and other social media communities rave about berberine on the basis of tiny samples (themselves and possibly friends) or misconceptions about published research. Others who've looked at the research come to different conclusions and offer different advice.
The obvious solution to this problem is to avoid seeking health advice from social media. Better to trust a health professional, a reputable health organization, a respected news agency, or a strong independent journalist. (Dr. Katelyn Jetelina's Substack, for instance, is one of my favorite sources of advice about public health issues.) More on reliability in a moment.
2. Sources differ in accuracy.
Next week I'll have much more to say about accuracy and bias. Here I just want to note that even the sources you consider reliable should be treated critically. As I argued earlier, the Washington Post, a news organization with strong science coverage, fell short in its treatment of berberine. My reading of the data, and thus my advice, differed from the Post's.
3. Sources differ in thoroughness.
The Post article was well-written but overstated the inconclusiveness of the berberine research as a result of cherry-picking findings that supported its thesis. As I mentioned, the article overlooked evidence, some of it grounded in competent methodology, that berberine might positively impact metabolism and weight. Thus others, including me, have come to a different conclusion and offer different advice.
The accuracy and completeness problems are tricky, because we don't necessarily have time to look at the research literature ourselves (plus the studies are difficult to follow, and some of them are behind paywalls). We have to trust that the writer examined the literature carefully and made an attempt to represent it accurately and completely. This leads me back to the first point: We have to trust that the source is reliable. How do we accomplish that?
Reliable science coverage
Briefly, the most reliable coverage of a study or group of studies will provide the following:
—a sense of the scientific and practical reasons for conducting the study
—key methodological details about sampling, measurement, and procedures
—the main findings and their implications
—potential flaws or limitations of the study
—reactions from the scientific community
It's a good sign when experts are interviewed, since they help ensure that the writer is depicting the study accurately. Hearing about study flaws or limitations from an independent expert is especially useful. That dissenting voice helps us evaluate the data, and it gives us confidence that whoever is describing the study isn't excessively biased. And, experts tend to be a great source of "insider" knowledge as to why study findings conflict.
I forget whether carbs are good or bad this week, so I'm going to have a donut now. It's important to gather data.
Thanks for reading!