You Need To Be Your Health Advocate

Author: Dr. Stephen Chaney

I taught medical students for 40 years. During that time, I did my best to emphasize the importance of basing their practice on “evidence-based medicine”. So, it broke my heart when I saw recent headlines claiming that more than 90% of healthcare interventions (drugs and medical procedures) were not based on high-quality evidence.

Even worse, the headlines claimed that the harm caused by healthcare interventions was not adequately investigated and may, therefore, be under-reported.

When I saw these headlines, I knew I had to investigate further to see if the claims were true and report what I found to you, my readers.

I would not have been surprised by headlines claiming that some healthcare interventions were based on low-quality evidence. For example:

Hormone replacement therapy was widely prescribed to manage menopause symptoms until it was discovered to increase the risk of breast cancer. Since then, the hormones used have been reformulated, it is only recommended for severe menopause symptoms, and only for the shortest possible time.

Antiarrhythmic drugs were widely prescribed to reduce mortality from heart attacks until a placebo-controlled trial showed they actually increased mortality.

A drug called oseltamivir was widely prescribed for the flu until a systematic review of clinical studies showed it was ineffective.

But I, like many of my colleagues, assumed that these cases were rare. However, recent reviews have called this assumption into question. But most of those reviews had a small sample size or did not adequately evaluate the quality of the studies included in the review.

The study (J Howick et al, Journal of Clinical Epidemiology, 148: 160-169, 2022) behind these headlines was designed to avoid those limitations and provide a more accurate estimate of the percentage of clinical interventions that are based on high-quality evidence.

It evaluated 1,567 healthcare interventions that had been studied in Cochrane Reviews, which are considered the gold-standard of evidence-based medicine (I will describe Cochrane Reviews in more detail below, so you can appreciate why they are considered the gold standard).

What Is A Cochrane Review?

At this point you are probably wondering what the Cochrane Review is and why it is considered the gold standard of evidence-based medicine. I have covered this in previous articles. But I am not expecting you to remember it (I never told you there would be a quiz). So, I will repeat the information here.

The Cochrane Collaboration consists of 30,000 volunteer scientific experts from across the globe whose sole mission is to analyze the scientific literature and publish reviews of health claims so that health professionals, patients, and policy makers can make evidence-based choices about health interventions.

The Cochrane Collaboration reviews all the relevant studies on a topic, exclude those that are biased or weak, and make their recommendations based on only the strongest studies. They use a systematic approach called GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) that has been endorsed by over 100 organizations worldwide to assess the quality of the studies. The scientists writing Cochrane Reviews are trained in how to use the GRADE evaluation system before they are allowed to write a review.

In one sense, Cochrane reviews are what is called a “meta-analysis”, in which data from numerous studies are grouped together so that a statistically significant conclusion can be reached. However, Cochrane Collaboration reviews differ from most meta-analyses found in the scientific literature in a very significant way.

Many published meta-analyses simply report “statistically significant” conclusions. However, statistics can be misleading. As Mark Twain said: “There are lies. There are damn lies. And then there are statistics”.

The problem is that the authors of most meta-analyses group studies together without considering the quality of studies included in their analysis. This creates a “Garbage In – Garbage Out” effect. If the quality of individual studies is low, the quality of the meta-analysis will also be low. Simply put, the conclusions from some published meta-analyses are not worth the paper they are written on.

The Cochrane Collaboration also reports statistically significant conclusions from their meta-analyses. However, they also carefully consider the quality of each individual study in their analysis based on the GRADE system. They look at possible sources of bias. They look at the design and size of the studies. Finally, they ask whether the conclusions are consistent from one study to the next. They clearly define the quality of evidence that backs up each of their conclusions as follows:

High-quality evidence.Further research is unlikely to change their conclusion. This is generally reserved for conclusions backed by multiple high-quality studies that have all come to the same conclusion.

Moderate-quality evidence.This conclusion is likely to be true, but further research could have an impact on it.

Low-quality evidence.Further research is needed and could alter the conclusion. They are not judging whether the conclusion is true or false. They are simply saying more research is needed to reach a definite conclusion.

Now perhaps you understand why Cochrane Reviews are considered the gold standard of evidence-based medicine.

How Was This Study Done?

The authors started with 6928 reviews that compared a healthcare intervention with either a placebo or no intervention between January 1, 2008, and March 5, 2021. They then randomly selected 1,567 reviews for this study. They asked the following 3 questions for each Cochrane Review:

Was the evidence for a positive outcome high-quality, as rated by the GRADE system?

Were the results statistically significant?

Did the review authors consider the intervention to be effective?

Is Your Doctor’s Advice Based On Good Science?

As I said earlier, the results were unnerving to say the least. When the authors applied their 3 criteria to the 1,567 Cochrane Reviews they found:

Only 10% of the medical interventions (drugs and medical procedures) were supported by high-quality evidence.

- In other words, 90% of the time the evidence wasn’t good enough to determine whether the intervention worked or not.

Only 6.8% of the interventions studied had a positive, statistically significant outcome.

Only 5.6% of the medical interventions studied were judged to be effective by the Cochrane Review authors.

The harm of medical interventions was poorly studied. Only 36.8% of clinical interventions were evaluated for potential harms of the intervention, and most of those data were of low quality.

- Of the Cochrane Reviews that evaluated potential harms, there was evidence of statistically significant harm in 22% of the interventions.

The authors concluded,

“Using rigorous methods for judging quality of evidence, more than 9 in 10 healthcare interventions studied within Cochrane Reviews do not have high-quality evidence to support their effectiveness and safety. This probably can be remedied by high-quality studies in priority areas.”

“Potential harms of healthcare interventions were measured more rarely than benefits…These studies should measure harms as rigorously as benefits.”

“Practitioners and the public should be aware that most frequently used interventions are not supported by high-quality evidence.”

Putting This Study Into Perspective

I should start by saying that this study does not reflect poorly on your doctor. They have your best interest in mind, and they are doing their best to keep up with a constantly changing medical landscape.

In most cases, the advice your doctor gives you is based on clinical guidelines issued by medical societies and government agencies. This study is an indictment of those agencies for not evaluating the quality of the clinical studies used to formulate their clinical guidelines.

With that out of the way, it is fair to ask whether criteria these authors used were too strict. And, in fact, the authors gave this quite a bit of thought. Here are some of the questions the authors asked.

Were the authors of the Cochrane Reviews biased in their evaluation? While you can never eliminate the possibility of bias:

- The Cochrane Collaboration puts a great deal of effort into training reviewers in how to use the GRADE system without bias.

- If the opinions of the review authors were removed as a criterion, it would have a minimal impact on the outcome of this study. As reported above, only 6.8% of healthcare interventions had a positive outcome that was supported by high-quality data.

2) Is the GRADE system for evaluating the quality of clinical studies too stringent? The authors considered this possibility, but:

- Prior to GRADE individual meta-analyses used different methods to evaluate the quality of clinical studies, so it was difficult to compare the conclusions of these meta-analyses.

- The GRADE system was designed, evaluated, and accepted by top experts around the world to unify how the quality of clinical studies is evaluated.

- This study found that only 30% of the healthcare interventions were supported by even moderate quality evidence according to GRADE. In other words, even when less stringent standards are used, a high percentage of healthcare interventions may be ineffective.

3) Does relying solely on Cochrane Reviews underrepresent the percentage of healthcare interventions based on high-quality evidence?

- Cochrane Reviews are primarily undertaken for interventions that are controversial and/or a least one major study suggests the intervention may be ineffective or harmful. Since Cochrane Reviews are less likely to have been conducted on well-established, non-controversial healthcare interventions, it is possible that this study underrepresented the percentage of healthcare interventions backed by high-quality studies.

- However, a recent study that did not use Cochrane Reviews or the GRADE system concluded that only 22% of healthcare interventions were likely to be beneficial. Once again, even when less stringent standards are used, a high percentage of healthcare interventions may be ineffective.

However, the authors did point out that there may be situations in which high-quality evidence is not needed to recommend a particular healthcare intervention. For example, when inaction leads to dire consequences and there are no other treatment options, a healthcare intervention supported by moderate or low-quality evidence might be preferable to no action at all.

You Need To Be Your Health Care Advocate

You are probably wondering what this study means for you. Unfortunately, the authors of this study did not provide a list of healthcare interventions that were not supported by high-quality evidence. So, I can’t provide you with a list of interventions to avoid.

At one point, the authors of this study said, “Patients, doctors, and policy makers should consider the lack of high-quality evidence supporting the benefits and harms of many interventions in their decision-making.” However,

Doctors are very busy. They don’t have time to read and evaluate the quality of clinical studies. They rely on the clinical guidelines issued by policy makers (medical societies and government agencies).

Policy makers don’t like to admit they were wrong and are very slow to revise their clinical guidelines.

That means you must be the advocate for your health. I’m not suggesting that you question every recommendation your doctor makes. However, you should research major healthcare interventions and discuss the pros and cons with your physician.

Dr. Google can be wildly inaccurate, but it is a place to start. You can look up the side effects of drugs your doctor is recommending, downsides of medical treatments they are recommending, and/or other treatment options for your medical condition. I like to focus on reliable sites such as the Cleveland Clinic, Mayo Clinic, and the NIH. WebMD is often, but not always, a reliable source.

Ask other health professionals about alternative approaches and/or other doctors they may recommend for your condition. Ask friends who have had the same condition what medical interventions worked for them and if they have other doctors they recommend.

Then discuss some of these with your doctor. He or she should be willing to discuss the pros and cons of their recommendations and alternate approaches. If not, ask for a second opinion or consult other doctors.

Of course, you should always be open to the possibility that no other good options exist, and some intervention is essential. I’m just suggesting you evaluate your options fully and discuss them with your doctor before starting any major healthcare intervention.

The Bottom Line

A recent study evaluated quality of clinical studies supporting many common healthcare interventions (drugs and medical treatments). The study found that:

Only 5.6% of healthcare interventions studied were supported by high-quality evidence.

This study is not an outlier. Previous studies have come to similar conclusions.

The evidence of harms caused by healthcare interventions has not been adequately studied and could be as high as 22%.

As someone who taught the importance of evidence-based medicine to medical students for 40 years, I was appalled by this finding. And as patients trying to navigate the medical system, you should be appalled as well.

The authors of this study said, “Patients, doctors, and policy makers should consider the lack of high-quality evidence supporting the benefits and harms of many interventions in their decision-making.” However,

Doctors are very busy. They don’t have time to read and evaluate the quality of clinical studies. They rely on the clinical guidelines issued by policy makers (medical societies and government agencies).

Policy makers don’t like to admit they were wrong and are very slow to revise their clinical guidelines.

For more details on this study and my discussion of how you can research major healthcare interventions recommended by your doctor, read the article above.

These statements have not been evaluated by the Food and Drug Administration. This information is not intended to diagnose, treat, cure, or prevent any disease.

Do Statins Really Work?

Author: Dr. Stephen Chaney

Statins – those ubiquitous drugs used to lower cholesterol levels – are big business!

Over 20 million Americans are currently being treated with statin drugs at a cost that runs into billions of dollars every year. And cardiologists have just recommended that another 20 million Americans consider using cholesterol lowering drugs. 44% of the men and 22% of the women in this country are now being told that they should be using statin drugs.

Some of my cardiologist friends are so convinced that statin drugs prevent death from heart attacks that they have said, only half-joking, that we should just add statins to the water supply.

Are Cholesterol-Lowering Drugs Right For You?

Is the faith of doctors in the power of statin drugs to prevent death from heart disease justified? To answer that question in full we need to look at people who have already survived a heart attack and people who have never had a heart attack separately.

If you’ve already had a heart attack the evidence is clear cut.

If you have had a heart attack, there is good evidence that statins will reduce your risk of dying from a second heart attack.

In the technical jargon of the scientific world that is referred to as secondary prevention.

But what about those millions of Americans who are being prescribed statin drugs who have never had a heart attack? This is something we scientists refer to as primary prevention.

What Do The Studies Actually Say About Statins And Primary Prevention?

Here the evidence is not clear at all. Two major reports have cast doubt on the assumption that statins actually do prevent heart attacks in people who have not already had a first heart attack.

In the first study, Dr. Kausik Ray and colleagues from Cambridge University in England performed a meta-analyis of 11 clinical studies involving over 65,000 participants (Ray et al, Arch. Int. Med., 170: 1024-1031, 2010). They focused on those participants in the studies who had not previously had a heart attack (primary prevention).

They found that the use of statins over an average of 3.7 years had no statistically significant effect on mortality. In short, statins had no effect on the risk of dying from heart disease or any other cause.

Dr. Sreenivasa Sechasai, one of the doctors involved in the study, said “We didn’t find a significant reduction in death despite having such a huge sample size. This is the totality of evidence in primary prevention. So if we can’t show a reduction with this data, it is unlikely to be there.”

The second study was a Cochrane Systemic Review of statins published January 19th, 2011. It stated that there was not enough scientific evidence to recommend the use of statins in people with no previous history of heart disease with some caveats (see below).

To help you understand the significance of that conclusion, let me give you a bit of background:

First you need to understand that the Cochrane Collaboration is an independent, non-profit organization that carefully reviews the scientific evidence behind medical treatments and proposed medical treatments.

Cochrane Reviews are considered the “Holy Grail” of evidence-based medicine (ie. medicine based on the best scientific evidence rather than what the pharmaceutical companies would have you believe).

So when a Cochrane Review concludes that there isn’t enough evidence to recommend use of statins in patients with no prior history of heart disease that is pretty big news in the medical world.

How Should These Studies Be Interpreted?

Please don’t misinterpret what I am saying. The Cochrane Review said that statin drugs are overprescribed, but it did not say that everyone who has not had a heart attack will not benefit from statins. It said that there are a number of risk factors that need to be considered in evaluating individual patients for statin use.

Simply put, that means that it is not as simple as saying that everyone with no previous history of heart disease should not be on statin drugs.

If you are currently taking statin drugs and you have no previous history of heart disease, you may want to discuss with your physician whether the Cochrane Review of statin drugs changes their opinion of whether se of those drugs is still warranted for you.

But the bottom line is that only your physician is trained to take into account all of the factors that increase your risk of heart disease and the best therapeutic approach for reducing your risk of heart attack.

There Is A Double Standard In The Medical Community

More importantly, these studies highlight the difficulty in showing that anything works when you start out with a healthy group of adults with no prior evidence of disease (primary prevention).

And, the way that doctors have responded to primary prevention studies shows that there is a double standard in how primary prevention trials are interpreted in the medical community. For example:

There is no good evidence that statins prevent fatal heart attacks in healthy people.

However, because statins do work in high risk patients, most doctors recommend their use by millions of Americans who have never had a heart attack.

There is also no good evidence that nutrients like vitamin E and omega-3 fatty acids prevent fatal heart attacks in healthy people.

However, there is evidence that both vitamin E and omega-3 fatty acids prevent heart attacks in high risk patients, yet most doctors will tell you they are a waste of money.

It is food for thought.

The Bottom Line

1) Statin drugs clearly save lives when used by people who have already had a heart attack.

2) On the other hand, there is no proof that statin drugs prevent heart attacks in people who have not previously had a heart attack

3) Statin drugs do have side effects. Increased risk of diabetes, liver damage, muscle damage and kidney failure are the best documented, although memory loss has also been reported.

4) I am not recommending that you stop using statin drugs without consulting your doctor. I am suggesting that you discuss the benefits and risks of statin drug use with your doctor.

5) Perhaps the most important poin tto come out of these studies is that it almost impossible to prove the benefit of any intervention in a primary prevention trial. If you can’t prove that statins work in healthy people, it is not surprising that it is difficult to prove that other interventions work.

6) Finally, the way that these studies have been interpreted shows that there is a clear double standard in how the medical community evaluates primary intervention trials.

Statin drugs don’t show any benefit in a primary prevention setting, yet most doctors still recommend them.

Vitamin E and omega-3 fatty acids don’t show any benefit in a primary prevention setting, and most doctors recommend against them.

These statements have not been evaluated by the Food and Drug Administration. This information is not intended to diagnose, treat, cure or prevent any disease.