What Is Peer Review?
Author: Dr. Stephen Chaney
Recent headlines claim that long-term melatonin use may be risky. Specifically, the headlines claim that long term melatonin use may increase the risk of:
- Heart failure by 90%.
- Hospitalization for heart failure by almost 3.5-fold.
- All-cause mortality by 2-fold.
Those statistics are frightening. But as Mark Twain said, “There are lies. There are damn lies, and then there are statistics.”
So, as someone who uses a melatonin supplement regularly, the claims about heart failure risk made me want to check out the study behind the headlines.
I set out to answer two questions:
- Are the claims about melatonin use and heart failure risk true?
- Are they significant?
How Was The Study Done?
Here is what we know about the study:
- The authors used a global data network of healthcare providers called the TriNetX Global Research Network and selected all patient records in the network of adults >18 who had a clinical diagnosis of insomnia.
- They compared insomnia patients that had been prescribed melatonin for at least a year with non-melatonin users. Patients on heart failure drugs were excluded from the analysis.
- Outcomes (new diagnosis of heart failure, hospitalization for heart failure, and all-cause mortality) were assessed 5 years after the patients were first prescribed melatonin.
- A small subset of the two groups (414 in each group) were compared with respect to things like:
-
- Demographics (things like age, sex, socioeconomic status, and income level).
-
- Other diagnosed diseases.
-
- Other medication use.
-
- Healthcare utilization.
-
- Blood and urine lab results.
-
- Vitals (things like weight, height, blood pressure, heart rate, etc.
I said, “Here is what we know about the study” because this was not a published clinical study. It was a poster presented at a scientific meeting.
That means that a lot of details about design of the experiments and how the data were analyzed are missing. It also means the study has not gone through the peer review process that is required for publication in a scientific journal.
To help you understand why that is important, let me explain the peer review process.
What Is Peer Review?
Poster presentations at scientific meetings, like this one, have a couple of purposes:
- They give graduate students and post-doctorate fellows a chance to learn how to present and defend their research. It also gives them a chance to form relationships with the leaders in their field who may become their next step up the career ladder.
- They foster discussions with other experts in the field who may spot flaws or offer helpful suggestions for improving the research before submitting it for publication.
The ultimate goal, of course, is to get the research to the point where it is ready to be submitted to a high-quality, peer-reviewed scientific journal (It is still “publish or perish” in the academic world).
This is an area in which I have extensive experience, having published over 100 articles in peer reviewed journals during my career at the University of North Carolina. I was also a reviewer for hundreds of articles during my time at UNC.
Once the manuscript has been submitted to a peer-reviewed scientific journal, they send it to two or three peers (other experts in the same area of research). Some of the peers are your friends. Others are your competitors. In either case, their goal is to make sure the paper adheres to the highest scientific standards. They review the paper for:
- Hypothesis to be tested: They ask, “Is the hypothesis credible and clearly stated?”
- Experimental Design: They ask, “Are the experiments designed in such a way that they adequately test the hypothesis?”
- Data Collection: They ask, “Was enough data gathered to test the hypothesis?” and “Were there gaps in the data that need to be filled?”
- Statistical Analysis: They ask, “Was the statistical analysis of the data sufficient to test the hypothesis?” and “Were there any important controls missing from the data collected or the statistical analysis?”
- Conclusions: They ask, “Are the conclusions of the authors adequately supported by the data collected and the statistical analysis?”
If the reviewers find defects in any of these areas, they have 3 options. They can:
- Ask the authors to rewrite and resubmit the manuscript.
- Ask the authors to perform additional experiments or use a more rigorous statistical method before resubmitting the manuscript.
- Reject the manuscript.
Hopefully, this description helps you understand the power of the peer review process and why the lack of peer review for this study is significant.
I am not saying the claims in this poster presentation are inaccurate. I’m saying the headlines about them may be premature because the study has yet to be peer-reviewed.
What Did This Study Show?
The abstract said that long-term (≥1 year) melatonin users had:
- An 89% higher risk of developing heart failure.
- A 3.44-fold higher risk of hospitalization for heart failure.
- A 2.09-fold higher risk of all-cause mortality.
The authors concluded, “In a large, multinational real-world cohort rigorously matched on >40 baseline variables, long-term melatonin supplementation in insomnia was associated with an 89% higher hazard of incident heart failure, a three-fold increase in heart failure–related hospitalizations, and a doubling of all-cause mortality over 5 years. These findings challenge the perception of melatonin as a benign chronic therapy and underscore the need for randomized trials to clarify its cardiovascular safety profile.”
Is Melatonin Safe?
At the beginning of this article I said I wanted to answer two questions:
- Are the claims about melatonin use and heart failure risk true?
- Are they significant?
The first question (Are the claims about melatonin use and heart failure risk true?) is difficult to answer.
- This is an abstract instead of a publication. That means I have no access to the data, statistical methods, and controls that went into the Author’s conclusions. So, I have no basis for analyzing the accuracy of the authors’ statements.
- This has not gone through peer review. The conclusions of the authors may be modified or may be rejected based on peer review.
There are several questions I and others who have discussed this study would like to see answered. For example:
- How can hospitalizations for heart failure be greater than the number of people with heart failure? Normally, between 20-80% (average = 45%) of people with congestive heart failure end up in the hospital. The discrepancy between hospitalizations and heart failure patients in this study suggests that:
-
- Some hospitalizations were for another reason and were mischaracterized as being due to heart failure…or…
-
- The same individual(s) were hospitalized multiple times during the 5-year follow-up.
-
- In either case hospitalizations for heart failure would be overcounted.
- How did they accurately distinguish between melatonin users and non-users since their study drew on data from both England and the United States?
-
- In England, melatonin is only available by prescription, so the distinction is clear.
-
- But in the United States a prescription is not required. Melatonin is infrequently prescribed. Most melatonin users obtain their melatonin over the counter and are not recorded in any database.
- Were the controls sufficient?
-
- The author’s conclusion states the data were “…matched on >40 baseline variables…”, but the Methods section states that only a small subset (414) of melatonin users were matched for those variables. That is less than 5% of the melatonin users in the study.
- Are the data on the relationship between melatonin use and all-cause mortality accurate?
-
- Again, the increase in all-cause mortality was greater than the number of melatonin users.
-
- That suggests that the increase in all-cause mortality may be due to some characteristics of severe insomniacs who choose to take melatonin rather than to melatonin itself.
These are the kinds of questions that need to be answered during the peer review process and might alter the conclusions or prevent the study from being published.
- So, the answer to the first question (Are the claims about melatonin use and heart failure risk true?) is, “We don’t know. The study has not gone through the peer review process and is unpublished.
Relative Risk Versus Absolute Risk
The answer to the second question (Are the claims about melatonin use and heart failure risk significant) is much clearer. It has to do with the difference between relative risk and absolute risk.
Simply put:
- Relative risk is the change in risk caused by an intervention (melatonin in this case) relative to the risk without the intervention.
-
- Relative risk is generally used in describing the results of clinical studies because it gives much larger numbers and is, therefore, more newsworthy. It is more likely to generate the headlines you see and grab your attention.
- Absolute risk is the actual change in risk that you experience.
-
- This is the risk that matters to you.
With those simple definitions, let’s review the claims of this study:
- The study said that long-term melatonin use increased the risk of heart failure by 90%. That sounds horrible, but it is an increase in relative risk!
-
- To calculate the absolute increase in the risk of heart failure, you need to first ask what the baseline risk is in non-melatonin users. In this study it was 2.7%.
-
- With that information, we can do some simple math. A 90% increase would increase the risk to 4.6%. 4.6% – 2.7% = 1.9%.
-
- That means the absolute increase in risk caused by long-term melatonin use is less than 2%.
- The study said that long-term melatonin use increases the risk of hospitalization 3.44-fold.
-
- On average, 45% of heart failure patients are hospitalized. Since the risk of heart failure in this study was 2.7%, that means the baseline for hospitalization should be 45% of 2.7% or around 1.2%.
-
- A 3.44-fold increase would increase that to 4.2% which means that the absolute risk of being hospitalized if you are a heart failure patient is around 3% (4.2% – 1,2%).
- Finally, if the increase in all-cause mortality is solely caused by melatonin use (which is unclear), melatonin use increases mortality risk from 4.3% to 7.8%, for an absolute increase in risk of 3.5%.
In short, if the claims about melatonin use and heart failure are true (which is uncertain) the absolute increase in risk is very small for the average, healthy adult.
What Does This Study Mean For You?
If you are an average, healthy adult you probably do not need to be concerned about the headlines linking melatonin use with heart failure.
- It is not yet clear whether the claims are accurate.
- And, if they are accurate, your increased risk is very small.
There are, however, some caveats.
- If you suffer from severe, chronic insomnia you should consult your healthcare provider first.
-
- Your insomnia may be caused by an underlying health condition.
-
- Your healthcare provider may suggest more effective treatments. Some of these treatments may have more side effects than melatonin use, but it always useful to know what your choices are.
- If you have been diagnosed with heart failure, you should avoid melatonin.
Finally, even though this study is very preliminary, there are also some individuals who may wish to consult with their healthcare provider before taking melatonin. These include people with conditions that increase the risk of heart failure such as coronary heart disease, heart attacks, diabetes, obesity, and high blood pressure.
The Bottom Line
Recent headlines have suggested that long-term melatonin use substantially increases the risk of heart failure, hospitalizations due to heart failure, and premature death.
However, these headlines were based on an abstract of a poster presented at a scientific meeting. That means:
- There is not adequate information available to analyze the data behind the claims.
- The study has not yet gone through the peer review process, so we don’t know whether the claims are accurate.
Based on my evaluation of the available information, my analysis is:
If you are an average, healthy adult you probably do not need to be concerned about the headlines linking melatonin use with heart failure.
- It is not yet clear whether the claims are accurate.
- And, if they are accurate, your increased risk is actually very small.
For more information on the study and who should consult with their healthcare provider before using melatonin, read the article above.
These statements have not been evaluated by the Food and Drug Administration. This information is not intended to diagnose, treat, cure or prevent any disease.
_____________________________________________________________________________
My posts and “Health Tips From the Professor” articles carefully avoid claims about any brand of supplement or manufacturer of supplements. However, I am often asked by representatives of supplement companies if they can share them with their customers.
My answer is, “Yes, as long as you share only the article without any additions or alterations. In particular, you should avoid adding any mention of your company or your company’s products. If you were to do that, you could be making what the FTC and FDA consider a “misleading health claim” that could result in legal action against you and the company you represent.
For more detail about FTC regulations for health claims, see this link.
https://www.ftc.gov/business-guidance/resources/health-products-compliance-guidance
________________________________________________________________________
About The Author
Dr. Chaney has a BS in Chemistry from Duke University and a PhD in Biochemistry from UCLA. He is Professor Emeritus from the University of North Carolina where he taught biochemistry and nutrition to medical and dental students for 40 years.
Dr. Chaney won numerous teaching awards at UNC, including the Academy of Educators “Excellence in Teaching Lifetime Achievement Award”.
Dr Chaney also ran an active cancer research program at UNC and published over 100 scientific articles and reviews in peer-reviewed scientific journals. In addition, he authored two chapters on nutrition in one of the leading biochemistry text books for medical students.
Since retiring from the University of North Carolina, he has been writing a weekly health blog called “Health Tips From the Professor”. He has also written two best-selling books, “Slaying the Food Myths” and “Slaying the Supplement Myths”. And most recently he has created an online lifestyle change course, “Create Your Personal Health Zone”. For more information visit https://chaneyhealth.com.
For the past 54 years Dr. Chaney and his wife Suzanne have been helping people improve their health holistically through a combination of good diet, exercise, weight control and appropriate supplementation.


























