I have written previously on this blog about our research highlighting the extremely low quality of published “systematic reviews” and “meta-analyses” in the otolaryngology and sleep surgery literature. We have shown that many studies do not adhere to established, accepted research methods for these studies, creating a dangerous situation where publications that pretend to be authoritative and certain in their conclusions in fact do not reflect the fact that they are based on publications that may be biased or have other fundamental issues. In short, I am sorry to say that I do not trust any systematic reviews for their conclusions and simply treat them as a resource to identify relevant articles on a topic that I then review myself. These are trends that are seen in every field of medicine, and numerous similar published studies in other fields have confirmed this.

A more recent study from a team at the Naval Medical Center in San Diego, California, has suggested that we in otolaryngology – head and neck surgery may be doing more poorly than other fields – and getting worse – when it comes to the broader array of research studies. They examined a large group of papers from the most-cited otolaryngology journals in 2010 and 2020 and evaluated multiple measures of research quality. They showed:

• 2020 papers had lower evidence quality than 2010
• 2020 papers had more reporting of confidence intervals, a basic method of presenting statistical results , although this was still only seen in 40% of all papers in 2020

Importantly, the literature in other fields (neurosurgery, orthopedic surgery, ophthalmology, and general surgery) had higher evidence quality in 2020 compared to otolaryngology, although they did have less reporting of confidence intervals.

Clearly, the medical literature could be of higher quality. Studies like this evaluate important limitations of the literature and areas for improvement. One wonders whether the proliferation of medical journals dilutes the overall quality of the literature, as, like material found on the Internet, all that glitters (published in medical journals) is not gold. I am not sure what the solution is, but I have enjoyed our multi-center research collaborations related to drug-induced sleep endoscopy and believe they have really advanced the field of sleep surgery.

I welcome the thoughts of others, as I do know that we can and must do better.