Annals of Internal Medicine promotes acupuncture for comparative effectiveness research

Often in my professional development I am taught to consider the source. Information presented by pharmaceutical companies is to be rejected. Blogs are to be read with an extra measure of caution. A journal's reputation means something. Not so much anymore. Now we have to evaluate medical claims on their own merits regardless of the source. That demands a little more effort. Why do I say this? A case in point is the trend over the last few years in so called top tier medical journals toward the promotion of quackery. Just recently the Annals of Internal Medicine published this study on acupuncture for treatment of seasonal allergic rhinitis. The paper is cosmetically scientific with all the jargon and trappings of evidence based medicine:

Results: Compared with sham acupuncture and with RM, acupuncture was associated with improvement in RQLQ score (sham vs. acupuncture mean difference, 0.5 point [97.5% CI, 0.2 to 0.8 point; P less than 0.001]; RM vs. acupuncture mean difference, 0.7 point [97.5% CI, 0.4 to 1.0 point; P less than 0.001]) and RMS (sham vs. acupuncture mean difference, 1.1 points [97.5% CI, 0.4 to 1.9 points; P less than 0.001]; RM vs. acupuncture mean difference, 1.5 points [97.5% CI, 0.8 to 2.2 points; P less than 0.001]). There were no differences after 16 weeks in the first year. After the 8-week follow-up phase in the second year, small improvements favoring real acupuncture over the sham procedure were noted (RQLQ mean difference, 0.3 point [95% CI, 0.03 to 0.6 point; P = 0.032]; RMS mean difference, 1.0 point [95% CI, 0.2 to 1.9 points; P = 0.018]).

Limitation: The study was not powered to detect rare adverse events, and the RQLQ and RMS values were low at baseline.

Conclusion: Acupuncture led to statistically significant improvements in disease-specific quality of life and antihistamine use measures after 8 weeks of treatment compared with sham acupuncture and with RM alone, but the improvements may not be clinically significant.

Initially I just glanced casually at the study. This type of thing is not so unusual anymore. It was just another promotion of a nutty idea by a mainstream medical journal under the guise of evidence based medicine. It was so dense with scientific verbiage and so circumspect sounding I knew it would take a little effort to expose it. And at the time I was busy. And tired.

Well, fortunately Mark Crislip over at Science Based Medicine did it for me. When you read his post I suggest you skip down to the middle where he begins the actual analysis of the paper. The real devil is in the details and he unpacks them nicely:

The big difference is that acupuncture is a form of magic that is likely to get published in formerly reliable journals complete with positive spin: Acupuncture in Patients With Seasonal Allergic Rhinitis. A Randomized Trial...

A careful and critical reading of the paper gives compelling evidence that acupuncture does nothing, in line with all the other well-designed acupuncture studies. The cognitive disconnect between the editorial and conclusion and the data presented in the paper is a wonder to behold. And most people will read only the spin in the abstract and the editorial, not noticing the fatal flaws, and erroneously conclude that acupuncture has efficacy.

Yet another reason why I am glad I let my Annals subscription lapse years ago.

It's all quite ridiculous. One of the more unintentionally funny papers in the medical literature. Or was it intentional? It's hard to know from just reading the paper whether or not it's a parody. But apparently it was intended to be taken seriously because in an accompanying editorial the journal elevates it to the realm of public policy, specifically “comparative effectiveness research.”

This type of nonsense has been going on in high impact medical journals for several years now and it is getting worse. It started out in BMJ about a decade ago and over time has metastasized to other journals. Over at Quackwatch Steve Barrett put BMJ on his black list years ago and subsequently added the Annals.

But the Annals and BMJ have managed to maintain a good reputation, so now a journal's reputation means little. Whether it's indexed in Medline means little. The same can be said of broad categories such as journals and blogs. It's strictly buyer beware. I've said for a long time that relying on the source to evaluate the quality of information is intellectually lazy. Now, as illustrated by the paper and editorial in question, it's flat out unreliable.

There's not much being done to stop the trend but as time and energy permit I'll do what I can to expose it. In the meantime maybe there's a positive here. It means, more than ever before, that we have to evaluate medical information on its own merits rather than considering too strongly the source. Something we should have been doing all along.