The latest woo from BMJ: acupuncture as an adjuvant for in vitro fertilization

Wallace Sampson, in a recent Science Based Medicine blog post, discusses “how to establish ineffectiveness in presence of conflicting information without submitting every nutty idea to infinite numbers of trials.” That, of course, is the problem with the National Center for Complementary and Alternative Medicine (NCCAM) which so far has spent about a billion dollars on such ideas. Its promotion of nutty ideas is often aided and abetted by the British Medical Journal (BMJ), earning it a place on Quackwatch’s list of nonrecommended periodicals under the category of “Journals, Excellent Except for Too Many Poorly Reasoned Articles on ‘Complementary’ and/or ‘Alternative" Medicine.’”

A systematic review and meta-analysis on acupuncture for in vitro fertilization (IVF) in this week’s BMJ suggests that the journal won’t be delisted any time soon. The review, which purported to demonstrate a favorable effect, included studies comparing acupuncture with sham acupuncture or no adjuvant treatment.

A commentary in the same issue concluded with:

So is this review by Manheimer and colleagues a well conducted review, worthy of consideration when making decisions about IVF? Yes. Is it perfect? No. However, several thousand systematic reviews are published each year in health care, and none of them is likely to be perfect. This one seems as good as many. Unless, of course, you know differently?

Well, yes, we know, or should know, differently. Questions abound. How, for example, can sham acupuncture points be defined when there’s no historic agreement or anatomic basis for the “real” ones? What conclusions would Bayesian analysis reach?

How would such an analysis be done? Sampson in his post suggests quantitative methods, but they are mostly applicable to individual trials. Bayes’ theorem evaluates data in light of prior knowledge and probability. BMJ makes only a nominal gesture at such an analysis:

What is already known on this topic:

In vitro fertilisation is lengthy, expensive, and stressful.

Safe, low cost, adjuvant treatments to improve success rates would benefit patients and reduce costs.


What this study adds:

Current evidence from methodologically sound trials showed an odds ratio of more than 1.6 for clinical pregnancy after in vitro fertilisation with adjuvant acupuncture.

On average, 10 women would need to be treated with acupuncture to bring about one additional clinical pregnancy. The magnitude of this effect depended on the baseline pregnancy rate.

The “analysis” is totally uncritical but is telling in that it implies no prior knowledge on the subject. There’s certainly no rationale on the basis of scientific principles.

I won’t attempt to do all the math, but here’s my semi-quantitative analysis.

P(AIB) = [P(BA) x P(A)] / P(B). P(AIB) (the probability of A given B) is the “bottom line” assessment after taking into account prior knowledge and the new evidence. P(A) is the “prior probability”, or the probability that the effect of acupuncture is true given prior knowledge and scientific plausibility. Since P(A) has to be infinitesimally small and occupies the numerator of the equation, the probability that acupuncture has a real effect on the success of IVF would have to be very small.