EBM's blind spot

The Cochrane Collaboration, arguably the exemplar of EBM, is a good place to illustrate EBM's blind spot, a phrase used by Kimball Atwood at Science Based Medicine in his description of EBM's inability to evaluate implausible health claims in the arena of complementary and alternative medicine (CAM).


Here's the listing of Cochrane CAM reviews. Few if any of the treatments covered in these reviews have enough scientific plausibility or prior evidence to warrant a share of our research funds, let alone a systematic review. But the Cochrane reviewers refuse to call a nutty idea a nutty idea. Submit the wackiest fairy dust treatment you can think of to the folks at Cochrane for review. The worst they're liable to say about it is that there's not enough evidence out there to know whether or not it's a valid treatment, so research (or more research) is warranted.


So here are a few examples.


Homeopathic treatment for influenza. Promising, say the reviewers, although more research needed.


Homeopathic treatment for asthma. RCTs needed, of course.


Acupuncture to enhance in vitro fertilization. It works, declare the Cochrane reviewers. But, they say, more research is needed.


Acupuncture for acute stroke (and the mechanism doesn't seem to matter to the reviewers, evidenced by the fact that they lumped hemorrhagic and ischemic strokes, conditions with entirely different mechanisms, together). The reviewers are not sure whether it works and recommend larger and more “methodologically sound” studies.


Acupuncture for vascular dementia. RCTs are urgently needed, say the reviewers!


EBM has become an enabler of pseudoscience, and that's its grand failure. Atwood explains why in the above link and multiple other posts linked there, and I summarize the reasons briefly here. That's the fatal flaw Richard Smith and DB didn't mention in their otherwise informative and accurate criticisms of EBM.