Dismantling evidence based medicine

I’m finding it harder and harder to defend evidence based medicine (EBM) these days. My usual defense is to explain away all the things that are wrong as distortions of EBM rather than EBM itself. And while this defense seems valid, the term EBM has become so synonymous with those distortions that maybe we need a new name. How about Science Based Medicine in recognition of the new blog of the same name? In this post I will present a roundup of blog entries and articles which express important concerns about EBM as it is promoted today.

EBM as originally conceived was a sound notion. What the EBM movement has become is problematic. Sackett and colleagues defined EBM in a classic editorial years ago:

Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.

There’s nothing wrong with that basic idea. The key word is judicious. But somewhere in the history of the movement the boosters issued some injudicious rules about how doctors should use evidence. The result was a devaluation of basic science, prior knowledge and pathophysiologic rationale. Plausibility was out the window. Preposterous claims were legitimized as questions for “research.” When chance variation conspired with publication bias, biased Medline indexing and conflicts of interest numerous forms of quackery (often euphemistically termed “complementary and alternative medicine” or CAM) appeared validated. While the evidence was weak it was enough to stimulate the infusion of millions of tax dollars into dubious research. Although unanticipated by EBM’s founders the movement helped fuel a “CAMbrian explosion.”

I began to realize this early in my blogging career when I wondered why we should even bother with research on homeopathy. I later elaborated on the importance of plausibility in a post citing Steve Barrett’s blistering attack on the Institute of Medicine’s pro-CAM report and Wallace Sampson’s classic paper documenting the promotion and advocacy of quackery in American medical schools. Sampson recognized early on the inability of EBM (or what EBM had become) to critically assess the claims of quackery and said in his paper:

With inadequate approaches that fail to uphold criteria for validity and plausibility, so called ‘evidence-based’ medicine remains fluid and loses its value to help physicians discern what is truly useful.

Several years later, in a paper published in The Medical Journal of Australia, Sampson and coauthor Kimball Atwood IV said this about EBM’s inability to evaluate quacky health claims (emphasis mine):

Evidence-based medicine (EBM), relying on results of randomised trials, should be a bulwark against the Absurd. However, the heterogeneity of clinical trial methods and designs, differing population bases, and varying endpoints often result in heterogeneity of outcomes. This has precluded systematic reviews of CAM methods from defining a line of inefficacy. EBM also does not include plausibility or consistency with basic science in its methods and reviews, leaving each to physician and patient interpretation. Moreover, there are no solid criteria for evaluating the quality of trials and reviews, especially for detecting erroneous, manipulated, and faked data. Thus, most CAM systems remain in an indeterminate limbo state, awaiting enough negative clinical trials to return consensus opinion to the state of decades prior.

In a recent Medscape Roundtable Discussion, while defending the core notions of EBM I criticized its de-emphasis on basic science and noted some of its other failings.

More recently, in a series of three posts, (the first two of which I discussed here) Atwood provided a more formal and quantitative discussion of the failings of EBM by using Bayesian analysis (warning---not light reading).

On Friday Retired Doc posted an insightful discussion on the consequences of EBM’s dogmatic evidence hierarchy. He quoted this paper by M.R. Tonelli which made the point that EBM has wrongly defined pathophysiologic rationale as “evidence”, then relegated it to the bottom of the hierarchy. That makes it possible for implausible CAM claims to be “validated” when, with the help of chance variation and publication bias, they happen to squeak by and pass the evidentiary test. The evidence, no matter how weak, always trumps basic science rationale even when the claim is clearly (or should be) refuted by the latter.