Has EBM been hijacked? Yes but not in the way John Ioannidis says it has.

Dr. David Gorski is a surgical oncologist who writes for Science Based Medicine.  I am an enthusiastic follower of his posts there and on his other blog.  I also have a passionate interest in evidence based medicine (EBM).  So when Dr. Gorski posted recently about the hijacking of EBM my interest was piqued.  Though after a careful read of this long post and some of its many links I feel I mostly agree with him, there are some points there that concern me.  Before addressing them I should provide some background.  John Ioannidis, whom Gorski cites heavily and was the inspiration for the post, is famous for his 2005 paper titled Why Most Published Research Findings Are False.  More recently he wrote the article Evidence-based medicine has been hijacked: a report to David Sackett.  In that paper, which is in the form of an open letter to one of the founders of EBM David Scakett (now deceased),  Ioannidis opines that there are widespread problems with  the medical research agenda consisting of corruption of clinical trials and investigators asking the wrong questions due to various conflicting interests.

Dr. Gorski in response is mainly favorable to the article but is concerned about a missing piece:

In his “report” to David Sackett, Ioannidis does touch on a number of pertinent and interesting points regarding the adoption of EBM but, as you will see, pretty much ignores the one huge elephant in the room.

The elephant is in reference to a popular distortion of EBM that has the effect of, well, I know of no better way to put it, enabling quackery.  The distortion in question is a tendency to devalue basic science when considering various forms of evidence.  There's a long story as to how it came about but suffice it to say here it was not the original intent of EBM's founders.  While that is the main point of Dr. Gorski's post there is much more, including some points that concern me, (and another elephant he made only indirect reference to) which I will address below.

As for Ioannidis's statement that most published research findings are false, to me it just added shock value to something we knew for decades but discussed in less loaded terms.  While he did unpack some of the reasons in a way that had not been done before we always knew that research findings are tentative and that modification of prior research by new research is the usual case.    This is something that has been acknowledged and accepted in medicine for a long time.  (I considered aspects of this phenomenon at some length in a post on medical reversal).  This is not to say we shouldn't be concerned about the quality of research.

At the risk of sounding like a stickler for correctness of terminology, while Ioannidis describes what might be called a hijacking of the research agenda it is not a hijacking of EBM.  EBM, at least as it was originally defined, is focused on how the individual clinician uses expertise to integrate the best available evidence with the needs of the individual patient at the point of care, not the research agenda.  The design and implementation of clinical trials is something separate from EBM.  Sackett himself in effect acknowledged this when in 2000 he retired from the field of EBM and migrated to the other field of clinical trials:

Dr Sackett eventually returned to Canada and, leaving the EBM field for others, devoted himself to researching and writing about randomized clinical trials in a wooden cabin on Irish Lake in Ontario. There he canoed and snowshoed with family and friends.

So while while Ioannidis points to some very important concerns what he describes is not the hijacking of EBM.  However, as Gorski points out EBM has indeed been hijacked.  I would take the discussion a step beyond what Gorski said which brings me to the other elephant in the room.  Take a look at this from his post, in reference to the discussion by Ioannidis about the profession's and industry's resistance to AHRQ:

So, yes, there is resistance to the AHRQ. However, these days it is far more business interests, such as drug and device manufacturers, than physicians groups who want to abolish both the AHRQ and the PCORI, mainly because the AHRQ and PCORI’s research threatens these companies’ bottom lines by showing which treatments work better in the “real world” and influencing the Centers for Medicare & Medicaid Services (CMS) regarding which new drugs and devices will be paid for. In fact, I’d argue that, while Ioannidis is correct that drug and device manufacturers want to kill AHRQ and PCORI, he’s missed a sea change in attitude among physicians towards such government agencies whose purpose is to evaluate and compare treatments for effectiveness after they’ve been approved. It might have been true that EBM was not popular 15 or 20 years ago, but as new generations of medical students have been inculcated with its principles and importance, EBM has been “baked in” to physician education, with a resultant change in attitude towards efforts to promote EBM. That’s not to say that physician groups don’t protect their turf. Just look at how radiologists, for example, react to new guidelines that increase the recommended age at which to start mammography or how primary care physicians react to legislation expanding the scope of practice of advanced practice nurses. However, extreme hostility to comparative effectiveness research and EBM-based guidelines has mostly retreated to fringe physician groups like the American Association of Physicians and Surgeons. Unfortunately, resistance to EBM as a constraint on physician autonomy is still fairly common, particularly among older physicians.

It gets tricky here because this paragraph is a little confusing, packing a lot in a small space and making what seem to me to be questionable assumptions.  The last sentence of the paragraph above has an embedded link to one of Gorski's old posts correctly pointing out that the notion of EBM as something that threatens physician autonomy is a straw man.  However, by equating attitudes towards government agencies with attitudes towards EBM he seems to imply that EBM promotes a top down approach to medicine.  (It's not clear to me whether that is his intended meaning).  EBM, in fact, seeks quite the opposite.  In a recent post on this very subject I quoted Sackett and some of the other founders, from one of their early articles in BMJ:

 Here's what some members of the EBM working group had to say in their seminal article in BMJ some years ago:

    Evidence based medicine is not 'cookbook' medicine...External clinical evidence can inform, but can never replace, individual clinical expertise…

    Clinicians who fear top down cookbooks will find the advocates of evidence based medicine joining them at the barricades.

The paper I quoted from is here.   In that paper we see the founders of EBM willing to fight for individual clinical judgment.  Who knew?

But getting back to another of Dr. Gorski's points, have physicians become more accepting of top down medicine in the last 10-15 years?  He seems to think they have but I don't know where he gets that.  This question isn't well informed by data but if anything the recent widespread physician outrage about ABIM, the ABIM foundation and its Choosing Wisely campaign would suggest otherwise.  (That is nicely chronicled at the blog of Westby G. Fisher, MD, FACC where most of the posts from the past two years are devoted to the topic). 

The agenda of top down medicine in its various forms as substitutes for EBM is the other elephant in the room:  EBM is being hijacked by the proponents of top down medicine.  Although folks in public policy circles don't use the term EBM very often it seems to be a widespread assumption in many of the policy discussions that these initiatives will make, even force, doctors to be more evidence based.  As policy wonk and futurist Bob Wachter once said in a discussion on top down initiatives that would stem from comparative effectiveness research:

We simply must find ways to drive the system to produce the highest quality, safest care at the lowest cost, and we need to drag the self-interested laggards along, kicking and screaming if need be.

The top down agenda is moving forward but fortunately at a slow creep and is no where near in place at the level Wachter wishes it to be.  Many of the new payment models under Obamacare (eg the ACO) are considered pilot projects and are not yet mandatory and may never be.  The AHRQ never grew the teeth that policy makers hoped it would have back when it was AHCPR.  Not all of the top down initiatives come from big government.  Many of the care pathways and performance measures offered as someone's version of EBM are locally driven.  I can only hope that doctors and medical educators will read and re-read that seminal 1996 BMJ paper and help cultivate and spread a true understanding of what EBM really is.