Should quality and safety measures be held to evidence based standards?

From time to time I and a few other bloggers have criticized the quality and safety enthusiasts for pushing measures that are not supported by evidence. Some instances involve truly evidence based therapies not shown to be effective when promulgated as ill conceived “core measures.” Most of the debates swirling around the field have appealed to evidence. At least until now.

In the latest issue of JAMA Donald M. Berwick, a leader at the Institute for Healthcare Improvement, writes that the prevailing standards of clinical evidence are too rigid for the evaluation of safety and quality practices. He cites what he believes to be the inadequacy of the randomized controlled trial and “injudicious assaults on bias.”

Although Berwick’s language is oblique to me he seems to be calling for less scientifically rigorous ways of evaluating quality and safety measures. Bob Wachter blogged about this last Saturday and described Berwick’s position more plainly:

He argues that traditional rules of evidence-based medicine should be suspended or relaxed when it comes to safety and quality practices, since the risks and costs of the practices are generally low (at least compared with drugs or devices), rigorous studies are difficult to do (since they deal with such complex settings), and the “evidence” from experience and theory is often sufficiently compelling to merit adoption.

Wow. It kind of reminds me of Andrew Weil arguing for a relaxed evidentiary standard for complementary and alternative medicine. Now don’t get me wrong. I’m not a stickler for evidence when it comes to simple common sense safety practices. I don’t need a systematic review to tell me I should wash my hands between patient encounters. But I do demand evidence when the measure in question is expensive (such as the IHI promoted universal screening measure for MRSA, debunked in the same JAMA issue) or mandated as a core quality measure without due regard for potential adverse consequences (four hour antibiotic rule, intensive glycemic control, the fifth vital sign---I could go on). I’m also concerned that our non-evidence based zeal for a zero defect health care environment unrealistically inflates public perceptions which could drive intrusive legislation and law suits. Wachter said it well:

On the other hand, in our zeal to “do something,” vigorously promoting or mandating practices with weak evidence risks squandering scarce resources, diverts us from better strategies, and subjects the safety field to the whims of opinions and biases. Berwick worries that our EBM pushback gives intellectual ammo to the dark forces of status quo. This is a reasonable concern. But given the public interest in quality and patient safety, I worry more that the distance between “this seems like a good idea” to “let’s include it as part of a campaign” to “let’s make it a new Joint Commission standard” to “let’s make it a state law” is perilously short. Accordingly, we should require awfully strong evidence that we’re doing the correct thing as we traverse that path, particularly when practices are complex and expensive.