Thursday, April 09, 2009

Leapfrog’s performance scores are meaningless

According to a recently published JAMA analysis:

On the Internet, hospital performance on the Safe Practices Survey is ranked by quartiles, which may suggest to consumers that hospitals in higher quartiles are safer than hospitals in lower quartiles. In this first study of the relationship between survey scores and hospital outcomes, we studied a national sample of hospitals and found no relationship between quartiles of score and in-hospital mortality, regardless of whether we adjusted for expected mortality risk and certain hospital characteristics.

The measures surveyed included, but were not limited to: (1) creating a safety culture, (2) ensuring an adequate nursing workforce, (3) ensuring that a pharmacist is active in medication use, (4) not providing patient care summaries from memory, (5) providing patient care information and orders to all clinicians, (6) requiring patient readback of informed consent, (7) documenting resuscitation or end-of-life directives, (8) preventing mislabeled radiographs, (9) providing risk assessment and prevention for deep vein thrombosis/venous thromboembolism, (10) providing anticoagulation services, (11) preventing aspiration, (12) preventing central venous line sepsis, and (13) requiring hand washing.

Public reporting has hospitals scrambling to do all sorts of things to buff their profiles. Unfortunately these “improvements” are largely cosmetic and have little beneficial impact on patient outcomes. That’s just one reason the performance movement is a failure.

1 comment:

David Block said...

Too bad you choose the right word: "buff." Scores PROVE that you're doing the right thing, and so it's not your fault if everything goes to hell. The notion of a "score" assumes that you can identify a process and that this process satisfies certain conditions of explicit analyticity. E.g., we know what we're talking about, we know where we started out, we know where we're going, everything we don't measure is an "externality" and therefore somebody else's problem. This Taylor system worked great for Taylor (he didn't have to take a factory job, for example) but everybody else has to live with the consequences.

My question is about the "buffing" in this context. Why would a CEO or a CFO prefer buffing - which, after all, is no more than padding a bra or stuffing your jockey's before a date - to the real thing? To sitting down across a table from the very doctors who saved their grandchild's life the night before, and saying something like, "Dave, God damn it, what the hell is really going on around here? And what do we need to do - together?"

David Block MD