Performance measures fail

----time and time again. That statement, which I've been making and backing up with evidence for years in these pages, is still at odds with popular belief. Just to make sure I'm not misunderstood I want to draw some distinctions. What I am criticizing is performance. I'm not talking about quality, evidence based medicine or guideline adherence. Performance is to be distinguished from all those.

Like guidelines, the processes targeted by performance metrics are, for the most part, supported by evidence. But that's where the similarity stops. Guidelines offer perspective and nuance on how to apply the best evidence. Performance takes an evidence based care process, isolates it from its appropriate clinical context and turns it into a game. The ensuing unintended consequences rule the day and the measures fail.

If there's one measure that's inherently more robust that the others it's got to be door-to-balloon time for STEMI. The evidence and physiologic rationale are so strong, how could it miss? But it does. The mortality rate for acute coronary syndrome has been declining for while. The advent of door-to-balloon performance did not impact the rate of decline. A new paper just out in NEJM takes a closer look. It starts out encouraging:

Methods

We analyzed annual trends in door-to-balloon times and in-hospital mortality using data from 96,738 admissions for patients undergoing primary PCI for ST-segment elevation myocardial infarction from July 2005 through June 2009 at 515 hospitals participating in the CathPCI Registry. In a subgroup analysis using a linked Medicare data set, we assessed 30-day mortality.

Results

Median door-to-balloon times declined significantly, from 83 minutes in the 12 months from July 2005 through June 2006 to 67 minutes in the 12 months from July 2008 through June 2009 (P less than 0.001). Similarly, the percentage of patients for whom the door-to-balloon time was 90 minutes or less increased from 59.7% in the first year to 83.1% in the last year (P less than 0.001).

So far so good, but there's more:

Despite improvements in door-to-balloon times, there was no significant overall change in unadjusted in-hospital mortality (4.8% in 2005–2006 and 4.7% in 2008–2009, P=0.43 for trend) or in risk-adjusted in-hospital mortality (5.0% in 2005–2006 and 4.7% in 2008–2009, P=0.34), nor was a significant difference observed in unadjusted 30-day mortality (P=0.64).

So this, arguably the strongest of all the core measures, fails as have so many of the others. A related article in Medpage Today offers speculation. Maybe some essentials of care are being overlooked as patients are rushed to the cath lab under pressure to “satisfy the measure.”

All would agree that time is muscle and that the shorter the time to reperfusion the better. Minimizing door-to-balloon time is a great endeavor. But, like so many other evidence based modalities, toxicity occurs when it becomes a performance metric.