Sunday, June 30, 2013

Is performance a good surrogate for quality?

No, but a casual glance at this paper might give that impression. It deserves a more careful read. From the article:
Importance Federal efforts about public reporting and quality improvement programs for hospitals have focused primarily on a small number of medical conditions. Whether performance on these conditions accurately predicts the quality of broader hospital care is unknown...
Methods Using national Medicare data, we compared hospital performance at 2322 US acute care hospitals on 30-day risk-adjusted mortality, aggregated across the 3 publicly reported conditions (acute myocardial infarction, congestive heart failure, and pneumonia), with performance on a composite risk-adjusted mortality rate across 9 other common medical conditions, a composite mortality rate across 10 surgical conditions, and both composites combined...
Results..Hospitals in the top quartile of performance on publicly reported conditions had a 3.6% lower absolute risk-adjusted mortality rate on the combined medical-surgical composite than those in the bottom quartile (9.4% vs 13.0%; P less than .001). These top performers on publicly reported conditions had 5 times greater odds of being in the top quartile on the overall combined composite risk-adjusted mortality rate (odds ratio [OR], 5.3; 95% CI, 4.3-6.5)...
Conclusions and Relevance Hospital performance on publicly reported conditions can potentially be used as a signal of overall hospital mortality rates.

You can read more here from Medpage Today.

What is important to understand is that the mortality composites were for conditions OTHER than the three publicly reported ones. The connection between performance measures, real quality and outcomes is intangible and there is no warrant from the findings of this study for a direct cause and effect relationship.

What might the intangible factors be? For one thing, hospitals that score high on performance measures may be more resource rich and better staffed. After all it takes considerable time and effort to dot the i's and cross the t's in the medical record in order to score well. Performance is in large part a labor intensive medical record game. Other factors might be that a high scoring hospital has more hospitalists, more robust systems in place or a stronger culture of excellence. We really don't know, do we?

So, my final take...

What we knew before this study:

Performance does not equal quality.

Although performance measures were implemented to incent adherence to evidence based treatments they led to unintended consequences and have, for the most part, not been proven to benefit patients.

What we know in light of this study:

While there is no direct cause and effect relationship between performance and quality, both may be influenced in like manner by poorly understood external factors.

No comments: