Featured Product
This Week in Quality Digest Live
Health Care Features
Claudine Mangen
If you have the energy to try and address organizational overwork, start small
Gregory Way
Drug designers are rethinking their response to medications that affect multiple targets
Adam Zewe
Research on machine-learning models that can help doctors consult patient records more easily
Karina Montoya
Analysis of social and economic impact bolsters the fight against hepatitis C
Tom Rish
Keep it organized and ready for inspection at any time

More Features

Health Care News
Google Docs collaboration, more efficient management of quality deviations
MIT course focuses on the impact of increased longevity on systems and markets
Delivers time, cost, and efficiency savings while streamlining compliance activity
First responders may benefit from NIST contest to reward high-quality incident command dashboards
Enhances clinical data management for medtech companies
Winter 2022 release of Reliance QMS focuses on usability, mobility, and actionable insights
The tabletop diagnostic yields results in an hour and can be programmed to detect variants of the SARS-CoV-2 virus
First Responder UAS Triple Challenge focuses on using optical sensors and data analysis to improve image detection and location
Free education source for global medical device community

More News

Peter Dizikes

Health Care

Hospital Rankings Hold Up

Some basic metrics do effectively diagnose care quality

Published: Wednesday, February 19, 2020 - 13:01

Given the complexities of healthcare, do basic statistics used to rank hospitals really work well? A study co-authored by MIT economists indicates that some fundamental metrics do, in fact, provide real insight about hospital quality.

“The results suggest a substantial improvement in health if you go to a hospital where the quality scores are higher,” says Joseph Doyle, an MIT economist and co-author of a new paper detailing the study’s results.

The study was designed to work around a difficult problem in evaluating hospital quality: Some high-performing hospitals may receive an above-average number of very sick patients. Accepting those difficult cases could, on the surface, worsen the aggregate outcomes of a given hospital’s patients and make such hospitals seem less effective than they are.

However, the scholars found a way to study equivalent pools of patients, thus allowing them to judge the hospitals in level terms. Overall, the study shows, when patient sickness levels are accounted for, hospitals that score well on quality measures have 30-day readmission rates that are 15 percent lower than a set of lesser-rated hospitals, and 30-day mortality rates that are 17 percent lower.

“It wasn’t clear going in whether these quality measures do a good job of sorting hospitals out,” Doyle adds. “These results suggest that they have predictive power.”

The paper, “Evaluating Measures of Hospital Quality: Evidence from Hospital Referral Patterns,” was written by Doyle, the Erwin H. Schell Professor of Management and Applied Economics at the MIT Sloan School of Management; John Graves, an assistant professor in the Department of Health Policy at Vanderbilt University; and Jonathan Gruber, the Ford Professor of Economics at MIT. It appears in the latest issue of the Review of Economics and Statistics.

Randomized evaluations

To conduct the study, the researchers used a method that eliminates the issue of studying a skewed sample of admissions. They studied areas across the country where dispatchers’ calls are assigned randomly to different ambulance companies. Those ambulance companies tend to deliver patients to particular hospitals. Thus, otherwise similar groups of patients are admitted to different hospitals in what is essentially a random pattern; this allows outcomes to be compared among hospitals.

The patient data came primarily from Medicare claims made across the country during the period 2008–2012 and covered more than 170,000 hospital admissions for patients who had just suffered a health event requiring “nondiscretionary” hospital admission. The patients also fit some basic criteria, such as not having previously been admitted recently for the same condition.

In addition to analyzing 30-day readmission and mortality rates, the researchers looked at patient satisfaction levels. All these criteria, and more, are commonly used in hospital assessments.

The researchers also found a 37-percent difference in one-year mortality, among highly rated and lower-rated hospitals.

“I thought our results were reasonable,” says Doyle. “They’re not too big to be believed, but they suggest a substantial improvement in health if you go to a hospital where the quality scores are much higher.”

As the authors note in the paper, the subject is topical in the health policy world. Some lawmakers and experts want the hospital payment system to evolve in the direction of reimbursement for quality and outcomes, rather than treatment. As such, it is important to be able to tell if those quality measures are sturdy.

“There’s been a lot of interest in whether these quality measures are informative or not, because there is a shift away from paying for the quantity of care provided to the quality of care provided,” Doyle says. “Most of the policymakers I’ve talked to want to use these quality measures.”

Management matters

Further research will be needed to help illuminate issues surrounding hospital quality in further depth. For instance, the current study is more focused on emergency care and not on care for chronic conditions; Doyle says that analysis of chronic care is “a fascinating question” that merits further investigation.

Doyle also acknowledges the need for further study to explain why certain hospitals fare better than others on basic quality measures. He notes that some were historically quicker than others to adopt what are now almost universal practices—the allotment of blood-thinning drugs to heart patients, for instance—and suggests the rate of adoption of new practices is an important factor in this area.

“Coming from a management school, we see that a lot of the variation in outcomes stems in large part from differences in management,” Doyle says. “Do you have the right procedures in places so that it’s easy for providers to do what the guidelines suggest? Improving management could yield big improvements in patient health.”

The research was supported by the National Institutes of Health.

First published Jan. 30, 2020, in MIT News.

Discuss

About The Author

Peter Dizikes’s picture

Peter Dizikes

Peter Dizikes writes for MIT News and MIT Technology Review at the Massachusetts Institute of Technology (MIT). Dizikes writing covers social sciences, humanities, and business, with many stories involving the history of science and science policy. Dizikes has reported and written articles for The New York Times, The Boston Globe, The Washington Post, Salon, Slate, Seed, Nature online, and other publications. He was an editor/writer for ABCNews.com and for Time. He frequently wrote features on education and also produced articles for university publications. Special projects included writing an educator’s guide to evolution, accompanying NOVA’s acclaimed documentary, Judgment Day.

 

Comments

Where Do We Get Them

So where/how do patients get to see these effective metrics?