When a national hospital safety group issued its biannual ratings last week, Oregon’s hospital executives were quick to slam its methodology and — in some cases — the reliability of health care ratings in general.
The Leapfrog Group gave St. Charles Redmond a D and St. Charles Bend a C in its biannual report. The scores took a hit from reportedly low intensive care unit staffing.
Leapfrog, an independent group that works with the help of an eight-member volunteer panel of patient safety experts, is among several national rating agencies that aggregate safety metrics from a variety of sources, including the Centers for Medicare & Medicaid Services and the American Hospital Association.
Some of the data used in the scoring, however, was up to two years old, underscoring questions about the value of Leapfrog’s ratings and that of the many other rating agencies, such as Healthgrades, the Joint Commission, U.S. News & World Report and Consumer Report.
Pamela Steinke, vice president of quality at St. Charles Health System, pointed out that two weeks ago she got a letter from Healthgrades notifying her that St. Charles Bend was being awarded for clinical excellence for the fourth year in a row.
“How do you get overall clinical excellence from one grading organization and then you’re middle-of-the-road average on another?” she said.
Steinke said she’s heard similar stories from other hospital administrators.
Leapfrog’s report based its scores on 28 measures, including outcomes and management structures in place to prevent errors and injuries. Eight of those measures were taken from a free, optional Leapfrog Hospital Survey, which St. Charles opted not to take.
Erica Mobley, a spokesperson for LeapFrog, said hospitals’ scores were not lowered if they didn’t take Leapfrog’s survey; those categories simply weren’t weighed in their score. A look at the scores of some of the 1,400 hospitals that did take the survey, however, showed they tended to score well, raising the question of whether their scores were inflated.
Mobley said that’s not the case, reasoning that some hospitals scored poorly on the survey.
“You can make the argument that the more data you have, the less likely one particular bad measure is to influence your score, because, obviously, there are more measures total,” she said.
In particular, Steinke questioned Leapfrog’s score of 5 out of a possible 100 for ICU physician staffing in both Bend and Redmond, which was the lowest score of any hospital. The ICU data came from a 2011 AHA survey.
ICUs are generally staffed by so-called intensivists, doctors who specialize in providing intensive care. St. Charles Bend does staff its ICU with intensivists, but St. Charles Redmond does not, Steinke said. Instead, physicians in Redmond can teleconference with intensivists elsewhere, she said. It’s important to be transparent about hospital safety data, but unless the data is of high integrity, it’s not very useful, as is the case with the multitude of ratings such as Leapfrog’s that are currently being released, said Charles Kilo, chief medical officer at Oregon Health & Science University.
“The amount of variability in the ratings ought to make people scratch their heads,” he said. “Why that degree of variability? It doesn’t make sense. Something isn’t adding up.”
In baseball, there’s little room for disagreement over whether a player hit a home run or a strike, he said. In health care, though, there aren’t clear truths around quality or safety. For example, a hospital that’s aggressive about getting patients home might have high readmission rates, while one that keeps patients in for extended periods of time would have lower rates. In addition, Kilo said, many rating agencies, including Leapfrog, make money from selling hospitals the right to promote positive ratings or from selling their reports.
“They pretend like they’re doing it for the public’s good, and that is not all why they’re doing it,” he said. “They’re doing it because each of them has a financial model that drives it, and in fact, their financial model influences how they report the data, and that’s a problem.”
Mobley emphasized that the ratings are not the end-all, but simply a tool for patients to start a conversation with their doctors. Preventable medical errors are now the third leading cause of death in the country, and consumers should stay informed, she said.
“It is important for people to know that there is wide variation on how hospitals perform on patient safety and that patient safety really is a serious problem,” Mobley said.