Being named one of the top 50 in the United States by U.S. News & World Report is a big deal for hospitals, and health policy researchers often point to these top 50 as models for health reform. Hospital leaders use the rankings as marketing tools to attract patients and funding, and patients often look to the survey when choosing a health care facility. U.S. News & World Report‘s method report states that its rankings are “based largely on hard data,” but an article in the April 14 issue of Annals of Internal Medicine1 raises some doubts.
U.S. News & World Report 2 combines three domains with approximately equal weighting, reports study author Ashwini R. Sehgal, MD. These are structure, process, and outcomes. Structure is the people and resources, process refers to how care is delivered, and outcomes are the results of such care. For the top 50, structure and outcomes were assessed by objective measures: staffing, patient volume, adverse events, mortality, writes Sehgal, a professor in the division of nephrology at MetroHealth Medical Center, one of Case Western Reserve University’s teaching hospitals in Cleveland. Process, however, was assessed primarily by one subjective measure: hospital reputation. How was reputation assessed? U.S. News & World Report surveyed 250 board-certified physicians across the country. These were randomly selected from the AMA’s Physician Masterfile in each specialty ranked (there were 12, including respiratory care). The physicians were asked to identify five hospitals in their specialty that provide the best care. Subjective and objective scores were combined as a weighted sum and transformed into a 1,000-point scale to create the total score. Then, for each specialty, the 50 hospitals (out of 1,859 eligible hospitals) with the highest total score were ranked numerically.
Sehgal wanted to understand the role of reputation score, and he looked at distribution of reputation score and other components of the total score for the top 50 hospitals; distribution of reputation scores among 100 unranked hospitals; top 50 hospitals’ ranking based on reputation score alone; and the association between reputation score and total score from all objective measures among the top 50.
The U.S. News & World Report rankings—arrived at by combining both subjective and objective measures—should be an accurate measurement of hospital quality, but Sehgal writes, “The relative standing of the top 50 hospitals largely reflects the subjective reputation of these hospitals … little relationship exists between reputation and objective measures in the top 50 hospitals.”
It appears that reputation greatly influenced the standings of the top 50. The reputation scores of the 100 randomly selected unranked hospitals were minimal, although many had high objective ratings. This means that hospitals lacking national recognition are unlikely to be highly ranked. What is the impact? Because reputation score is determined by asking a relatively few specialists to identify the five best hospitals in their specialty, only nationally recognized hospitals are likely to come to mind. High rankings enhance reputation, perpetuating it through the years. Reputations are quickly damaged, however, and Sehgal warns that highly ranked hospitals should be concerned about rankings based primarily on reputation. Additionally, there is the danger that hospitals with already high rankings may see no reason to improve quality.
So, although this survey is highly publicized by the press and touted by the highly ranked facilities, all may not be as appears, and we would all do well to carefully assess what it means to be a “top hospital.”
- Sehgal AR. The role of reputation in U.S. News & World Report‘s rankings of the top 50 American hospitals. Ann Intern Med. 2010;152:521-5.
- America’s Best Hospitals. Available at: health.usnews.com/best-hospitals. Accessed April 14, 2010.