American College Of Surgeons - Inspiring Quality: Highest Standards, Better Outcomes

American College of Surgeons Issues Comments on Surgical Quality and Readmissions Studies in the Journal of the American Medical Association


CHICAGO (February 3, 2015 - 11 am ET): Studies in the February 3 issue of the Journal of the American Medical Association (JAMA) discuss one of the American College of Surgeons’ quality improvement programs, the National Surgical Quality Improvement Program (NSQIP). NSQIP is a clinical, risk-adjusted surgical outcomes registry that originated in the Department of Veterans Affairs (VA) and was demonstrated to improve surgical care for veterans. NSQIP transitioned to the private sector and similarly, it has been repeatedly reported to achieve improved surgical care and outcomes.

In the Feb.3 issue of JAMA, one study shows important new and novel clinical details about hospital readmission using NSQIP data. Two additional articles question whether the reporting of data is associated with improvement.

The study by Merkow et al identifies reasons for surgical readmissions. The study shows the overall rate of unplanned hospital readmissions for operations is 5.7 percent, largely due to surgical complications. The most common reason for readmission was surgical site infection. As Dr. Lucian Leape concludes in his accompanying editorial, the findings of this study using the clinical data from NSQIP provide an “unprecedented opportunity” to “make substantial reductions in surgical complications.” This study demonstrates the importance of using rigorously-collected, clinically-appropriate data in health care.

The subsequent two studies (by Osborne et al, and Etzioni et al.) question whether knowing surgical results is associated with improvement – and they both conclude that there is no association between having data reported back and improvement.  

It needs to be recognized, however, these studies have several significant flaws, some of which were highlighted in the associated editorial by Dr. Donald Berwick. Here are some of the problems.

  1. They did not use the right data. Both studies relied on the use of claims data, yet recent Annals of Surgery studies (Lawson 2012 and 2015) found claims data are inaccurate and inappropriate for measuring surgical complications, invalidating the use of claims data for studying surgical quality and complications.  That is one of the reasons why the Centers for Medicare and Medicaid Services (CMS) is moving away from using claims data for quality measurement. A common data saying is “garbage in garbage out.”
  2. Both studies failed to consider how quality improvement is performed in the real world. First, the studies evaluated combined overall rates of complications. Real-world experience shows hospitals tend to focus on specific complications one by one, such as surgical site infections, or a specific specialty, such as urology or orthopaedics. Second, when performance is averaged across many outcomes, methodological problems arise. As Berwick correctly noted in his editorial, “the methodological limitations of these studies…involve the loss of key, local, contextually specific information that large-scale studies of average effects ignore by design.” Simply put, these studies obscure the improvement that is happening in the real world.

Also in his accompanying editorial, Dr. Berwick emphasizes additional study problems by underscoring the fact that “it is implausible that knowing results is not useful.”  Since NSQIP was created more than 20 years ago, hospitals have continued to use their data to do better by their patients. In NSQIP, literally hundreds of hospitals are sharing their local, context-specific quality achievements and demonstrating the value of using data to get better.

See also:  Examples of Quality Improvement in ACS NSQIP® hospitals

Remarks from Clifford Ko, MD, FACS, ACS Director of the Division of Research and Optimal Patient Care

“Studies have consistently shown claims data should not be used to measure quality, and that’s why leading organizations including CMS have moved to using clinical registry data. It is irresponsible to use data that are known to be an inaccurate measure of quality to determine the effectiveness of a quality improvement program. These studies reinforce the need to move beyond claims data for use in quality measurement.”

“Quality is a journey not a destination. The health care community must continually double down on efforts. Rather than focusing on studies that attempt to take a defeatist stance, we need to seek solutions.”

“Data are essential, but not sufficient. Hospitals that do well must act on their data. Correctly diagnosing the patient does not, by itself, lead to improved health. Similarly, once hospitals understand where their problems lie, they must identify and implement solutions.”

# # #

About the American College of Surgeons

The American College of Surgeons is a scientific and educational organization of surgeons that was founded in 1913 to raise the standards of surgical practice and improve the quality of care for surgical patients. The College is dedicated to the ethical and competent practice of surgery. Its achievements have significantly influenced the course of scientific surgery in America and have established it as an important advocate for all surgical patients. The College has more than 80,000 members and is the largest organization of surgeons in the world. For more information, visit


Sally Garneski
American College of Surgeons

Sarah Stakston
Weber Shandwick