Considerable variability exists between surgeons’ assessments of a patient's individual
preoperative surgical risk. Surgical risk calculators are not routinely used despite
their validation. We sought to compare thoracic surgeons’ prediction of patients’
risk of postoperative adverse outcomes vs a surgical risk calculator, the Surgical
Risk Preoperative Assessment System (SURPAS). We developed vignettes from 30 randomly
selected patients who underwent thoracic surgery in the American College of Surgeons’
National Surgical Quality Improvement Program database. Twelve thoracic surgeons estimated
patients’ preoperative risks of postoperative morbidity and mortality. These were
compared to SURPAS estimates of the same vignettes. C-indices and Brier scores were
calculated for the surgeons’ and SURPAS estimates. Agreement between surgeon estimates
was examined using intraclass correlation coefficients (ICCs). Surgeons estimated
higher morbidity risk compared to SURPAS for low-risk patients (ASA classes 1–2, 11.5%
vs 5.1%, P ≤ 0.001) and lower morbidity risk compared to SURPAS for high-risk patients (ASA
class 5, 37.6% vs 69.8%, P < 0.001). This trend also occurred in high-risk patients for mortality (ASA 5, 11.1%
vs 44.3%, P < 0.001). C-indices for SURPAS vs surgeons were 0.84 vs 0.76 (P = 0.3) for morbidity and 0.98 vs 0.85 (P = 0.001) for mortality. Brier scores for SURPAS vs surgeons were 0.1579 vs 0.1986
for morbidity (P = 0.03) and 0.0409 vs 0.0543 for mortality (P = 0.006). ICCs showed that surgeons had moderate risk agreement for morbidity (ICC = 0.654)
and mortality (ICC = 0.507). Thoracic surgeons and patients could benefit from using
a surgical risk calculator to better estimate patients’ surgical risks during the
informed consent process.
Graphical Abstract
Graphical Abstract
Keywords
Abbreviations:
SURPAS (Surgical Risk Preoperative Assessment System), ACS (American College of Surgeons), NSQIP (National Surgical Quality Improvement Program), ICC (intraclass correlation coefficient), ASA (American Society of Anesthesiologists), VASQIP (Veterans Administration Surgical Quality Improvement Program), CPT (Current Procedural Terminology), RVU (Relative Value Unit), ThORN (Thoracic Surgery Outcomes Research Network), RedCap (Research Electronic Data Capture), VATS (Video Assisted Thoracic Surgery), IRR (Incidence Rate Ratio), CI (Confidence Interval), IQR (Interquartile Range)To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Seminars in Thoracic and Cardiovascular SurgeryAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Systematic review and narrative synthesis of surgeons perceptions of postoperative outcomes and risks.BJS Open. 2020; 4: 16-26
- Development and evaluation of the universal ACS NSQIP surgical risk calculator: A decision aid and informed consent tool for patients and surgeons.J Am Coll Surg. 2013; 217 (e831-833): 833-842
- Practical guide to surgical data sets: Veterans Affairs Surgical Quality Improvement Program (VASQIP).JAMA Surg. 2018; 153: 768-769
- Society of Thoracic Surgeons Quality Measurement Task Force. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 1–coronary artery bypass grafting surgery.Ann Thorac Surg. 2009; 88 (PMID:19559822): S2-22https://doi.org/10.1016/j.athoracsur.2009.05.053
- Society of Thoracic Surgeons Quality Measurement Task Force. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 2–isolated valve surgery.Ann Thorac Surg. 2009; 88 (PMID:19559823): S23-S42https://doi.org/10.1016/j.athoracsur.2009.05.056
- Surgical Risk Preoperative Assessment System (SURPAS): I. Parsimonious, clinically meaningful groups of postoperative complications by factor analysis.Ann Surg. 2016; 263: 1042-1048
- Surgical Risk Preoperative Assessment System (SURPAS): II. Parsimonious risk models for postoperative adverse outcomes addressing need for laboratory variables and surgeon specialty-specific models.Ann Surg. 2016; 264: 10-22
- Surgical Risk Preoperative Assessment System (SURPAS): III. Accurate preoperative prediction of 8 adverse outcomes using 8 predictor variables.Ann Surg. 2016; 264: 23-31
- Comparison of accuracy of prediction of postoperative mortality and morbidity between a new, parsimonious risk calculator (SURPAS) and the ACS Surgical Risk Calculator.Am J Surg. 2020; 219: 1065-1072https://doi.org/10.1016/j.amjsurg.2019.07.036
- The preoperative risk tool SURPAS accurately predicts outcomes in emergency surgery.Am J Surg. 2021; (S0002-9610(21)00007-6. Epub ahead of print. PMID:33485618)https://doi.org/10.1016/j.amjsurg.2021.01.004
- Accurate preoperative prediction of unplanned 30-day postoperative readmission using 8 predictor variables.Surgery. 2019; 166: 812-819
- Accurate preoperative prediction of discharge destination using 8 predictor variables: A NSQIP analysis.J Am Coll Surg. 2020; 230 (e62): 64-75
- Refining the predictive variables in the "Surgical Risk Preoperative Assessment System" (SURPAS): A descriptive analysis.Patient Saf Surg. 2019; 13: 28
- External validation of surgical risk preoperative assessment system in pulmonary resection.Ann Thorac Surg. 2021; 112: 228-237
- Performance comparison between SURPAS and ACS NSQIP surgical risk calculator in pulmonary resection.Ann Thorac Surg. 2021; 111: 1643-1651
- Assessment of attitudes towards future implementation of the “Surgical Risk Preoperative Assessment System” (SURPAS) tool: A pilot study among patients, surgeons, and hospital administrators.Pat Saf Surg. 2018; 12: 12
- The value of the “Surgical Risk Preoperative Assessment System” (SURPAS) in preoperative consultation for elective surgery: A pilot study.Pat Saf Surg. 2020; 14: 31
- Use of Surgical Risk Preoperative Assessment System (SURPAS) and patient satisfaction during informed consent for surgery.J Am Coll Surg. 2020; 230: 1025-1034
- How accurate are the surgical risk preoperative assessment system (SURPAS) universal calculators in total joint arthroplasty?.Clin Orthop Relat Res. 2020; 478: 241-251
- Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach.Biometrics. 1988; 44: 837-845
- Assessing predictive accuracy: How to compare Brier scores.J Clin Epidemiol. 1991; 44: 1141-1146
- Probabilistic prediction in patient management and clinical trials.Stat Med. 1986; 5: 421-433
- Intraclass correlation – A discussion and demonstration of basic features.PLOS ONE. 2019; 14e0219854https://doi.org/10.1371/journal.pone.0219854
- Comparing clinical judgment with the MySurgeryRisk algorithm for preoperative risk assessment: A pilot usability study.Surgery. 2019; 165 (Epub 2019 Feb 18 PMID:30792011 PMCID: PMC6502657): 1035-1045https://doi.org/10.1016/j.surg.2019.01.002
- Artificial intelligence and surgical decision-making.JAMA Surg. 2020; 155 (PMID:31825465 PMCID: PMC7286802): 148-158https://doi.org/10.1001/jamasurg.2019.4917
Article info
Publication history
Published online: November 12, 2021
Footnotes
Funding: This work was supported by an internal grant from the Department of Surgery, University of Colorado School of Medicine. The funding organization had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Conflicts of Interest: The authors report no conflicts of interest. The authors of this manuscript and the developers of SURPAS have no financial interests in SURPAS. The ACS-NSQIP and participating hospitals are the source of these data; they have not verified and are not responsible for the statistical validity of the data analysis or the conclusions derived by the authors.
Identification
Copyright
© 2021 Elsevier Inc. All rights reserved.
ScienceDirect
Access this article on ScienceDirectLinked Article
- Commentary: Risk Assessment Before Thoracic Surgery: The Human FactorSeminars in Thoracic and Cardiovascular SurgeryVol. 34Issue 4
- PreviewIn this study Dyas et al 1 compared the surgeon ability to estimate perioperative risk with the risk predicted by the Surgical Risk Preoperative Assessment System(SURPAS), a previously validated scoring system. To this purpose the investigators asked 20 surgeons to estimate the risk of 30 random thoracic surgical patients drawn from the American College of Surgeons’ National Surgical Quality Improvement Program (NSQIP) database. The surgeons had to estimate their surgical risk from vignettes summarizing patients’ main characteristics and risk factors.
- Full-Text
- Preview
