Advertisement

Comparison of Preoperative Surgical Risk Estimated by Thoracic Surgeons vs a Standardized Surgical Risk Prediction Tool

  • Adam R. Dyas
    Affiliations
    Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, Colorado
    Search for articles by this author
  • Kathryn L. Colborn
    Affiliations
    Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, Colorado

    Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, Colorado
    Search for articles by this author
  • Michael R. Bronsert
    Affiliations
    Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, Colorado

    Adult and Child Center for Health Outcomes Research and Delivery Science, University of Colorado School of Medicine, Aurora, Colorado
    Search for articles by this author
  • William G. Henderson
    Affiliations
    Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, Colorado

    Adult and Child Center for Health Outcomes Research and Delivery Science, University of Colorado School of Medicine, Aurora, Colorado

    Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, Colorado
    Search for articles by this author
  • Nicholas J. Mason
    Affiliations
    Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, Colorado
    Search for articles by this author
  • Paul D. Rozeboom
    Affiliations
    Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, Colorado
    Search for articles by this author
  • Nisha Pradhan
    Affiliations
    Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, Colorado
    Search for articles by this author
  • Anne Lambert-Kerzner
    Affiliations
    Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, Colorado

    Adult and Child Center for Health Outcomes Research and Delivery Science, University of Colorado School of Medicine, Aurora, Colorado
    Search for articles by this author
  • Robert A. Meguid
    Correspondence
    Address reprint requests to Robert A. Meguid, MD, MPH, FACS, Division of Cardiothoracic Surgery, Department of Surgery, University of Colorado Denver, 12631 E. 17th Ave, C-310, Aurora, CO 80045.
    Affiliations
    Surgical Outcomes and Applied Research Program, Department of Surgery, University of Colorado School of Medicine, Aurora, Colorado

    Adult and Child Center for Health Outcomes Research and Delivery Science, University of Colorado School of Medicine, Aurora, Colorado
    Search for articles by this author
Published:November 12, 2021DOI:https://doi.org/10.1053/j.semtcvs.2021.11.008
      Considerable variability exists between surgeons’ assessments of a patient's individual preoperative surgical risk. Surgical risk calculators are not routinely used despite their validation. We sought to compare thoracic surgeons’ prediction of patients’ risk of postoperative adverse outcomes vs a surgical risk calculator, the Surgical Risk Preoperative Assessment System (SURPAS). We developed vignettes from 30 randomly selected patients who underwent thoracic surgery in the American College of Surgeons’ National Surgical Quality Improvement Program database. Twelve thoracic surgeons estimated patients’ preoperative risks of postoperative morbidity and mortality. These were compared to SURPAS estimates of the same vignettes. C-indices and Brier scores were calculated for the surgeons’ and SURPAS estimates. Agreement between surgeon estimates was examined using intraclass correlation coefficients (ICCs). Surgeons estimated higher morbidity risk compared to SURPAS for low-risk patients (ASA classes 1–2, 11.5% vs 5.1%, P ≤ 0.001) and lower morbidity risk compared to SURPAS for high-risk patients (ASA class 5, 37.6% vs 69.8%, P < 0.001). This trend also occurred in high-risk patients for mortality (ASA 5, 11.1% vs 44.3%, P < 0.001). C-indices for SURPAS vs surgeons were 0.84 vs 0.76 (P = 0.3) for morbidity and 0.98 vs 0.85 (P = 0.001) for mortality. Brier scores for SURPAS vs surgeons were 0.1579 vs 0.1986 for morbidity (P = 0.03) and 0.0409 vs 0.0543 for mortality (P = 0.006). ICCs showed that surgeons had moderate risk agreement for morbidity (ICC = 0.654) and mortality (ICC = 0.507). Thoracic surgeons and patients could benefit from using a surgical risk calculator to better estimate patients’ surgical risks during the informed consent process.

      Graphical Abstract

      Keywords

      Abbreviations:

      SURPAS (Surgical Risk Preoperative Assessment System), ACS (American College of Surgeons), NSQIP (National Surgical Quality Improvement Program), ICC (intraclass correlation coefficient), ASA (American Society of Anesthesiologists), VASQIP (Veterans Administration Surgical Quality Improvement Program), CPT (Current Procedural Terminology), RVU (Relative Value Unit), ThORN (Thoracic Surgery Outcomes Research Network), RedCap (Research Electronic Data Capture), VATS (Video Assisted Thoracic Surgery), IRR (Incidence Rate Ratio), CI (Confidence Interval), IQR (Interquartile Range)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Seminars in Thoracic and Cardiovascular Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Dilaver N.M.
        • Gwilym B.L.
        • Preece R.
        • et al.
        Systematic review and narrative synthesis of surgeons perceptions of postoperative outcomes and risks.
        BJS Open. 2020; 4: 16-26
        • Bilimoria K.Y.
        • Liu Y.
        • Paruch J.L.
        • et al.
        Development and evaluation of the universal ACS NSQIP surgical risk calculator: A decision aid and informed consent tool for patients and surgeons.
        J Am Coll Surg. 2013; 217 (e831-833): 833-842
        • Massarweh N.N.
        • Kaji A.H.
        • Itani K.M.F.
        Practical guide to surgical data sets: Veterans Affairs Surgical Quality Improvement Program (VASQIP).
        JAMA Surg. 2018; 153: 768-769
        • Shahian D.M.
        • O'Brien S.M.
        • Filardo G.
        • et al.
        Society of Thoracic Surgeons Quality Measurement Task Force. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 1–coronary artery bypass grafting surgery.
        Ann Thorac Surg. 2009; 88 (PMID:19559822): S2-22https://doi.org/10.1016/j.athoracsur.2009.05.053
        • O'Brien S.M.
        • Shahian D.M.
        • Filardo G.
        • et al.
        Society of Thoracic Surgeons Quality Measurement Task Force. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 2–isolated valve surgery.
        Ann Thorac Surg. 2009; 88 (PMID:19559823): S23-S42https://doi.org/10.1016/j.athoracsur.2009.05.056
        • Meguid R.A.
        • Bronsert M.R.
        • Juarez-Colunga E.
        • et al.
        Surgical Risk Preoperative Assessment System (SURPAS): I. Parsimonious, clinically meaningful groups of postoperative complications by factor analysis.
        Ann Surg. 2016; 263: 1042-1048
        • Meguid R.A.
        • Bronsert M.R.
        • Juarez-Colunga E.
        • et al.
        Surgical Risk Preoperative Assessment System (SURPAS): II. Parsimonious risk models for postoperative adverse outcomes addressing need for laboratory variables and surgeon specialty-specific models.
        Ann Surg. 2016; 264: 10-22
        • Meguid R.A.
        • Bronsert M.R.
        • Juarez-Colunga E.
        • et al.
        Surgical Risk Preoperative Assessment System (SURPAS): III. Accurate preoperative prediction of 8 adverse outcomes using 8 predictor variables.
        Ann Surg. 2016; 264: 23-31
        • Khaneki S.
        • Bronsert M.R.
        • Henderson W.G.
        • et al.
        Comparison of accuracy of prediction of postoperative mortality and morbidity between a new, parsimonious risk calculator (SURPAS) and the ACS Surgical Risk Calculator.
        Am J Surg. 2020; 219: 1065-1072https://doi.org/10.1016/j.amjsurg.2019.07.036
        • Rozeboom P.D.
        • Bronsert M.R.
        • Henderson W.G.
        • et al.
        The preoperative risk tool SURPAS accurately predicts outcomes in emergency surgery.
        Am J Surg. 2021; (S0002-9610(21)00007-6. Epub ahead of print. PMID:33485618)https://doi.org/10.1016/j.amjsurg.2021.01.004
        • Gibula D.R.
        • Singh A.B.
        • Bronsert M.R.
        • et al.
        Accurate preoperative prediction of unplanned 30-day postoperative readmission using 8 predictor variables.
        Surgery. 2019; 166: 812-819
        • Singh A.B.
        • Bronsert M.R.
        • Henderson W.G.
        • et al.
        Accurate preoperative prediction of discharge destination using 8 predictor variables: A NSQIP analysis.
        J Am Coll Surg. 2020; 230 (e62): 64-75
        • Henderson W.G.
        • Bronsert M.R.
        • Hammermeister K.E.
        • et al.
        Refining the predictive variables in the "Surgical Risk Preoperative Assessment System" (SURPAS): A descriptive analysis.
        Patient Saf Surg. 2019; 13: 28
        • Chudgar N.P.
        • Yan S.
        • Hsu M.
        • et al.
        External validation of surgical risk preoperative assessment system in pulmonary resection.
        Ann Thorac Surg. 2021; 112: 228-237
        • Chudgar N.P.
        • Yan S.
        • Hsu M.
        • et al.
        Performance comparison between SURPAS and ACS NSQIP surgical risk calculator in pulmonary resection.
        Ann Thorac Surg. 2021; 111: 1643-1651
        • Lambert-Kerzner A.
        • Ford K.L.
        • Hammermeister K.E.
        • et al.
        Assessment of attitudes towards future implementation of the “Surgical Risk Preoperative Assessment System” (SURPAS) tool: A pilot study among patients, surgeons, and hospital administrators.
        Pat Saf Surg. 2018; 12: 12
        • Bronsert M.R.
        • Lambert-Kerzner A.
        • Henderson W.G.
        • et al.
        The value of the “Surgical Risk Preoperative Assessment System” (SURPAS) in preoperative consultation for elective surgery: A pilot study.
        Pat Saf Surg. 2020; 14: 31
        • Wiesen B.M.
        • Bronsert M.R.
        • Aasen D.M.
        • et al.
        Use of Surgical Risk Preoperative Assessment System (SURPAS) and patient satisfaction during informed consent for surgery.
        J Am Coll Surg. 2020; 230: 1025-1034
        • Trickey A.W.
        • Ding Q.
        • Harris A.H.S.
        How accurate are the surgical risk preoperative assessment system (SURPAS) universal calculators in total joint arthroplasty?.
        Clin Orthop Relat Res. 2020; 478: 241-251
        • DeLong E.R.
        • DeLong D.M.
        • Clarke-Pearson D.L.
        Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach.
        Biometrics. 1988; 44: 837-845
        • Redelmeier D.A.
        • Bloch D.A.
        • Hickam D.H.
        Assessing predictive accuracy: How to compare Brier scores.
        J Clin Epidemiol. 1991; 44: 1141-1146
        • Spiegelhalter D.J.
        Probabilistic prediction in patient management and clinical trials.
        Stat Med. 1986; 5: 421-433
        • Liljequist D.
        • Elfving B.
        • Roaldsen K.S.
        Intraclass correlation – A discussion and demonstration of basic features.
        PLOS ONE. 2019; 14e0219854https://doi.org/10.1371/journal.pone.0219854
        • Brennan M.
        • Puri S.
        • Ozrazgat-Baslanti T.
        • et al.
        Comparing clinical judgment with the MySurgeryRisk algorithm for preoperative risk assessment: A pilot usability study.
        Surgery. 2019; 165 (Epub 2019 Feb 18 PMID:30792011 PMCID: PMC6502657): 1035-1045https://doi.org/10.1016/j.surg.2019.01.002
        • Loftus T.J.
        • Tighe P.J.
        • Filiberto A.C.
        • et al.
        Artificial intelligence and surgical decision-making.
        JAMA Surg. 2020; 155 (PMID:31825465 PMCID: PMC7286802): 148-158https://doi.org/10.1001/jamasurg.2019.4917

      Linked Article

      • Commentary: Risk Assessment Before Thoracic Surgery: The Human Factor
        Seminars in Thoracic and Cardiovascular SurgeryVol. 34Issue 4
        • Preview
          In this study Dyas et al 1 compared the surgeon ability to estimate perioperative risk with the risk predicted by the Surgical Risk Preoperative Assessment System(SURPAS), a previously validated scoring system. To this purpose the investigators asked 20 surgeons to estimate the risk of 30 random thoracic surgical patients drawn from the American College of Surgeons’ National Surgical Quality Improvement Program (NSQIP) database. The surgeons had to estimate their surgical risk from vignettes summarizing patients’ main characteristics and risk factors.
        • Full-Text
        • PDF