Machine learning-based prediction of new onset of atrial fibrillation after mitral valve surgery

Abstract

Background

New-onset postoperative atrial fibrillation (nPOAF) is a common complication after cardiac surgery (30–50%), being associated with unfavorable long-term outcomes. Using the Society of Thoracic Surgeons National Adult Cardiac Database, we used machine learning (ML) to predict nPOAF and related 30-day outcomes following mitral valve (MV) surgery. A total of 27,856 MV operations were performed at 910 centers between 7/1/2017 and 6/30/2020 on patients without AF or a prior permanent pacemaker. The primary endpoint was nPOAF postoperatively. ML techniques utilized included penalized logistic regression, gradient boosting, decision trees, and random forests.

Results

The overall incidence of nPOAF was 35.4% and that of new pacemaker insertion was 5.6%. Patients who developed nPOAF were older (67 ± 10 vs 60 ± 13 years), had more mitral valve stenosis (14.1% vs 11.7%), and hypertension (72.1% vs 63.3%). They underwent more mitral valve replacement (39.1% vs 32.7%) and coronary artery bypass grafting (23.9% vs 16%). For predicting nPOAF, ML methods offer sensitivity, specificity and precision superior to logistic regression. The accuracy rate was identical with penalized and non-penalized logistic regression (0.672).

Conclusions

Predicting nPOAF and its short-term sequelae following MV surgery remains highly challenging. Machine learning methods offer a moderate degree of improvement in predicting nPOAF even in large national-level studies, in the absence of multi-modal data, such as real-time wearables data, electrocardiograms, heart rhythm monitoring, or cardiac imaging.

Background

Atrial fibrillation (AF) is the most common heart rhythm dysfunction in the USA, affecting more than 2 million individuals [1]. Its prevalence is expected to increase in the coming decades [1].

New-onset postoperative AF (POAF) is a common complication (30–50%) after cardiac surgery [1, 2]. It is associated with unfavorable near-term and long-term outcomes, including a higher risk of stroke, prolonged hospital length of stay, and strained hospital resources [3,4,5]. It is therefore important to develop methods to predict its occurrence. In our study, we adopted the Society of Thoracic Surgeons (sts.org) definition for new-onset POAF: the occurrence of any postoperative, in-hospital, atrial fibrillation/flutter episode longer than 1 h and/or requiring treatment [6, 7]. This definition applies only to patients who were not in AF at the start of surgery. Preoperative AF status was the first rhythm documented on the anesthesia record upon entry to the operating room. POAF diagnosis tools included ECG recording (1 or more leads), continuous ECG monitoring for 48–72 h postoperatively, loop memory monitors, symptom event monitors, patch recorders, or implantable loop recorders.

The 2017 HRS/EHRA/ECAS/APHRS/SOLACE expert consensus statement on catheter and surgical ablation of atrial fibrillation identifies modifiable (e.g., hypertension, obesity, alcohol consumption) and non-modifiable (e.g., age, sex, race, family history) risk factors for developing atrial fibrillation during the life course [1]. Enhancing the ability to predict the risk of new-onset AF following cardiac surgery may lead to better individualized treatment strategies [1]. State-of-the art machine learning methods have produced excellent results in surgical outcome prediction [8,9,10]. Using data from the Society of Thoracic Surgeons (STS) Adult Cardiac Surgery Database (ACSD) retrospectively for patients undergoing MV surgery, we sought to determine whether machine learning methods produce superior new-onset POAF predictions compared to standard methods.

Study population

The STS National ACSD version 2.9 was queried from July 1, 2017, to June 30, 2020. Deidentified data were obtained via provider user files, and included demographics, comorbid conditions, pre-/intra-/postoperative characteristics, and 30-day outcomes. We included patients age ≥ 18 years who underwent elective surgical MV repair or replacement (with or without tricuspid valve surgery and PFO closure) via full conventional sternotomy, partial sternotomy, right thoracotomy, or robotic initial operative approach. We excluded patients who had: (1) a history of AF or a prior permanent pacemaker, (2) a concomitant procedure for AF (cardioversion, catheter ablation), (3) a transcatheter MV repair or replacement procedure, (4) a cardiac congenital or concomitant surgical procedure (aortic or pulmonic valve surgery, aortic aneurysm), (5) a heart/heart–lung transplant or the implantation of a VAD/temporary/permanent assist device, (6) a percutaneous or port access operative approach, or (7) other non-cardiac procedures. In Figure 1, Consolidated Standards of Reporting Trials (CONSORT) diagram [11, 12] delineates the study cohort. The primary endpoint was the occurrence of de novo POAF postoperatively.

Fig. 1

CONSORT diagram depicting the study final analytic cohort upon applying inclusion/exclusion criteria

Missing data

For binary variables (‘Yes/No’), missingness was equated with a negative response (‘No’). Missing height and weight (n = 8 each), perfusion, and cross-clamp times (n = 59 and 89, respectively) were mean-imputed. Mode imputation was used when gender information was missing (n = 2).

Statistical analyses

Data were summarized using means/standard deviations, medians/interquartile ranges, and frequency counts/percentages, as appropriate. Group comparisons, including by POAF status, were based on the two-sample t test with unequal variances, Wilcoxon’s rank sum, and the Chi-square test.

To predict de novo POAF, we compared standard methodology (logistic regression) to several machine learning (ML) techniques: penalized logistic regression, random forest, and extreme gradient boosting (XGBoost) [13]. Preoperative MV lesion set information is presented in Supplemental Table 1. The list of variables used to predict POAF is available in Supplemental Table 2 and includes clinically important pre- and intraoperative variables available. Training and testing datasets have been created by randomly splitting the study data in 70:30 ratio. Classification error calculated on the test dataset was defined as the number of incorrect predictions (false negatives or false positives) divided by the size of the validation dataset. Method performance was assessed and compared across the different classification approaches based on the following optimal threshold measures: the concordance index or the area under the receiver operating characteristic (AUROC) curve, the F1 score, precision (or positive predicted value (PPV)), recall (sensitivity), accuracy, specificity, and negative predicted value (NPV). Test data permutation feature importance has been calculated based on the algorithm developed by Fisher et al. (2019) [14] and uses classification error as loss function. Statistical significance was declared at two-sided 5% level, and there were no adjustments for multiplicity. All analyses have been completed in SAS v 9.4 (SAS Institute, Cary, NC) and R v.4.1 (www.r-project.org) using package mlr3 [15]. Test data hyperparameter tuning was performed using the default options in mlr3, using a grid search. For in-depth detail, please refer to mlr3tuning.mlr-org.com.

Results

During the study period, a total of 28,856 MV operations were performed at 910 centers on patients. Of them, 9,863 (35.4%) experienced new-onset POAF. Risk factors for POAF included older age, diabetes, dyslipidemia, hypertension, cerebrovascular disease, prior myocardial infarction, sleep apnea, and chronic lung disease. Patients with POAF were more often on preoperative beta-blocker medication and had a higher STS PROM score (Table 1). Patients who developed POAF had longer postoperative hospital length of stay (Table 3). They experienced more complications postoperatively: higher stroke rates, more bleeding, more prolonged ventilation, and more renal failure (Table 2). Thirty-day mortality, hospital readmission, and arrhythmia-related hospital readmission rates were higher among patients who experienced POAF (Table 3).

Table 1 Preoperative characteristics by new-onset postoperative atrial fibrillation status

Table 2 Operative characteristics by new-onset postoperative atrial fibrillation status

Table 3 Select operative and thirty-day outcomes by new-onset postoperative atrial fibrillation status

Table 4 presents a comparison of de novo POAF prediction via ML and standard methods. The highest concordance index (AUROC) was achieved by penalized and non-penalized logistic regression models. Accuracy was relatively similar across methodologies, and numerically largest for logistic regression (0.667). XGBoost had the highest recall (sensitivity) at 0.311, and the highest F1 score. Random forest had the highest specificity (0.926) and precision (0.585). Feature importance (Supplemental Table 3 includes top 10%) reveals that all methods employed identify older age at surgery as top risk factor for postoperative nPOAF. Other risk factors identified were valvular disease (MV stenosis, AV/TV insufficiency, PV disease), as well as intraoperative characteristics (initial operative approach, coronary artery bypass grafting, perfusion time).

Table 4 Summary measures of de novo postoperative atrial fibrillation prediction methods

Discussion

There is vast literature identifying older age, surgery for valvular heart disease, and prior history of major cardiovascular disease as independent risk factors for postoperative AF following adult cardiac surgery. These findings were confirmed independently by Auer et al. (2005) [16], Omae and Kanmura (2012) [17], Greenberg et al. (2017) [18], Rezaei et al. (2020) [19], and Lopes and Agrawal (2022) [20], among others. Based on feature importance quantification, our study further confirms that patient older age, heart valve disease (MV stenosis, AV/TV insufficiency, PV disease), and intraoperative elements (initial operative approach, coronary artery bypass grafting, perfusion time) are associated with a greater risk of de novo postoperative POAF.

In our study, machine learning methods for predicting POAF performed better than standard statistical methodology in terms of precision, recall, F1 score, and specificity; their performance was comparable in terms of the other metrics, including the AUROC. Although artificial intelligence-enabled electrocardiogram interpretation may further benefit POAF prediction accuracy, scaling up such capabilities remains highly challenging [21,22,23,24].

In the current study, there were gains associated with the use of ML methods, but also limitations. The informational content available to predict POAF appears to be insufficient, or at least, insufficiently nonlinear in nature, to capitalize on the advantages of machine learning methods. These findings suggest that other multi-model features may need to be collected routinely to predict POAF better. They may include proteomic and multiomic molecular data [25,26,27,28], and real-time electrocardiograms and photoplethysmograms [29,30,31]. Growing capabilities of 24/7 wearable devices offer novel insights into AF pathogenesis by providing real-time physiological data, such as body temperature, pH, physical activity phenotype, and sleep patterns. Novel sweat monitors inform in real time about electrolyte, metabolic, and stress biomarkers, which could be critical to predicting POAF and other life-threatening arrhythmias [31, 32]. Integration of wearable devices into the clinical flow in the near future is progressing slowly due to regulatory challenges and lack of data standards [33,34,35,36,37]. Presently, fitness and consumer electronics industries lead the way in developing wearable physiological monitors, which offer ECG/PPG sensors and ML-enabled algorithms for detection of AF in general population. Close partnership among clinical, engineering, regulatory, and industrial communities, is needed to accelerate this potentially groundbreaking technology, which will likely revolutionize real-time detection, prediction, and prevention of life-threatening heart rhythm disorders. Leadership of professional clinical societies, such as the American Heart Association and the American College of Cardiology is needed to foster such a partnership.

Study limitations are those inherent to retrospective observational registries and include incomplete data reported by participants and the possibility of residual confounding. Hospital-to-hospital variation is a source of variability that undoubtedly plays an important role in limiting the ability to predict POAF.

Conclusions

Predicting the occurrence of POAF using machine learning methods remains challenging for multiple reasons. This study of STS ACS national registry data suggests that a further expansion of multi-modal real-time data sources may improve POAF prediction. In practice, this will be challenging due to resource allocation, regulatory aspects, and operational considerations.