Validation of seven risk scores in an independent cohort: the challenge of predicting recurrence after atrial fibrillation ablation
International Journal of Arrhythmia volume 23, Article number: 29 (2022)
Several predictive scores for atrial fibrillation (AF) recurrence after AF ablation have been developed. We compared the predictive value of seven previously described risk scores ((CHADS2 and CHA2DS2-VASC, HATCH, APPLE, CAAP-AF, BASE-AF2, MB-LATER) for prediction of AF recurrence risk at 12 months after AF ablation in our patient cohort. Further, we aimed to identify additional variables to predict recurrences after AF ablation.
We used data from our digital AF ablation registry to compare the previously published scores in an independent cohort (n = 883, 50.8% with paroxysmal AF). The scores were chosen based on earlier publications and availability of relevant data.
The BASE-AF2 (AUC 0.630, p < 0.001), MB-LATER (AUC 0.612, p < 0.001), CAAP-AF (AUC 0.591, p < 0.001), APPLE (AUC 0.591, p < 0.001) and CHA2DS2-VASC (AUC 0.547, p = 0.018) scores had a statistically significant but modest predictive value for 12-month AF recurrence. None of the scores were significantly superior. Other analyzed scores had no predictive value. There was no difference in the predictive value for 12-month recurrence of AF between first procedure vs. redo procedure and RF ablation vs. cryoablation. Unlike other scores, MB-LATER showed better predictive value for paroxysmal vs. persistent AF (AUC 0.632 vs. 0.551, p = 0.038). In the multivariate logistic regression, only age (p = 0.006), number of prior electrical cardioversions (p < 0.001) and early AF recurrence (p < 0.001) were independent predictors of AF recurrence.
Despite numerous available scores, predicting recurrences after AF ablation remains challenging. New predictors are needed, potentially based on interventions, as well as novel genetic, functional and anatomic parameters.
Ablation therapy of atrial fibrillation (AF) is an established procedure to achieve rhythm control in patients with paroxysmal atrial fibrillation (PAF) or persistent atrial fibrillation (persAF). Its efficacy was shown to be superior to antiarrhythmic drug therapy, with a comparable or better safety profile [1,2,3]. Most frequently used technologies for atrial fibrillation ablation are radiofrequency current ablation (RFA) and cryoballoon ablation (Cryo). Both were shown to have similar efficacy and safety in patients with PAF  and persAF .
A continuing challenge in interventional therapy of atrial fibrillation is arrhythmia recurrence in a significant proportion of patients, with long-term recurrence estimated to be around 45–50% after one procedure and around 20% after several procedures, with significant heterogeneity [6, 7]. Both the significant risk of major complications ranging from 4.5 to 6% [8,9,10,11,12] and the high initial cost of the ablation procedure make identifying patients in whom the benefit likely outweighs the risks and costs a crucial clinical question.
Two recent studies compared AF recurrence prediction scores after AF ablation: Mulder et al. compared 10 risk scores (APPLE, ATLAS, BASE-AF2, CAAP-AF, CHADS2, CHA2DS2-VASc, DR-FLASH, HATCH, LAGO and MB-LATER) on a cohort of 482 patients undergoing AF ablation . All scores, except the HATCH score, demonstrated statistically significant but poor predictive value for recurrent AF after ablation (AUC 0.553–0.669). Jastrzębski et al. compared 6 scores (APPLE, CAAP-AF, SCALE-CryoAF, MB-LATER, CHADS2 and CHA2DS2-VASc) on a cohort of 597 patients undergoing AF balloon cryoablation and used the insights as basis for development of a simplified risk score . All scores showed significant but poor predictive value (AUC 0.551–0.624). The developed simplified predictive score based only on AF type and left atrial diameter showed a comparable predictive value, but was not validated on an independent cohort.
We sought to compare the predictive value of seven previously published scores of AF recurrence risk at 12 months after AF ablation in an independent cohort and to identify subgroups that could potentially benefit more from these risk prediction scores. Further, we aimed to identify additional variables to predict recurrences after AF ablation.
Study population and risk scores
Using data from our digital AF ablation registry, we evaluated and compared seven previously described risk scores (CHADS2 and CHA2DS2-VASC, HATCH, APPLE, CAAP-AF, BASE-AF2, MB-LATER) [15,16,17,18,19,20,21,22] for the prediction of AF recurrence after AF ablation during a 12-month follow-up (FU). Our single-center registry contains all of the consecutive patients that undergo left atrial ablation procedures in our center and runs on the online electronic data capture software REDCap® [17, 18]. Of those, only consecutive patients undergoing AF ablation procedures with either RFA or Cryo were analyzed. The scores were chosen based on earlier publications and availability of relevant data at our center. CHADS2 and CHA2DS2-VASC scores, although not initially developed for prediction of arrhythmia recurrence, showed some limited and inconsistent predictive value and have thus been used as baselines for comparison with newer scores [20,21,22]. Two of the scores (BASE-AF2 and MB-LATER) have early (blanking phase) recurrence as one of the variables and can thus only be used for prediction of long-term recurrence. This also prevents their use for pre-ablation decision making, but might still be useful for determining the intensity of post-ablation screening and potential for redo ablation.
The definitions of all scores can be found in Table 1. Score-relevant baseline data were collected. Scores calculated for all patients at the time of the ablation procedure or at 3-month follow-up were applicable (for BASE-AF2 and MB-LATER scores).
Patients undergoing de novo ablation of AF received a pulmonary vein isolation (PVI) using cryoablation or PVI including possible additional lesions at operators’ discretion using radiofrequency (RF) ablation. Ablation procedures for recurrences of AF after initial ablation included re-isolation of the pulmonary veins by RF ablation, with additional substrate modification at the operators’ discretion.
Early recurrence and recurrence after ablation were defined as any AF or atrial tachycardia episode lasting ≥ 30 s in the first 3 months after ablation and from the end of the 3-month period to 12 months after ablation, respectively. Evaluating physicians were not blinded to the predictive variables. Follow-up of patients was scheduled at regular intervals 3 and 12 months after ablation and included one or more 24-h to 72-h-holter-ECGs, occasional 12-lead ECGs and history. Outcome-relevant data from implanted cardiac devices (CIED), such as dual-chamber implantable cardioverter defibrillators (ICD) and pacemakers, or implantable loop recorders were analyzed when available.
A predefined subgroup analysis was performed in the following subgroups: first procedure vs. redo procedure, paroxysmal vs. persistent AF and RF ablation vs. cryoablation. Data collection was approved by the local ethics committee.
Continuous variables are presented as mean ± standard deviation, and categorical variables are summarized as number and percentages. The Kolmogorov–Smirnov test was used to test normality. Continuous variables were compared using unpaired Student’s t test, and categorical variables were compared using the Chi-square test. The association of variables with AF recurrence was analyzed using univariate and multivariate logistic regression analysis. The area under the ROC curve was used to test the performance of scores. The scores (as well as performance of individual scores between subgroups) were compared using DeLong’s method. A two-sided p value of < 0.05 was considered statistically significant. Analyses were performed using the SPSS software, version 20.0 (IBM Corporation, Armonk, NY, USA).
We included all consecutive patients undergoing AF ablation between January 2018 and January 2020.
Of the initial population (n = 883, 63% male, mean age 65.7 ± 11.5 years, BMI 27.5 ± 4.5 kg/m2, LA 40.9 ± 6.4 mm), 50.8% had paroxysmal AF, and 58% (n = 514) received RF ablation versus cryoablation. Additional substrate modifications performed at operators’ discretion in RF ablation patients are stated in Table 2, 3.
At discharge, 30.4% (n = 268) of pts were on class Ic or III antiarrhythmic drugs and 697 (78.9%) were on beta-blockers. Holter-ECG recordings and device interrogation were available for 46.0% and 16.3% of pts at 12-month FU. In the remaining pts, recurrence was assessed on the basis of symptoms and occasional ECGs (37.7%). Recurrence rates after 3 and 12 months were 30.1% and 38.7% (Table 2). Recurrence rates were significantly higher with holter-ECG and device interrogation methods, as compared to occasional ECGs and history only (Table 4).
ROC curve analysis of scores
For 12-month recurrence of AF, the BASE-AF2 (AUC 0.630, p < 0.001), MB-LATER (AUC 0.612, p < 0.001), CAAP-AF (AUC 0.591, p < 0.001), APPLE (AUC 0.591, p < 0.001) and CHA2DS2-VASC (AUC 0.547, p = 0.018) scores had a statistically significant but modestly predictive value. The CHADS2 (AUC 0.523, p = 0.243) and HATCH (AUC 0.507, p = 0.705) scores showed no significant predictive value (Fig. 1). For the scores with significant predictive value, none of the scores was significantly superior.
In the subgroup analysis, there was no difference in the predictive value for 12-month recurrence of AF between first procedure vs. redo procedure (p-values of 0.942, 0.596, 0.628, 0.056, 0.404, 0.414 and 0.148), and RF ablation vs. cryoablation (p-values of 0.668, 0.424, 0.442, 0.149, 0.641, 0.391 and 0.086) for CHADS2, CHA2DS2-VASC, HATCH, APPLE, CAAP-AF, BASE-AF and MB-LATER scores, while MB-LATER showed better predictive value for paroxysmal vs. persistent AF (AUC 0.632 vs. 0.551, p = 0.038). CHADS2, CHA2DS2-VASC, HATCH, APPLE, CAAP-AF and BASE-AF showed no difference (p-values of 0.855, 0.513, 0.876, 0.275, 0.912 and 0.071).
Additionally, a logistic regression analysis was performed to analyze the predictive value of available clinical parameters in our registry. In the univariate logistic regression, age (p = 0.005), glomerular filtration rate (GFR) (p = 0.034), number of prior electrical cardioversions (p < 0.001), number of prior AF ablations (p = 0.019), AF type (p = 0.006) and early (≤ 3 months) AF recurrence (p < 0.001) were associated with arrhythmia recurrence after 12 months. In the multivariate logistic regression, only age (p = 0.006), number of prior electrical cardioversions (p < 0.001) and early AF recurrence (p < 0.001) remained as independent predictors.
This study aimed to compare the predictive value of seven previously published scores for predicting AF recurrence after AF ablation on our patient population as well as to identify clinical variables predictive of AF recurrence.
We found statistically significant but modestly predictive values for the BASE-AF2, MB-LATER, CAAP-AF, APPLE and CHA2DS2-VASC scores. None of the scores showed superior predictive value for one of the tested subgroups. In the multivariate analysis, age, number of prior electrical cardioversions and early AF recurrence showed independent association with AF recurrence after 12 months. The number of prior electrical cardioversions has thus far not been included in the most commonly used predictive clinical scores, but it might be dependent on site-specific decision making and therefore not replicable as a predictive variable.
Our results are consistent with the two recent studies comparing AF recurrence risk prediction scores, which also showed only modest predictive value for all scores [13, 14]. This underscores the ongoing challenge of predicting AF recurrence after AF ablation. Balk et al. performed a systematic review of 45 studies reporting on clinical predictors of AF recurrence and concluded that no individual group of preprocedural patient characteristics except for AF type consistently and independently predict recurrence of AF after RFA . Some of the possible reasons for these inconsistencies were heterogeneity of the studied populations, ablation techniques, variable definitions and measurements as well as different definitions of AF recurrence and different screening methods. These and other non-identifiable reasons might be driving the unreliability of predictive scores for AF recurrence. Moreover, Dretzke et al. argued that c-statistics reported for development studies are often higher than those of validation studies , which is also supported by our data (Fig. 2).
Novel parameters might be necessary to improve on the predictions. Some recent studies are showing promise:
Okawa et al. tested the predictive value of response to pharmacological cardioversion before pulmonary vein isolation and showed that non-response to an antiarrhythmic drug in this setting was an independent predictor of AF recurrence after ablation (hazard ratio 1.34; 95% confidence interval 1.01–1.77; p = 0.040). This highlights the potential predictive value of medical cardioversion and possibly other interventions as tools selecting patients that are more likely to benefit from ablation.
Left atrial strain measures have been shown to be superior to left atrial size for prediction of AF recurrence after ablation [20, 21]. Hwang et al. showed that analysis of left atrial strain and strain rate using a deep convolutional neural network model showed superior prediction of AF recurrence than logistic regression models of left atrial dimension, emptying fraction, peak systolic global strain or combinations thereof .
Choe et al. evaluated a genetic risk score (GRS) based on quantifying five single nucleotide polymorphisms previously associated with atrial fibrillation in prediction of AF recurrence after ablation. Patients with a high GRS showed a significantly higher hazard ratio for AF recurrence compared to patients with low risk (HR 2.66, 95% CI 1.32–5.37) . However, other studies, including a large European study based on more than 3000 patients, showed no independent predictive value [24, 25]. Possible reasons might be population heterogeneity, small and variable effects of individual nucleotide polymorphisms, as well as different study designs and ablation practices . Advances in genetic analysis and combination with clinical predictors might yield better results in the future.
This was an observational single-center study with associated limitations. Recurrence was assessed only on the basis of symptoms and occasional ECGs in a significant proportion of the patients.
In this evaluation of AF risk scores, the BASE-AF2, MB-LATER, CAAP-AF, APPLE and CHA2DS2-VASC scores had a statistically significant but low predictive value for 12-month AF recurrence after catheter ablation. Other previously published scores had no predictive value in this large validation cohort.
Despite numerous available scores, predicting recurrences after AF ablation remains challenging. New simple and robust predictors are needed, potentially based on interventions, as well as novel genetic, functional and anatomic parameters.
Availability of data and materials
The datasets are analyzed during the current study available from the corresponding author on reasonable request.
Pappone C, Augello G, Sala S, Gugliotta F, Vicedomini G, Gulletta S, et al. A randomized trial of circumferential pulmonary vein ablation versus antiarrhythmic drug therapy in paroxysmal atrial fibrillation. J Am Coll Cardiol. 2006;48(11):2340–7.
Stabile G, Bertaglia E, Senatore G, De Simone A, Zoppo F, Donnici G, et al. Catheter ablation treatment in patients with drug-refractory atrial fibrillation: a prospective, multi-centre, randomized, controlled study (Catheter Ablation For The Cure Of Atrial Fibrillation Study)†. Eur Heart J. 2006;27(2):216–21.
Wilber DJ, Pappone C, Neuzil P, De Paola A, Marchlinski F, Natale A, et al. Comparison of antiarrhythmic drug therapy and radiofrequency catheter ablation in patients with paroxysmal atrial fibrillation: a randomized controlled Trial. JAMA. 2010;303(4):333.
Kuck K-H, Brugada J, Fürnkranz A, Metzner A, Ouyang F, Chun KRJ, et al. Cryoballoon or Radiofrequency Ablation for Paroxysmal Atrial Fibrillation. N Engl J Med. 2016;374(23):2235–45.
Shi LB, Rossvoll O, Tande P, Schuster P, Solheim E, Chen J. Cryoballoon vs. radiofrequency catheter ablation: insights from NOrwegian randomized study of PERSistent Atrial Fibrillation (NO-PERSAF study). EP Europace. 2022;24(2):226–33.
Ganesan AN, Shipp NJ, Brooks AG, Kuklik P, Lau DH, Lim HS, et al. Long-term outcomes of catheter ablation of atrial fibrillation: a systematic review and meta-analysis. JAHA. 2013. https://doi.org/10.1161/JAHA.112.004549.
Sultan A, Lüker J, Andresen D, Kuck KH, Hoffmann E, Brachmann J, et al. Predictors of atrial fibrillation recurrence after catheter ablation: data from the German Ablation Registry. Sci Rep. 2017;7(1):16678.
Mujović N, Marinković M, Lenarczyk R, Tilz R, Potpara TS. Catheter ablation of atrial fibrillation: an overview for clinicians. Adv Ther. 2017;34(8):1897–917.
Baman TS, Jongnarangsin K, Chugh A, Suwanagool A, Guiot A, Madenci A, et al. Prevalence and predictors of complications of radiofrequency catheter ablation for atrial fibrillation. J Cardiovasc Electrophysiol. 2011;22(6):626–31.
Hoffmann BA, Kuck K-H, Andresen D, Spitzer SG, Hoffmann E, Schumacher B, et al. Impact of structural heart disease on the acute complication rate in atrial fibrillation ablation: results from the German Ablation Registry. J Cardiovasc Electrophysiol. 2014;25(3):242–9.
Aldhoon B, Wichterle D, Peichl P, Čihák R, Kautzner J. Complications of catheter ablation for atrial fibrillation in a high-volume centre with the use of intracardiac echocardiography. Europace. 2013;15(1):24–32.
Michowitz Y, Rahkovich M, Oral H, Zado ES, Tilz R, John S, et al. Effects of sex on the incidence of cardiac tamponade after catheter ablation of atrial fibrillation: results from a worldwide survey in 34 943 atrial fibrillation ablation procedures. Circ Arrhythm Electrophysiol. 2014;7(2):274–80.
Mulder MJ, Kemme MJB, Hopman LHGA, Kuşgözoğlu E, Gülçiçek H, van de Ven PM, et al. Comparison of the predictive value of ten risk scores for outcomes of atrial fibrillation patients undergoing radiofrequency pulmonary vein isolation. Int J Cardiol. 2021;344:103–10.
Jastrzębski M, Kiełbasa G, Fijorek K, Bednarski A, Kusiak A, Sondej T, et al. Comparison of six risk scores for the prediction of atrial fibrillation recurrence after cryoballoon-based ablation and development of a simplified method, the 0-1-2 PL score. J Arrhythmia. 2021;37(4):956–64.
Balk EM, Garlitski AC, Alsheikh-Ali AA, Terasawa T, Chung M, Ip S. Predictors of atrial fibrillation recurrence after radiofrequency catheter ablation: a systematic review. J Cardiovasc Electrophysiol. 2010;21(11):1208–16. https://doi.org/10.1111/j.1540-8167.2010.01798.x.
Dretzke J, Chuchu N, Agarwal R, Herd C, Chua W, Fabritz L, et al. Predicting recurrent atrial fibrillation after catheter ablation: a systematic review of prognostic models. Europace. 2020;22(5):748–60. https://doi.org/10.1093/europace/euaa041.
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.
Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, et al. The REDCap consortium: Building an international community of software partners. J Biomed Inform. 2019. https://doi.org/10.1016/j.jbi.2019.103208.
Keisuke O, Shohei H, Takeshi M, Ryu T, Yuya S, Masahiro S, et al. Effect of preprocedural pharmacologic cardioversion on pulmonary vein isolation in patients with persistent atrial fibrillation. Heart Rhythm. 2021. https://doi.org/10.1016/j.hrthm.2021.04.027.
Hammerstingl C, Schwekendiek M, Momcilovic D, Schueler R, Sinning JM, Schrickel JW, et al. Left atrial deformation imaging with ultrasound based two-dimensional speckle-tracking predicts the rate of recurrence of paroxysmal and persistent atrial fibrillation after successful ablation procedures. J Cardiovasc Electrophysiol. 2012;23:247–55. https://doi.org/10.1111/j.1540-8167.2011.02177.x.
Montserrat S, Gabrielli L, Bijnens B, Borràs R, Berruezo A, Poyatos S, et al. Left atrial deformation predicts success of first and second percutaneous atrial fibrillation ablation. Heart Rhythm. 2015;12:11–8. https://doi.org/10.1016/j.hrthm.2014.08.032.
Hwang Y-T, Lee H-L, Lu C-H, Chang P-C, Wo H-T, Liu H-T, et al. A novel approach for predicting atrial fibrillation recurrence after ablation using deep convolutional neural networks by assessing left atrial curved M-mode speckle-tracking images. Front Cardiovasc Med. 2021. https://doi.org/10.3389/fcvm.2020.605642/full.
Choe WS, Kang JH, Choi EK, Shin SY, Lubitz SA, Ellinor PT, et al. A genetic risk score for atrial fibrillation predicts the response to catheter ablation. Korean Circ J. 2019;49(4):338–49. https://doi.org/10.4070/kcj.2018.0161.
Shoemaker MB, Husser D, Roselli C, Al Jazairi M, Chrispin J, Kühne M, et al. Genetic susceptibility for atrial fibrillation in patients undergoing atrial fibrillation ablation. Circ Arrhythm Electrophysiol. 2020. https://doi.org/10.1161/Circep.119.007676.
Choi E, Park JH, Lee J, Nam CM, Hwang MK, Uhm J, et al. Korean atrial fibrillation (AF) network: genetic variants for AF Do not predict ablation success. JAHA. 2015. https://doi.org/10.1161/JAHA.115.002046.
The study had no external funding.
Ethics approval and consent to participate
The pseudonymized data collection and analysis was approved by the Ethics Committee of the University of Cologne Medical School. No formal consent was necessary.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Karlo, F., Daniel, S., Arian, S. et al. Validation of seven risk scores in an independent cohort: the challenge of predicting recurrence after atrial fibrillation ablation. Int J Arrhythm 23, 29 (2022). https://doi.org/10.1186/s42444-022-00080-0