Skip to main content

Validation of seven risk scores in an independent cohort: the challenge of predicting recurrence after atrial fibrillation ablation

Abstract

Purpose

Several predictive scores for atrial fibrillation (AF) recurrence after AF ablation have been developed. We compared the predictive value of seven previously described risk scores ((CHADS2 and CHA2DS2-VASC, HATCH, APPLE, CAAP-AF, BASE-AF2, MB-LATER) for prediction of AF recurrence risk at 12 months after AF ablation in our patient cohort. Further, we aimed to identify additional variables to predict recurrences after AF ablation.

Methods

We used data from our digital AF ablation registry to compare the previously published scores in an independent cohort (n = 883, 50.8% with paroxysmal AF). The scores were chosen based on earlier publications and availability of relevant data.

Results

The BASE-AF2 (AUC 0.630, p < 0.001), MB-LATER (AUC 0.612, p < 0.001), CAAP-AF (AUC 0.591, p < 0.001), APPLE (AUC 0.591, p < 0.001) and CHA2DS2-VASC (AUC 0.547, p = 0.018) scores had a statistically significant but modest predictive value for 12-month AF recurrence. None of the scores were significantly superior. Other analyzed scores had no predictive value. There was no difference in the predictive value for 12-month recurrence of AF between first procedure vs. redo procedure and RF ablation vs. cryoablation. Unlike other scores, MB-LATER showed better predictive value for paroxysmal vs. persistent AF (AUC 0.632 vs. 0.551, p = 0.038). In the multivariate logistic regression, only age (p = 0.006), number of prior electrical cardioversions (p < 0.001) and early AF recurrence (p < 0.001) were independent predictors of AF recurrence.

Conclusion

Despite numerous available scores, predicting recurrences after AF ablation remains challenging. New predictors are needed, potentially based on interventions, as well as novel genetic, functional and anatomic parameters.

Background

Ablation therapy of atrial fibrillation (AF) is an established procedure to achieve rhythm control in patients with paroxysmal atrial fibrillation (PAF) or persistent atrial fibrillation (persAF). Its efficacy was shown to be superior to antiarrhythmic drug therapy, with a comparable or better safety profile [1,2,3]. Most frequently used technologies for atrial fibrillation ablation are radiofrequency current ablation (RFA) and cryoballoon ablation (Cryo). Both were shown to have similar efficacy and safety in patients with PAF [4] and persAF [5].

A continuing challenge in interventional therapy of atrial fibrillation is arrhythmia recurrence in a significant proportion of patients, with long-term recurrence estimated to be around 45–50% after one procedure and around 20% after several procedures, with significant heterogeneity [6, 7]. Both the significant risk of major complications ranging from 4.5 to 6% [8,9,10,11,12] and the high initial cost of the ablation procedure make identifying patients in whom the benefit likely outweighs the risks and costs a crucial clinical question.

Several predictive scores for AF recurrence after ablation have been developed. The predictive value of such scores has thus far been limited [13, 14].

Two recent studies compared AF recurrence prediction scores after AF ablation: Mulder et al. compared 10 risk scores (APPLE, ATLAS, BASE-AF2, CAAP-AF, CHADS2, CHA2DS2-VASc, DR-FLASH, HATCH, LAGO and MB-LATER) on a cohort of 482 patients undergoing AF ablation [13]. All scores, except the HATCH score, demonstrated statistically significant but poor predictive value for recurrent AF after ablation (AUC 0.553–0.669). Jastrzębski et al. compared 6 scores (APPLE, CAAP-AF, SCALE-CryoAF, MB-LATER, CHADS2 and CHA2DS2-VASc) on a cohort of 597 patients undergoing AF balloon cryoablation and used the insights as basis for development of a simplified risk score [14]. All scores showed significant but poor predictive value (AUC 0.551–0.624). The developed simplified predictive score based only on AF type and left atrial diameter showed a comparable predictive value, but was not validated on an independent cohort.

Aims

We sought to compare the predictive value of seven previously published scores of AF recurrence risk at 12 months after AF ablation in an independent cohort and to identify subgroups that could potentially benefit more from these risk prediction scores. Further, we aimed to identify additional variables to predict recurrences after AF ablation.

Methods

Study population and risk scores

Using data from our digital AF ablation registry, we evaluated and compared seven previously described risk scores (CHADS2 and CHA2DS2-VASC, HATCH, APPLE, CAAP-AF, BASE-AF2, MB-LATER) [15,16,17,18,19,20,21,22] for the prediction of AF recurrence after AF ablation during a 12-month follow-up (FU). Our single-center registry contains all of the consecutive patients that undergo left atrial ablation procedures in our center and runs on the online electronic data capture software REDCap® [17, 18]. Of those, only consecutive patients undergoing AF ablation procedures with either RFA or Cryo were analyzed. The scores were chosen based on earlier publications and availability of relevant data at our center. CHADS2 and CHA2DS2-VASC scores, although not initially developed for prediction of arrhythmia recurrence, showed some limited and inconsistent predictive value and have thus been used as baselines for comparison with newer scores [20,21,22]. Two of the scores (BASE-AF2 and MB-LATER) have early (blanking phase) recurrence as one of the variables and can thus only be used for prediction of long-term recurrence. This also prevents their use for pre-ablation decision making, but might still be useful for determining the intensity of post-ablation screening and potential for redo ablation.

The definitions of all scores can be found in Table 1. Score-relevant baseline data were collected. Scores calculated for all patients at the time of the ablation procedure or at 3-month follow-up were applicable (for BASE-AF2 and MB-LATER scores).

Table 1 Point calculation of the analyzed scores

Patients undergoing de novo ablation of AF received a pulmonary vein isolation (PVI) using cryoablation or PVI including possible additional lesions at operators’ discretion using radiofrequency (RF) ablation. Ablation procedures for recurrences of AF after initial ablation included re-isolation of the pulmonary veins by RF ablation, with additional substrate modification at the operators’ discretion.

Early recurrence and recurrence after ablation were defined as any AF or atrial tachycardia episode lasting ≥ 30 s in the first 3 months after ablation and from the end of the 3-month period to 12 months after ablation, respectively. Evaluating physicians were not blinded to the predictive variables. Follow-up of patients was scheduled at regular intervals 3 and 12 months after ablation and included one or more 24-h to 72-h-holter-ECGs, occasional 12-lead ECGs and history. Outcome-relevant data from implanted cardiac devices (CIED), such as dual-chamber implantable cardioverter defibrillators (ICD) and pacemakers, or implantable loop recorders were analyzed when available.

Table 2 Baseline characteristics

A predefined subgroup analysis was performed in the following subgroups: first procedure vs. redo procedure, paroxysmal vs. persistent AF and RF ablation vs. cryoablation. Data collection was approved by the local ethics committee.

Statistical analysis

Continuous variables are presented as mean ± standard deviation, and categorical variables are summarized as number and percentages. The Kolmogorov–Smirnov test was used to test normality. Continuous variables were compared using unpaired Student’s t test, and categorical variables were compared using the Chi-square test. The association of variables with AF recurrence was analyzed using univariate and multivariate logistic regression analysis. The area under the ROC curve was used to test the performance of scores. The scores (as well as performance of individual scores between subgroups) were compared using DeLong’s method. A two-sided p value of < 0.05 was considered statistically significant. Analyses were performed using the SPSS software, version 20.0 (IBM Corporation, Armonk, NY, USA).

Results

We included all consecutive patients undergoing AF ablation between January 2018 and January 2020.

Baseline characteristics

Of the initial population (n = 883, 63% male, mean age 65.7 ± 11.5 years, BMI 27.5 ± 4.5 kg/m2, LA 40.9 ± 6.4 mm), 50.8% had paroxysmal AF, and 58% (n = 514) received RF ablation versus cryoablation. Additional substrate modifications performed at operators’ discretion in RF ablation patients are stated in Table 2, 3.

Table 3 Lesion sets additional to (Re-)PVI in RF patients

At discharge, 30.4% (n = 268) of pts were on class Ic or III antiarrhythmic drugs and 697 (78.9%) were on beta-blockers. Holter-ECG recordings and device interrogation were available for 46.0% and 16.3% of pts at 12-month FU. In the remaining pts, recurrence was assessed on the basis of symptoms and occasional ECGs (37.7%). Recurrence rates after 3 and 12 months were 30.1% and 38.7% (Table 2). Recurrence rates were significantly higher with holter-ECG and device interrogation methods, as compared to occasional ECGs and history only (Table 4).

Table 4 Recurrence rate 12 months after ablation according to monitoring method

ROC curve analysis of scores

For 12-month recurrence of AF, the BASE-AF2 (AUC 0.630, p < 0.001), MB-LATER (AUC 0.612, p < 0.001), CAAP-AF (AUC 0.591, p < 0.001), APPLE (AUC 0.591, p < 0.001) and CHA2DS2-VASC (AUC 0.547, p = 0.018) scores had a statistically significant but modestly predictive value. The CHADS2 (AUC 0.523, p = 0.243) and HATCH (AUC 0.507, p = 0.705) scores showed no significant predictive value (Fig. 1). For the scores with significant predictive value, none of the scores was significantly superior.

Fig. 1
figure 1

ROC curves of different scores for prediction of 12-month arrhythmia recurrence

In the subgroup analysis, there was no difference in the predictive value for 12-month recurrence of AF between first procedure vs. redo procedure (p-values of 0.942, 0.596, 0.628, 0.056, 0.404, 0.414 and 0.148), and RF ablation vs. cryoablation (p-values of 0.668, 0.424, 0.442, 0.149, 0.641, 0.391 and 0.086) for CHADS2, CHA2DS2-VASC, HATCH, APPLE, CAAP-AF, BASE-AF and MB-LATER scores, while MB-LATER showed better predictive value for paroxysmal vs. persistent AF (AUC 0.632 vs. 0.551, p = 0.038). CHADS2, CHA2DS2-VASC, HATCH, APPLE, CAAP-AF and BASE-AF showed no difference (p-values of 0.855, 0.513, 0.876, 0.275, 0.912 and 0.071).

Additionally, a logistic regression analysis was performed to analyze the predictive value of available clinical parameters in our registry. In the univariate logistic regression, age (p = 0.005), glomerular filtration rate (GFR) (p = 0.034), number of prior electrical cardioversions (p < 0.001), number of prior AF ablations (p = 0.019), AF type (p = 0.006) and early (≤ 3 months) AF recurrence (p < 0.001) were associated with arrhythmia recurrence after 12 months. In the multivariate logistic regression, only age (p = 0.006), number of prior electrical cardioversions (p < 0.001) and early AF recurrence (p < 0.001) remained as independent predictors.

Discussion

This study aimed to compare the predictive value of seven previously published scores for predicting AF recurrence after AF ablation on our patient population as well as to identify clinical variables predictive of AF recurrence.

We found statistically significant but modestly predictive values for the BASE-AF2, MB-LATER, CAAP-AF, APPLE and CHA2DS2-VASC scores. None of the scores showed superior predictive value for one of the tested subgroups. In the multivariate analysis, age, number of prior electrical cardioversions and early AF recurrence showed independent association with AF recurrence after 12 months. The number of prior electrical cardioversions has thus far not been included in the most commonly used predictive clinical scores, but it might be dependent on site-specific decision making and therefore not replicable as a predictive variable.

Our results are consistent with the two recent studies comparing AF recurrence risk prediction scores, which also showed only modest predictive value for all scores [13, 14]. This underscores the ongoing challenge of predicting AF recurrence after AF ablation. Balk et al. performed a systematic review of 45 studies reporting on clinical predictors of AF recurrence and concluded that no individual group of preprocedural patient characteristics except for AF type consistently and independently predict recurrence of AF after RFA [15]. Some of the possible reasons for these inconsistencies were heterogeneity of the studied populations, ablation techniques, variable definitions and measurements as well as different definitions of AF recurrence and different screening methods. These and other non-identifiable reasons might be driving the unreliability of predictive scores for AF recurrence. Moreover, Dretzke et al. argued that c-statistics reported for development studies are often higher than those of validation studies [16], which is also supported by our data (Fig. 2).

Fig. 2
figure 2

Comparison of c-statistics (including confidence intervals when available) between development studies (DS) and present analysis (OP)

Novel parameters might be necessary to improve on the predictions. Some recent studies are showing promise:

Okawa et al. tested the predictive value of response to pharmacological cardioversion before pulmonary vein isolation and showed that non-response to an antiarrhythmic drug in this setting was an independent predictor of AF recurrence after ablation (hazard ratio 1.34; 95% confidence interval 1.01–1.77; p = 0.040). This highlights the potential predictive value of medical cardioversion and possibly other interventions as tools selecting patients that are more likely to benefit from ablation.

Left atrial strain measures have been shown to be superior to left atrial size for prediction of AF recurrence after ablation [20, 21]. Hwang et al. showed that analysis of left atrial strain and strain rate using a deep convolutional neural network model showed superior prediction of AF recurrence than logistic regression models of left atrial dimension, emptying fraction, peak systolic global strain or combinations thereof [22].

Choe et al. evaluated a genetic risk score (GRS) based on quantifying five single nucleotide polymorphisms previously associated with atrial fibrillation in prediction of AF recurrence after ablation. Patients with a high GRS showed a significantly higher hazard ratio for AF recurrence compared to patients with low risk (HR 2.66, 95% CI 1.32–5.37) [23]. However, other studies, including a large European study based on more than 3000 patients, showed no independent predictive value [24, 25]. Possible reasons might be population heterogeneity, small and variable effects of individual nucleotide polymorphisms, as well as different study designs and ablation practices [25]. Advances in genetic analysis and combination with clinical predictors might yield better results in the future.

Study limitations

This was an observational single-center study with associated limitations. Recurrence was assessed only on the basis of symptoms and occasional ECGs in a significant proportion of the patients.

Conclusion

In this evaluation of AF risk scores, the BASE-AF2, MB-LATER, CAAP-AF, APPLE and CHA2DS2-VASC scores had a statistically significant but low predictive value for 12-month AF recurrence after catheter ablation. Other previously published scores had no predictive value in this large validation cohort.

Despite numerous available scores, predicting recurrences after AF ablation remains challenging. New simple and robust predictors are needed, potentially based on interventions, as well as novel genetic, functional and anatomic parameters.

Availability of data and materials

The datasets are analyzed during the current study available from the corresponding author on reasonable request.

References

  1. Pappone C, Augello G, Sala S, Gugliotta F, Vicedomini G, Gulletta S, et al. A randomized trial of circumferential pulmonary vein ablation versus antiarrhythmic drug therapy in paroxysmal atrial fibrillation. J Am Coll Cardiol. 2006;48(11):2340–7.

    Article  CAS  Google Scholar 

  2. Stabile G, Bertaglia E, Senatore G, De Simone A, Zoppo F, Donnici G, et al. Catheter ablation treatment in patients with drug-refractory atrial fibrillation: a prospective, multi-centre, randomized, controlled study (Catheter Ablation For The Cure Of Atrial Fibrillation Study)†. Eur Heart J. 2006;27(2):216–21.

    Article  Google Scholar 

  3. Wilber DJ, Pappone C, Neuzil P, De Paola A, Marchlinski F, Natale A, et al. Comparison of antiarrhythmic drug therapy and radiofrequency catheter ablation in patients with paroxysmal atrial fibrillation: a randomized controlled Trial. JAMA. 2010;303(4):333.

    Article  CAS  Google Scholar 

  4. Kuck K-H, Brugada J, Fürnkranz A, Metzner A, Ouyang F, Chun KRJ, et al. Cryoballoon or Radiofrequency Ablation for Paroxysmal Atrial Fibrillation. N Engl J Med. 2016;374(23):2235–45.

    Article  Google Scholar 

  5. Shi LB, Rossvoll O, Tande P, Schuster P, Solheim E, Chen J. Cryoballoon vs. radiofrequency catheter ablation: insights from NOrwegian randomized study of PERSistent Atrial Fibrillation (NO-PERSAF study). EP Europace. 2022;24(2):226–33.

    Article  Google Scholar 

  6. Ganesan AN, Shipp NJ, Brooks AG, Kuklik P, Lau DH, Lim HS, et al. Long-term outcomes of catheter ablation of atrial fibrillation: a systematic review and meta-analysis. JAHA. 2013. https://doi.org/10.1161/JAHA.112.004549.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Sultan A, Lüker J, Andresen D, Kuck KH, Hoffmann E, Brachmann J, et al. Predictors of atrial fibrillation recurrence after catheter ablation: data from the German Ablation Registry. Sci Rep. 2017;7(1):16678.

    Article  CAS  Google Scholar 

  8. Mujović N, Marinković M, Lenarczyk R, Tilz R, Potpara TS. Catheter ablation of atrial fibrillation: an overview for clinicians. Adv Ther. 2017;34(8):1897–917.

    Article  Google Scholar 

  9. Baman TS, Jongnarangsin K, Chugh A, Suwanagool A, Guiot A, Madenci A, et al. Prevalence and predictors of complications of radiofrequency catheter ablation for atrial fibrillation. J Cardiovasc Electrophysiol. 2011;22(6):626–31.

    Article  Google Scholar 

  10. Hoffmann BA, Kuck K-H, Andresen D, Spitzer SG, Hoffmann E, Schumacher B, et al. Impact of structural heart disease on the acute complication rate in atrial fibrillation ablation: results from the German Ablation Registry. J Cardiovasc Electrophysiol. 2014;25(3):242–9.

    Article  Google Scholar 

  11. Aldhoon B, Wichterle D, Peichl P, Čihák R, Kautzner J. Complications of catheter ablation for atrial fibrillation in a high-volume centre with the use of intracardiac echocardiography. Europace. 2013;15(1):24–32.

    Article  Google Scholar 

  12. Michowitz Y, Rahkovich M, Oral H, Zado ES, Tilz R, John S, et al. Effects of sex on the incidence of cardiac tamponade after catheter ablation of atrial fibrillation: results from a worldwide survey in 34 943 atrial fibrillation ablation procedures. Circ Arrhythm Electrophysiol. 2014;7(2):274–80.

    Article  Google Scholar 

  13. Mulder MJ, Kemme MJB, Hopman LHGA, Kuşgözoğlu E, Gülçiçek H, van de Ven PM, et al. Comparison of the predictive value of ten risk scores for outcomes of atrial fibrillation patients undergoing radiofrequency pulmonary vein isolation. Int J Cardiol. 2021;344:103–10.

    Article  Google Scholar 

  14. Jastrzębski M, Kiełbasa G, Fijorek K, Bednarski A, Kusiak A, Sondej T, et al. Comparison of six risk scores for the prediction of atrial fibrillation recurrence after cryoballoon-based ablation and development of a simplified method, the 0-1-2 PL score. J Arrhythmia. 2021;37(4):956–64.

    Article  Google Scholar 

  15. Balk EM, Garlitski AC, Alsheikh-Ali AA, Terasawa T, Chung M, Ip S. Predictors of atrial fibrillation recurrence after radiofrequency catheter ablation: a systematic review. J Cardiovasc Electrophysiol. 2010;21(11):1208–16. https://doi.org/10.1111/j.1540-8167.2010.01798.x.

    Article  PubMed  Google Scholar 

  16. Dretzke J, Chuchu N, Agarwal R, Herd C, Chua W, Fabritz L, et al. Predicting recurrent atrial fibrillation after catheter ablation: a systematic review of prognostic models. Europace. 2020;22(5):748–60. https://doi.org/10.1093/europace/euaa041.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.

    Article  Google Scholar 

  18. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, et al. The REDCap consortium: Building an international community of software partners. J Biomed Inform. 2019. https://doi.org/10.1016/j.jbi.2019.103208.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Keisuke O, Shohei H, Takeshi M, Ryu T, Yuya S, Masahiro S, et al. Effect of preprocedural pharmacologic cardioversion on pulmonary vein isolation in patients with persistent atrial fibrillation. Heart Rhythm. 2021. https://doi.org/10.1016/j.hrthm.2021.04.027.

    Article  Google Scholar 

  20. Hammerstingl C, Schwekendiek M, Momcilovic D, Schueler R, Sinning JM, Schrickel JW, et al. Left atrial deformation imaging with ultrasound based two-dimensional speckle-tracking predicts the rate of recurrence of paroxysmal and persistent atrial fibrillation after successful ablation procedures. J Cardiovasc Electrophysiol. 2012;23:247–55. https://doi.org/10.1111/j.1540-8167.2011.02177.x.

    Article  PubMed  Google Scholar 

  21. Montserrat S, Gabrielli L, Bijnens B, Borràs R, Berruezo A, Poyatos S, et al. Left atrial deformation predicts success of first and second percutaneous atrial fibrillation ablation. Heart Rhythm. 2015;12:11–8. https://doi.org/10.1016/j.hrthm.2014.08.032.

    Article  PubMed  Google Scholar 

  22. Hwang Y-T, Lee H-L, Lu C-H, Chang P-C, Wo H-T, Liu H-T, et al. A novel approach for predicting atrial fibrillation recurrence after ablation using deep convolutional neural networks by assessing left atrial curved M-mode speckle-tracking images. Front Cardiovasc Med. 2021. https://doi.org/10.3389/fcvm.2020.605642/full.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Choe WS, Kang JH, Choi EK, Shin SY, Lubitz SA, Ellinor PT, et al. A genetic risk score for atrial fibrillation predicts the response to catheter ablation. Korean Circ J. 2019;49(4):338–49. https://doi.org/10.4070/kcj.2018.0161.

    Article  CAS  PubMed  Google Scholar 

  24. Shoemaker MB, Husser D, Roselli C, Al Jazairi M, Chrispin J, Kühne M, et al. Genetic susceptibility for atrial fibrillation in patients undergoing atrial fibrillation ablation. Circ Arrhythm Electrophysiol. 2020. https://doi.org/10.1161/Circep.119.007676.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Choi E, Park JH, Lee J, Nam CM, Hwang MK, Uhm J, et al. Korean atrial fibrillation (AF) network: genetic variants for AF Do not predict ablation success. JAHA. 2015. https://doi.org/10.1161/JAHA.115.002046.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

The study had no external funding.

Author information

Authors and Affiliations

Authors

Contributions

FK helped in study concept, data analysis, writing of the manuscript. SD and SA were involved in study concept and revision. vdBJ-H, WJ, SC and DS contributed to data acquisition, revision. LJ helped in study concept, data analysis, revision. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Filipovic Karlo.

Ethics declarations

Ethics approval and consent to participate

The pseudonymized data collection and analysis was approved by the Ethics Committee of the University of Cologne Medical School. No formal consent was necessary.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Karlo, F., Daniel, S., Arian, S. et al. Validation of seven risk scores in an independent cohort: the challenge of predicting recurrence after atrial fibrillation ablation. Int J Arrhythm 23, 29 (2022). https://doi.org/10.1186/s42444-022-00080-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s42444-022-00080-0