Comparison of the Performance of Five Different Scoring Systems in Patients with ST-Elevation Myocardial Infarction

Flora Özkalaycı Kaçar1, Mehmet Saygı1, Tanıl Özer2, Ali Karagöz2, İbrahim Halil Tanboğa3, Vecih Oduncu4

1Clinic of Cardiology, Hisar Intercontinental Hospital, İstanbul, Türkiye
2Clinic of Cardiology, Kartal Koşuyolu Training and Research Hospital, İstanbul, Türkiye
3Department of Biostatistics and Cardiology, Nişantaşı University Faculty of Medicine, İstanbul, Türkiye
4Department of Cardiology, Bahçeşehir University Faculty of Medicine, İstanbul, Türkiye

Keywords: ST-elevation myocardial infarction; mortality; risk score

Abstract

Introduction: Although most of the scoring systems are used for long-term mortality assessment in STelevation myocardial infarction (STEMI), there is still lack of data comparing model performances. In this study, it was aimed to compare five scoring systems for predicting long-term mortality in patients presented with STEMI.

Patients and Methods: This is a retrospective observational study consisting of 1689 consecutive STEMI patients who underwent PCI between 2009 and 2013. Patient data was obtained from the electronic data base of the hospital. Each patients’ mortality risk was assessed with five different risk scores and recorded.

Results: A total of 1689 patients with STEMI were included into the study. Median follow-up time was one year. Risk scores were calculated for each patient. Although similar statistical significance was presented among all scores, modified age, creatinine clearance, and ejection fraction score (mACEF) were demonstrated to be more significant than relevant scoring systems in clinical respect.

Conclusion: Among five scores, the mACEF score was demonstrated to be the most significant model in clinical respect for the prediction of mortality.

Introduction

Coronary artery disease (CAD) is a well-known cause of mortality worldwide(1). In this context, short- and long-term management of STEMI is extremely important.

Guidelines are provided in terms of managing these patients(2). Despite all appropriate treatment strategies, mortality risk remains in the long-term follow-up.

Several risk scoring systems have been developed to predict short- and long-term mortality risk in patients with STEMI. It is vital to apply appropriate treatment modalities to patients presented with acute coronary syndromes. Thrombolysis in myocardial infarction (TIMI) score was developed for the assessment of risk of death and ischemic events in patients with non-STEMI and unstable angina. Likewise, TIMI risk index (TRI) has been developed to predict the risk of mortality at 30 days in patients treated with fibrinolytics and later validated for STEMI patients undergoing pPCI for predicting one-year mortality(3-5).

The primary angioplasty in myocardial infarction (PAMI) risk score has been developed for predicting mortality in sixmonths in patients underwent PCI. Likewise, the controlled abciximab and device investigation to lower late angioplasty complications (CADILLAC) risk score has originally been developed for the prediction of one-year mortality in patients undergoing invasive procedures(6,7). Thrombolysis in myocardial infarction risk index has been developed as a simple tool for predicting mortality over 30 days using age, heart rate, and systolic blood pressure (SBP)(8). Modified age, creatinine clearance, and ejection fraction score (mACEF) are calculated in patients undergoing percutaneous coronary intervention(9). Some investigators have already tested these risk scores for long-term follow-up. The comparison among some risks scores have been done by several study groups(10-12). Although some investigators have tested these risk scores to predict long-term mortality in patients with STEMI, there is still lack of evidence on their usefulness and in long-term follow-up, and moreover, we still do not have enough information about the superiority of these scores to each other. Therefore, herewith we tried to compare five different risk scores regarding to their predictive value for long-term all-cause mortality in patients with STEMI underwent PCI.

Materials and Methods

Study Design and Patients Selection

One thousand seven hundred and eighty (1780) patients had been admitted to our centre with STEMI and had undergone PCI between 2009-2013. Patients whose age was over 80 years (n= 55), patients with known or recently diagnosed cancer (n= 22); patients with inflammatory or connective-tissue disease (n= 10); and patients who died during PCI (n= 4) were excluded. Overall, 1689 patients were retrospectively enrolled to the study (Figure 1). The patients were followed up to 365 days. Medical data was obtained from electronic health records of the hospital. Each patient’s mortality risk was assessed with five different risk scores and recorded. Risk scores were compared to each other. The study protocol complied with the Declaration of Helsinki.

Figure 1. Study flow chart.

Definitions

STEMI was defined as typical chest pain accompanied by persistent ST-elevation at least two contiguous derivations or a new onset left bundle branch block (LBBB)(13).

Blood tests were obtained in all patients on admission or in the first 24 hours. Heart failure was assessed according to Killip classification(14). A transthoracic echocardiography was performed by a Vivid 3 instrument (GE; Horten, Norway). Left ventricular ejection fraction (LVEF) was measured using modified Simpson method. Coronary angiographies were performed by Siemens Artis interventional angiography system(15). Anaemia was haemoglobin (Hb) levels 12.0 g/dL in females and 13.0 g/dL in males(16). The Cockcroft-Gault formula was used for estimating creatinine clearance in all patients(17). All patients were treated according to relevant guidelines(18). TIMI, PAMI CADILLAC, TIMI risk index, mACEF scores were calculated in all patients. Age, presence of diabetes mellitus (DM) or hypertension (HT) or anginal complaints, systolic blood pressure (SBP), heart rate, Killip class, weight, anterior myocardial infarction (MI)/new onset LBBB and ischemic time were used to calculate the TIMI score(3,5). Age, Killip class, heart rate, presence of DM, anterior MI/new onset LBBB were used to calculate PAMI risk score(6). CADILLAC risk score was calculated using LVEF, Killip class, renal failure, post TIMI flow, age, presence of anaemia and presence of three vessel disease. For relevant risk score, numerical points were assigned to each risk factor and therefore, the risk was calculated(7). TRI was calculated using the following formula: [HR x (age/10)2 ]/SBP. mACEF was calculated using the following formula: age/LVEF (%) + 1 point (for every 10 mL/min reduction in creatinine clearance below 60 mL/min/1.73 m2 ) (up to a maximum of six points)(8,9).

Statistics

Percent and n were used for categorical variables. Median and interquartile range were used for numerical variables. The primary outcome was time to all-cause mortality. MannWhitney U test was used to compare continuous variables, and the χ2 test or Fisher’s exact test was used for categorical variables. All statistical analysis was carried out using R-software v. 3.5.1 (R statistical software, Institute for Statistics and Mathematics, Vienna, Austria). A p value of <0.05 was considered statistically significant.

Results

Among these patients, 821 of them presented with anterior MI, 831 with inferior MI and 37 of them with posterior MI. Median age of the patients was 56 (IQR= 48-65). Overall, 80% of the patients were males. About 40% of the patients had HT, 22.7% had DM. More than half (55.1%) of the patients were smokers. Overall, 4.6% (n= 79) of the patients had cardiogenic shock. Clinical follow-up after PCI performed for a median of 365 days (IQR= 361-365). The number who died of all-cause mortality was 180. Patients’ basal characteristics are presented in Table 1. Apart from high density lipoprotein (HDL) and platelet level, all laboratory and clinical values were statistically significantly different between the two groups who died of all-cause death and survived during the follow-up period. Those who died had a mean follow-up of 39.5 days (IQR= 3-192). According to multivariable Cox regression analyses, the mACEF score was the strongest predictor for all-cause death in patients with STEMI. Among all of these risk scores, the mACEF score had the highest predictive accuracy with a c-index of 0.850 and a likelihood ratio-X2 of 324. Likewise, among all other risk scores, mACEF score was the best model to explain the variance with an adjusted R2 of 0.227 (Table 2). In multivariable cox regression analyses, hazard ratios (HR) of the risk scores are demonstrated in Table 3. The CADILLAC risk score has the highest HR with a value of 13.0 (95% CI, 6.29-26.8) among the other risk scores, yet the confidence interval is quite wide. On the contrary, TRI had the lowest HR with a value of 2.32 (95% CI, 1.36-3.66). Both risk models have the same statistical significance in predicting one-year all-cause mortality (Table 3). Adjusted hazard ratio plots for relevant scoring systems are presented in Figure 2. Besides, AUCROC comparison of the models are presented in Figure 3.

Table 1. Comparison of baseline characteristics between the survivors and dead patients

Table 2. Multivariable cox regression analyses for different risk scoring systems

Table 3. Multivariable Cox proportional regression for all-cause-death

Figure 2. Adjusted hazard ratio plot.

Figure 3. AUCROC comparison of the models.

Discussion

Among five scores, the mACEF score was demonstrated to be the strongest model in terms of predicting one-year allcause death. Although the other four scores were demonstrated to have similar statistically significant predictive value, the performance of the relevant risk scores may alter in clinical respect.

Despite the advances in medical and invasive therapy, coronary artery disease remains to be the most common cause of death worldwide(19).

Specifically, mortality rates differ in patients with STEMI in different trials, which indicates the importance of risk stratification and maintenance therapy after appropriate invasive procedure(20).

Previously, several studies have demonstrated the strength of different risk scores in different patient cohorts, yet each score had its own disadvantages especially in terms of application in clinical practice(21-24). Among these disadvantages, the time of obtaining required data such as angiographic futures, observer dependency interpreting for categoric variables, the duration of concomitant disease involved in the scoring systems such as DM can be counted among these disadvantages.

In order to provide both accurate and practical evaluation, risk assessment should be based on objective parameters rather than subjective parameters. Another issue is the cohort that we work on. For example, while TIMI and GRACE score demonstrated to have similar predictive performance in STEMI patients, the GRACE score was demonstrated to be better in clinical practice in terms of its applicability to all spectrum of acute coronary syndromes(25).

Another study comparing two risk scores (TIMI and CADILLAC) in terms of predicting in-hospital mortality in patients presented with STEMI and without cardiogenic shock, similar statistical significance has been demonstrated. Despite the similarity in statistical significance between the two scores, the TIMI risk score was superior to CADILLAC score according to c-index(26). In our study, CADILLAC score demonstrated to be better than TIMI with a c-index of 0.844 vs 0.771, respectively (Table 2).

This difference might be attributed to the longer follow-up period and a more heterogeneous patient population.

In another study by Huang et al., mACEF has been found superior to ACEF and GRACE score predicting in-hospital death in STEMI patients, yet there was no significant difference in NSTEMI patients(27).

According to the study of Kao et al. comparing the TIMI, GRACE, PAMI and CADILLAC risk scores in Taiwanese diabetic patients with STEMI, they have not included mACEF score to their study and demonstrated that CADILLAC risk score is the most effective model predicting six months, one year and two years all-cause mortality. In another study carried out by Lev et al., TIMI, PAMI, GRACE and CADILLAC score have been compared. According to c-statistics, TIMI and PAMI scores performed similar and well yet, CADILLAC score achieved the best performance in terms of predicting 30-days and one year mortality. The strength of CADILLAC scoring system was attributed to its ability to give information about angiographic features, LVEF and presence of anaemia, which are already indicators for poor prognosis(28,29).

In our study, CADILLAC score was demonstrated to be the strongest model after mACEF score. Although mACEF score is calculated using a few parameters comparing with relevant scores, it is still demonstrated to be the strongest to predict long-term all cause-death in clinical respect. The less component a model includes, the more applicable it is for clinical practice. CADILLAC score could be interpreted only after coronary angiography, on the other hand, mACEF score could be calculated on admission, and our results confirm its reliability in predicting long-term all-cause mortality in STEMI patients. Another problem we may encounter in scoring systems with multiple parameters is that they are not user friendly and practical. We tried to compare the models including variables easier to obtain at admission and during the intervention procedure. Some parameters including relevant scoring systems are relatively subjective such as Killip score. On the other hand, creatinine clearance and age are not debatable values depending on observers and are already determinants of poor prognosis. Therefore, better predictive value of the mACEF risk score among other scoring models may be attributed to the more objective components of the model. Ejection fraction, which is included in mACEF, is already a well-known determinant for long-term survival in patients with STEMI(28).

Limitations

The main limitation of our study is the retrospective nature of the study design. Secondly, the follow-up period is relatively short comparing with similar studies. Thirdly, the duration of DM, and if present, chronic kidney disease were not known, therefore the effect of the relevant disease on cardiovascular system cannot be interpreted very clearly for each person. Another limitation is, considering the wide ranges of confidence intervals, the study population was small in numbers, therefore further investigations should be carried out for more certain results.

Finally, a large scale prospective, multicentre study is required for comparing the risk scores and to decide on both the simplest and the strongest model to apply to our clinical practice.

Conclusion

Although similar statistical significance was presented among all scores, mACEF was demonstrated to be the strongest model among all five risk scores to predict long-term allcause mortality in clinical respect.

Cite this article as: Özkalaycı Kaçar F, Saygı M, Özer T, Karagöz A, Tanboğa İH, Oduncu V. Comparison of the performance of five different scoring systems in patients with ST-elevation myocardial infarction. Koşuyolu Heart J 2023;26(1):20-26.

Ethics Committee Approval

The study was approved by the ethics committee of Kartal Koşuyolu High Specialization Training and Research Hospital Clinical Research Ethics Committee (Decision no: 2022/13/626, Date: 06.09.2022).

Peer Review

Externally peer-reviewed.

Author Contributions

Concept/Design - FÖK; Analysis/Interpretation - İHT, AK; Data Collection - TÖ, MS; Writing - VO; Critical Revision - AK; Statistical Analysis - İHT; Overall Responsibility - FÖK.

Conflict of Interest

The authors have no conflicts of interest to declare.

Financial Disclosure

The authors declare that this study has received no financial support.

References

  1. Schiele F, Hochadel M, Tubaro M, Meneveau N, Wojakowski W, Gierlotka M, et al. Reperfusion strategy in Europe: Temporal trends in performance measures for reperfusion therapy in ST-elevation myocardial infarction. Eur Heart J 2010;31(21):2614-24. [Crossref] DOI: 10.1093/eurheartj/ehq305
  2. Ibanez B, James S, Agewall S, Antunes MJ, Bucciarelli-Ducci C, Bueno H, et al. 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation. Eur Heart J 2018;39(2):79-82. [Crossref] DOI: 10.5603/KP.2018.0041
  3. Morrow DA, Antman EM, Charlesworth A, Cairns R, Murphy SA, Lemos JA de, et al. TIMI risk score for ST-elevation myocardial infarction: A convenient, bedside, clinical score for risk assessment at presentation an intravenous nPA for treatment of infarcting myocardium early II trial substudy. Circulation 2000;102:2031-7. [Crossref] DOI: 10.1161/01.CIR.102.17.2031
  4. Kozieradzka A, Kamiński KA, Maciorkowska D, Olszewska M, Dobrzycki S, Nowak K, et al. GRACE, TIMI, Zwolle and CADILLAC risk scores-do they predict 5-year outcomes after ST-elevation myocardial infarction treated invasively? Int J Cardiol 2011;148:70-5. [Crossref] DOI: 10.1016/j.ijcard.2009.10.026
  5. Morrow DA, Antman EM, Parsons L, de Lemos JA, Cannon CP, Giugliano RP, et al. Application of the TIMI risk score for ST-elevation MI in the National Registry of Myocardial Infarction 3. JAMA 2001;286:1356- 9. [Crossref] DOI: 10.1001/jama.286.11.1356
  6. Addala S, Grines CL, Dixon SR, Stone GW, Boura JA, Ochoa AB, et al. Predicting mortality in patients with ST-elevation myocardial infarction treated with primary percutaneous coronary intervention (PAMI risk score). Am J Cardiol 2004;93:629-32. [Crossref] DOI: 10.1016/j.amjcard.2003.11.036
  7. Halkin A, Singh M, Nikolsky E, Grines CL, Tcheng JE, Garcia E, et al. Prediction of mortality after primary percutaneous coronary intervention for acute myocardial infarction. The CADILLAC risk score. J Am Coll Cardiol 2005;45:1397-405. [Crossref] DOI: 10.1016/j.jacc.2005.01.041
  8. Morrow DA, Antman EM, Giugliano RP, Cairns R, Charlesworth A, Murphy SA, et al. A simple risk index for rapid initial triage of patients with ST-elevation myocardial infarction: An InTIME II sub-study. Lancet 2001;358(9293):1571-5. [Crossref] DOI: 10.1016/S0140-6736(01)06649-1
  9. Wykrzykowska JJ, Garg S, Onuma Y, de Vries T, Goedhart D, Morel MA, et al. Value of age, creatinine, and ejection fraction (ACEF score) in assessing risk in patients undergoing percutaneous coronary interventions in the ‘All-Comers’ LEADERS trial. Circ Cardiovasc Interv 2011;4:47-56. [Crossref] DOI: 10.1161/CIRCINTERVENTIONS.110.958389
  10. Littnerova S, Kala P, Jarkovsky J, Kubkova L, Prymusova K, Kubena P, et al. GRACE score among six risk scoring systems (CADILLAC, PAMI, TIMI, Dynamic TIMI, Zwolle) demonstrated the best predictive value for prediction of long-term mortality in patients with ST-elevation myocardial infarction. PLoS One 2015;10(4):e0123215. [Crossref] DOI: 10.1371/journal.pone.0123215
  11. Kao YT, Hsieh YC, Hsu CY, Huang CY, Hsieh MH, Lin YK, et al. Comparison of the TIMI, GRACE, PAMI and CADILLAC risk scores for prediction of long-term cardiovascular outcomes in Taiwanese diabetic patients with ST-segment elevation myocardial infarction: From the registry of the Taiwan Society of Cardiology. PLoS One 2020;15(2):e0229186. [Crossref] DOI: 10.1371/journal.pone.0229186
  12. Lev EI, Kornowski R, Vaknin-Assa H, Porter A, Teplitsky I, Ben-Dor I, et al. Comparison of the predictive value of four different risk scores for outcomes of patients with ST-elevation acute myocardial infarction undergoing primary percutaneous coronary intervention. Am J Cardiol 2008;102:6-11. [Crossref] DOI: 10.1016/j.amjcard.2008.02.088
  13. Thygesen K, Alpert JS, White HD. Task force members, ESC Committee for practice guidelines, document reviewers, universal definition of myocardial infarction: Kristian Thygesen, Joseph S. Alpert and Harvey D. White on behalf of the Joint ESC/ACCF/AHA/WHF task force for the redefinition of myocardial infarction. Eur Heart J 2007;28(20):2525-38. [Crossref]
  14. Killip T, Kimball JT. Treatment of myocardial infarction in a coronary care unit. A two year experience with 250 patients. Am J Cardiol 1967;20(4):457-64. [Crossref] DOI: 10.1016/0002-9149(67)90023-9
  15. Lang RM, Bierig M, Devereux RB, Flachskampf FA, Foster E, Pellikka PA, et al. Recommendations for chamber quantification. Eur J Echocardiogr 2006;7(2):79-108. [Crossref] DOI: 10.1016/j.euje.2005.12.014
  16. Blanc B, Finch CA, Hallberg L, Herbert V, Lawkowicz W, Layrisse M. Nutritional anaemias. Report of a WHO Scientific Group. WHO Tech Rep Ser 1968:1-40.
  17. Cockcroft DW, Gault MH. Prediction of creatinine clearance from serum creatinine. Nephron 1976;16(1):31-41. [Crossref] DOI: 10.1159/000180580
  18. Van de Werf F, Ardissino D, Betrui A, Cokkinos DV, Falk E, Fox KA, et al. Management of acute myocardial infarction in patients with ST-segment elevation. Eur Heart J 2003;24:28-66. [Crossref] DOI: 10.1016/S0195-668X(02)00618-8
  19. Steg PG, James SK, Atar D, Badano LP, Blömstrom-Lundqvist C, Borger MA, et al. Task force on the management of ST-elevation acute myocardial infarction of the European Society of Cardiology (ESC). ESC guidelines for management of acute myocardial infarction in patientspresenting with ST-segment elevation. Eur Heart J 2012;33:2569-619. [Crossref]
  20. Windecker S, Hernández-Antolín RA, Stefanini GG, Wijns W, Zamorano JL. Management of ST-elevation myocardial infarction according to European and American guidelines. EuroIntervention 2014;10:23-31. [Crossref] DOI: 10.4244/EIJV10STA5
  21. Schellings DA, Adiyaman A, Giannitsis E, Hamm C, Suryapranata H, Ten Berg JM, et al. Early discharge after primary percutaneous coronary intervention: The added value of N-terminal pro-brain natriuretic peptide to the Zwolle Risk Score. J Am Heart Assoc 2014;3:e001089. [Crossref] DOI: 10.1161/JAHA.114.001089
  22. Raposeiras-Roubín S, Abu-Assi E, Cabanas-Grandío P, Agra-Bermejo RM, Gestal-Romarí S, Pereira-López E, et al. Walking beyond the GRACE (Global Registry of Acute Coronary Events) model in the death risk stratification during hospitalization in patients with acute coronary syndrome: What do the AR-G [ACTION (Acute Coronary Treatment and Intervention Outcomes Network) Registry and GWTG (Get With the Guidelines) Database], NCDR (National Cardiovascular Data Registry), and EuroHeart Risk Scores Provide? JACC Cardiovasc Interv 2012;5:1117-25. [Crossref] DOI: 10.1016/j.jcin.2012.06.023
  23. Brkovic V, Dobric M, Beleslin B, Giga V, Vukcevic V, Stojkovic S, et al. Additive prognostic value of the SYNTAX score over GRACE, TIMI, ZWOLLE, CADILLAC and PAMI risk scores in patients with acute STsegment elevation myocardial infarction treated by primary percutaneous coronary intervention. Int J Cardiovasc Imaging 2013;29:1215-28. [Crossref] DOI: 10.1007/s10554-013-0202-1
  24. Méndez-Eirín E, Flores-Ríos X, García-López F, Pérez-Pérez AJ, Estévez-Loureiro R, Piñón-Esteban P, et al. Comparison of the prognostic predictive value of the TIMI, PAMI, CADILLAC, and GRACE risk scores in STEACS undergoing primary or rescue PCI. Rev Esp Cardiol 2012;65:227-33. [Crossref] DOI: 10.1016/j.rec.2011.10.021
  25. Chan Pin Yin D, Azzahhafi J, James S. Risk assessment using risk scores in patients with acute coronary syndrome. J Clin Med 2020;9(9):3039. [Crossref] DOI: 10.3390/jcm9093039
  26. González-Pacheco H, Arias-Mendoza A, Álvarez-Sangabriel A, JuárezHerrera Ú, Damas F, Eid-Lidt G, et al. The TIMI risk score for STEMI predicts in-hospital mortality and adverse events in patients without cardiogenic shock undergoing primary angioplasty. Arch Cardiol Mex 2012;82(1):7-13.
  27. Huang J, Wei X, Wang Y, Jiang M, Lin Y, Su Z, et al. Comparison of prognostic value among 4 risk scores in patients with acute coronary syndrome: Findings from the improving care for cardiovascular disease in China-ACS (CCC-ACS) Project. Med Sci Monit 2021;27:e928863. [Crossref] DOI: 10.12659/MSM.928863
  28. Halkin A, Stone GW, Dixon SR. Impact, and determinants of left ventricular function in patients undergoing primary percutaneous coronary intervention in acute myocardial infarction. Am J Cardiol 2005;96:325- 31. [Crossref] DOI: 10.1016/j.amjcard.2005.03.069
  29. Nikolsky E, Aymong ED, Halkin A, Grines CL, Cox DA, Garcia E, et al. Impact of anemia in patients with acute myocardial infarction undergoing primary percutaneous coronary intervention: Analysis from the Controlled Abciximab and Device Investigation to Lower Late Angioplasty Complications (CADILLAC) Trial. J Am Coll Cardiol 2004;44:547-53. [Crossref] DOI: 10.1016/j.accreview.2004.10.057

Figure and Tables

Figure 1. Study flow chart.
Figure 2. Adjusted hazard ratio plot.
Figure 3. AUCROC comparison of the models.
Table 1. Comparison of baseline characteristics between the survivors and dead patients
Table 2. Multivariable cox regression analyses for different risk scoring systems
Table 3. Multivariable Cox proportional regression for all-cause-death