(Malang State University, Indonesia)(2) * Anik Nur Handayani
(Malang State University, Indonesia)(3) Jevri Tri Ardiansah
(Malang State University, Indonesia)(4) Kohei Arai
(Saga University, Saga, Japan)*corresponding author
AbstractHeart failure is a leading cause of morbidity and mortality worldwide, and early prediction of outcomes is critical for timely intervention and improved patient care. Accurate prediction models can help clinicians identify high-risk patients, optimize treatment strategies, and reduce healthcare costs. In this study, we developed and evaluated machine learning models to predict mortality in patients with heart failure using a medical dataset of 299 patients with 13 clinical variables collected in 2015. Four models were tested, including a Feedforward Neural Network (FNN), Random Forest, XGBoost, and an ensemble model combining all three models. The experimental process included data preprocessing, feature scaling, and stratified cross-validation to ensure robust evaluation. The results showed that the ensemble model achieved the best performance with an ROC-AUC of 0.9134 and an F1 score of 0.7439, outperforming individual models such as Random Forest (ROC-AUC: 0.9117) and XGBoost (ROC-AUC: 0.9130). FNN, despite having the highest accuracy (0.8455), showed lower performance in terms of recall and precision, likely due to its sensitivity to overfitting on small datasets. These results highlight the effectiveness of ensemble learning in medical prediction tasks, especially for handling complex, high-dimensional health data. The proposed ensemble model has the potential to be integrated into clinical decision support systems, enabling real-time risk assessment and personalized treatment plans for heart failure patients. Future research should explore larger, multicenter datasets, incorporate advanced feature engineering techniques, and investigate the integration of deep learning architectures such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs) to process sequential data such as ECG signals.
KeywordsHeart Failure Prediction;Ensemble Learning;Early Detection;Patient Outcomes;AUC-ROC
|
DOIhttps://doi.org/10.31763/aet.v3i3.1750 |
Article metrics10.31763/aet.v3i3.1750 Abstract views : 16 | PDF views : 11 |
Cite |
Full Text Download
|
References
[1] M. S. Akter, H. Shahriar, R. Chowdhury, and M. R. C. Mahdy, “Forecasting the Risk Factor of Frontier Markets: A Novel Stacking Ensemble of Neural Network Approach,” Futur. Internet, vol. 14, no. 9, p. 252, Aug. 2022, doi: 10.3390/fi14090252.
[2] R. Miotto, F. Wang, S. Wang, X. Jiang, and J. T. Dudley, “Deep learning for healthcare: review, opportunities and challenges,” Brief. Bioinform., vol. 19, no. 6, pp. 1236–1246, Nov. 2018, doi: 10.1093/bib/bbx044.
[3] E. Choi, A. Schuetz, W. F. Stewart, and J. Sun, “Using recurrent neural network models for early detection of heart failure onset,” J. Am. Med. Informatics Assoc., vol. 24, no. 2, pp. 361–370, Mar. 2017, doi: 10.1093/jamia/ocw112.
[4] L. Rasmy et al., “A study of generalizability of recurrent neural network-based predictive models for heart failure onset risk using a large and heterogeneous EHR data set,” J. Biomed. Inform., vol. 84, pp. 11–16, Aug. 2018, doi: 10.1016/j.jbi.2018.06.011.
[5] T. R. Albernaz, E. P. De Souza, M. N. R. Da Silva, and H. S. Carvalho, “An Approach To Computer-Aided Diagnosis Of Heart Disorders Using Wavelets And Deep Learning Applied To Electrocardiograms (Ekgs),” Rev. FOCO, vol. 16, no. 9, p. e2974, Sep. 2023, doi: 10.54751/revistafoco.v16n9-164.
[6] L. M. Dang et al., “Toward explainable heat load patterns prediction for district heating,” Sci. Reports 2023 131, vol. 13, no. 1, pp. 1–13, May 2023, doi: 10.1038/s41598-023-34146-3.
[7] L.-L. Xu et al., “Machine learning in predicting T-score in the Oxford classification system of IgA nephropathy,” Front. Immunol., vol. 14, p. 1224631, Aug. 2023, doi: 10.3389/fimmu.2023.1224631.
[8] C. Huang, F. Li, L. Wei, X. Hu, and Y. Yang, “Landslide Susceptibility Modeling Using a Deep Random Neural Network,” Appl. Sci., vol. 12, no. 24, p. 12887, Dec. 2022, doi: 10.3390/app122412887.
[9] X. Wang, X. Zhao, G. Song, J. Niu, and T. Xu, “Machine Learning-Based Evaluation on Craniodentofacial Morphological Harmony of Patients After Orthodontic Treatment,” Front. Physiol., vol. 13, p. 862847, May 2022, doi: 10.3389/fphys.2022.862847.
[10] W. Lin et al., “Korotkoff sounds dynamically reflect changes in cardiac function based on deep learning methods,” Front. Cardiovasc. Med., vol. 9, p. 940615, Aug. 2022, doi: 10.3389/fcvm.2022.940615.
[11] Y. Zhang et al., “Opening the black box: interpretable machine learning for predictor finding of metabolic syndrome,” BMC Endocr. Disord., vol. 22, no. 1, p. 214, Aug. 2022, doi: 10.1186/s12902-022-01121-4.
[12] A. Baghbani, N. Bouguila, and Z. Patterson, “Short-Term Passenger Flow Prediction Using a Bus Network Graph Convolutional Long Short-Term Memory Neural Network Model,” Transp. Res. Rec. J. Transp. Res. Board, vol. 2677, no. 2, pp. 1331–1340, Feb. 2023, doi: 10.1177/03611981221112673.
[13] A. Ferencek, D. Kofjač, A. Škraba, B. Sašek, and M. K. Borštnar, “Deep Learning Predictive Models for Terminal Call Rate Prediction during the Warranty Period,” Bus. Syst. Res. J., vol. 11, no. 2, pp. 36–50, Oct. 2020, doi: 10.2478/bsrj-2020-0014.
[14] J. Kwon et al., “Artificial intelligence assessment for early detection of heart failure with preserved ejection fraction based on electrocardiographic features,” Eur. Hear. J. - Digit. Heal., vol. 2, no. 1, pp. 106–116, May 2021, doi: 10.1093/ehjdh/ztaa015.
[15] R. D. Prince, A. Akhondi-Asl, N. M. Mehta, and A. Geva, “A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation,” Crit. Care Explor., vol. 3, no. 5, p. e0426, May 2021, doi: 10.1097/CCE.0000000000000426.
[16] Q. A. Hidayaturrohman and E. Hanada, “Predictive Analytics in Heart Failure Risk, Readmission, and Mortality Prediction: A Review,” Cureus, vol. 16, no. 11, p. 11, Nov. 2024, doi: 10.7759/cureus.73876.
[17] I. D. Mienye and Y. Sun, “A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects,” IEEE Access, vol. 10, pp. 99129–99149, 2022, doi: 10.1109/ACCESS.2022.3207287.
[18] D.-K. Nguyen, C.-H. Lan, and C.-L. Chan, “Deep Ensemble Learning Approaches in Healthcare to Enhance the Prediction and Diagnosing Performance: The Workflows, Deployments, and Surveys on the Statistical, Image-Based, and Sequential Datasets,” Int. J. Environ. Res. Public Health, vol. 18, no. 20, p. 10811, Oct. 2021, doi: 10.3390/ijerph182010811.
[19] Y. Gu et al., “Predicting medication adherence using ensemble learning and deep learning models with large scale healthcare data,” Sci. Rep., vol. 11, no. 1, p. 18961, Sep. 2021, doi: 10.1038/s41598-021-98387-w.
[20] P. Tian et al., “Machine Learning for Mortality Prediction in Patients With Heart Failure With Mildly Reduced Ejection Fraction,” J. Am. Heart Assoc., vol. 12, no. 12, p. e029124, Jun. 2023, doi: 10.1161/JAHA.122.029124.
[21] J. Zhang, U. Norinder, and F. Svensson, “Deep Learning-Based Conformal Prediction of Toxicity,” J. Chem. Inf. Model., vol. 61, no. 6, pp. 2648–2657, Jun. 2021, doi: 10.1021/acs.jcim.1c00208.
[22] D. Veritti, L. Rubinato, V. Sarao, A. De Nardin, G. L. Foresti, and P. Lanzetta, “Behind the mask: a critical perspective on the ethical, moral, and legal implications of AI in ophthalmology,” Graefe’s Arch. Clin. Exp. Ophthalmol., vol. 262, no. 3, pp. 975–982, Mar. 2024, doi: 10.1007/s00417-023-06245-4.
[23] M. S. Barkhordari and L. M. Massone, “Failure Mode Detection of Reinforced Concrete Shear Walls Using Ensemble Deep Neural Networks,” Int. J. Concr. Struct. Mater., vol. 16, no. 1, p. 33, Dec. 2022, doi: 10.1186/s40069-022-00522-y.
[24] J. Tromp et al., “Age-Related Characteristics and Outcomes of Patients With Heart Failure With Preserved Ejection Fraction,” J. Am. Coll. Cardiol., vol. 74, no. 5, pp. 601–612, Aug. 2019, doi: 10.1016/j.jacc.2019.05.052.
[25] S. Paul and R. V. Paul, “Anemia in Heart Failure,” J. Cardiovasc. Nurs., vol. 19, no. Supplement, pp. S57–S66, Nov. 2004, doi: 10.1097/00005082-200411001-00008.
[26] B. Zareini et al., “Type 2 Diabetes Mellitus and Impact of Heart Failure on Prognosis Compared to Other Cardiovascular Diseases,” Circ. Cardiovasc. Qual. Outcomes, vol. 13, no. 7, pp. 386–394, Jul. 2020, doi: 10.1161/CIRCOUTCOMES.119.006260.
[27] J. P. Curtis et al., “The association of left ventricular ejection fraction, mortality, and cause of death in stable outpatients with heart failure,” J. Am. Coll. Cardiol., vol. 42, no. 4, pp. 736–742, Aug. 2003, doi: 10.1016/S0735-1097(03)00789-7.
[28] J. Slivnick and B. C. Lampert, “Hypertension and Heart Failure,” Heart Fail. Clin., vol. 15, no. 4, pp. 531–541, Oct. 2019, doi: 10.1016/j.hfc.2019.06.007.
[29] J. Wang et al., “Impact of heart failure and preoperative platelet count on the postoperative short‐term outcome in infective endocarditis patients,” Clin. Cardiol., vol. 47, no. 1, p. e24171, Jan. 2024, doi: 10.1002/clc.24171.
[30] J. C. Schefold, M. Lainscak, L. M. Hodoscek, S. Blöchlinger, W. Doehner, and S. von Haehling, “Single baseline serum creatinine measurements predict mortality in critically ill patients hospitalized for acute heart failure,” ESC Hear. Fail., vol. 2, no. 4, pp. 122–128, Dec. 2015, doi: 10.1002/ehf2.12058.
[31] S. Peng, J. Peng, L. Yang, and W. Ke, “Relationship between serum sodium levels and all-cause mortality in congestive heart failure patients: A retrospective cohort study based on the Mimic-III database,” Front. Cardiovasc. Med., vol. 9, p. 1082845, Jan. 2023, doi: 10.3389/fcvm.2022.1082845.
[32] N. Fluschnik et al., “Gender differences in characteristics and outcomes in heart failure patients referred for end‐stage treatment,” ESC Hear. Fail., vol. 8, no. 6, pp. 5031–5039, Dec. 2021, doi: 10.1002/ehf2.13567.
[33] D. Kamimura et al., “Cigarette Smoking and Incident Heart Failure,” Circulation, vol. 137, no. 24, pp. 2572–2582, Jun. 2018, doi: 10.1161/CIRCULATIONAHA.117.031912.
[34] A. Abdin et al., “‘Time is prognosis’ in heart failure: time‐to‐treatment initiation as a modifiable risk factor,” ESC Hear. Fail., vol. 8, no. 6, pp. 4444–4453, Dec. 2021, doi: 10.1002/ehf2.13646.
[35] K. Seu, M.-S. Kang, and H. Lee, “An Intelligent Missing Data Imputation Techniques: A Review,” JOIV Int. J. Informatics Vis., vol. 6, no. 1–2, p. 278, May 2022, doi: 10.30630/joiv.6.1-2.935.
[36] M. Ahsan, M. Mahmud, P. Saha, K. Gupta, and Z. Siddique, “Effect of Data Scaling Methods on Machine Learning Algorithms and Model Performance,” Technologies, vol. 9, no. 3, p. 52, Jul. 2021, doi: 10.3390/technologies9030052.
[37] V. Sarraju, J. Pal, and S. Kamilya, “SRS: Gender-based heart disease prediction using stratified random sampling approach,” in AIP Conference Proceedings, May 2024, vol. 3164, no. 1, p. 020005, doi: 10.1063/5.0216559.
[38] P. Mooijman, C. Catal, B. Tekinerdogan, A. Lommen, and M. Blokland, “The effects of data balancing approaches: A case study,” Appl. Soft Comput., vol. 132, p. 109853, Jan. 2023, doi: 10.1016/j.asoc.2022.109853.
[39] T. E. Tarigan, E. Susanti, M. I. Siami, I. Arfiani, A. A. Jiwa Permana, and I. M. Sunia Raharja, “Performance Metrics of AdaBoost and Random Forest in Multi-Class Eye Disease Identification: An Imbalanced Dataset Approach,” Int. J. Artif. Intell. Med. Issues, vol. 1, no. 2, pp. 84–94, Nov. 2023, doi: 10.56705/ijaimi.v1i2.98.
[40] S. Das, S. P. Nayak, B. Sahoo, and S. C. Nayak, “Evaluating Ensemble Models on Imbalanced Data Sets: A Comparative Study across Varied Minority Class Ratios,” in 2024 International Conference on Emerging Systems and Intelligent Computing (ESIC), Feb. 2024, pp. 774–779, doi: 10.1109/ESIC60604.2024.10481583.
[41] N. Buslim, “Ensemble learning techniques to improve the accuracy of predictive model performance in the scholarship selection process,” J. Appl. Data Sci., vol. 4, no. 3, pp. 264–275, Sep. 2023, doi: 10.47738/jads.v4i3.112.
[42] A. Mohammed and R. Kora, “A comprehensive review on ensemble deep learning: Opportunities and challenges,” J. King Saud Univ. - Comput. Inf. Sci., vol. 35, no. 2, pp. 757–774, Feb. 2023, doi: 10.1016/j.jksuci.2023.01.014.
[43] J. Zhao, J. Jin, S. Chen, R. Zhang, B. Yu, and Q. Liu, “A weighted hybrid ensemble method for classifying imbalanced data,” Knowledge-Based Syst., vol. 203, p. 106087, Sep. 2020, doi: 10.1016/j.knosys.2020.106087.
[44] J. Qiu, “An Analysis of Model Evaluation with Cross-Validation: Techniques, Applications, and Recent Advances,” Adv. Econ. Manag. Polit. Sci., vol. 99, no. 1, pp. 69–72, Sep. 2024, doi: 10.54254/2754-1169/99/2024OX0213.
[45] Y. Wen, M. Kalander, C. Su, and L. Pan, “An Ensemble Noise-Robust K-fold Cross-Validation Selection Method for Noisy Labels,” arxiv Artif. Intell., pp. 1–9, 2021, [Online]. Available at: http://arxiv.org/abs/2107.02347.
[46] C. Miller, T. Portlock, D. M. Nyaga, and J. M. O’Sullivan, “A review of model evaluation metrics for machine learning in genetics and genomics,” Front. Bioinforma., vol. 4, p. 1457619, Sep. 2024, doi: 10.3389/fbinf.2024.1457619.
[47] L. Sweet, C. Müller, M. Anand, and J. Zscheischler, “Cross-Validation Strategy Impacts the Performance and Interpretation of Machine Learning Models,” Artif. Intell. Earth Syst., vol. 2, no. 4, Oct. 2023, doi: 10.1175/AIES-D-23-0026.1.
[48] P. Mahajan, S. Uddin, F. Hajati, M. A. Moni, and E. Gide, “A comparative evaluation of machine learning ensemble approaches for disease prediction using multiple datasets,” Health Technol. (Berl)., vol. 14, no. 3, pp. 597–613, May 2024, doi: 10.1007/s12553-024-00835-w.
[49] X. Yang, L. Wen, M. Sun, J. Yang, and B. Zhang, “Prediction of cardiac deterioration in acute heart failure patients: Evaluation of the efficacy of single laboratory indicator models versus comprehensive models,” Medicine (Baltimore)., vol. 103, no. 44, p. e40266, Nov. 2024, doi: 10.1097/MD.0000000000040266.
[50] Q. Wang et al., “Machine learning‐based risk prediction of malignant arrhythmia in hospitalized patients with heart failure,” ESC Hear. Fail., vol. 8, no. 6, pp. 5363–5371, Dec. 2021, doi: 10.1002/ehf2.13627.
[51] W. Chua et al., “An angiopoietin 2, FGF23, and BMP10 biomarker signature differentiates atrial fibrillation from other concomitant cardiovascular conditions,” Sci. Rep., vol. 13, no. 1, p. 16743, Oct. 2023, doi: 10.1038/s41598-023-42331-7.
[52] C. Vlachas et al., “Random forest classification algorithm for medical industry data,” SHS Web Conf., vol. 139, p. 03008, May 2022, doi: 10.1051/shsconf/202213903008.
[53] J. C. Yang, “The prediction and analysis of heart disease using XGBoost algorithm,” Appl. Comput. Eng., vol. 41, no. 1, pp. 61–68, Feb. 2024, doi: 10.54254/2755-2721/41/20230711.
[54] S. Decherchi, E. Pedrini, M. Mordenti, A. Cavalli, and L. Sangiorgi, “Opportunities and Challenges for Machine Learning in Rare Diseases,” Front. Med., vol. 8, p. 747612, Oct. 2021, doi: 10.3389/fmed.2021.747612.
[55] Q. Gao, “Application of Machine Learning in the field of Heart Disease Prediction and its Accuracy Study,” Sci. Technol. Eng. Chem. Environ. Prot., vol. 1, no. 8, Aug. 2024, doi: 10.61173/24749p02.
[56] D. Bertsimas, L. Mingardi, and B. Stellato, “Machine Learning for Real-Time Heart Disease Prediction,” IEEE J. Biomed. Heal. Informatics, vol. 25, no. 9, pp. 3627–3637, Sep. 2021, doi: 10.1109/JBHI.2021.3066347.
[57] M. Qiu, L.-L. Ding, and H.-R. Zhou, “Factors affecting the efficacy of SGLT2is on heart failure events: a meta-analysis based on cardiovascular outcome trials,” Cardiovasc. Diagn. Ther., vol. 11, no. 3, pp. 699–706, Jun. 2021, doi: 10.21037/cdt-20-984.
[58] P. Bhattarai and M. Karki, “The Unrepaired Tetralogy of Fallot: A Tale of Delayed Presentation and Limited Access to Care,” Cureus, vol. 16, no. 1, Jan. 2024, doi: 10.7759/cureus.52407.
[59] E. M. DeFilippis et al., “Impact of socioeconomic deprivation on evaluation for heart transplantation at an urban academic medical center,” Clin. Transplant., vol. 36, no. 6, p. e14652, Jun. 2022, doi: 10.1111/ctr.14652.
[60] R. S. Walia and R. Mankoff, “Impact of Socioeconomic Status on Heart Failure,” J. Community Hosp. Intern. Med. Perspect., vol. 13, no. 6, p. 24, Nov. 2023, doi: 10.55729/2000-9666.1258.
[61] O. A. Akinyemi et al., “Evaluating the Predictive Accuracy of Socioeconomic Metrics on Heart Failure Risk and Outcomes in Maryland,” Cureus, vol. 16, no. 9, Sep. 2024, doi: 10.7759/cureus.69474.
[62] A. Sinha et al., “Interconnected Clinical and Social Risk Factors in Breast Cancer and Heart Failure,” Front. Cardiovasc. Med., vol. 9, p. 847975, May 2022, doi: 10.3389/fcvm.2022.847975.
[63] L. de Tantillo, B. E. McCabe, M. Zdanowicz, J. Ortega, J. M. Gonzalez, and S. Chaparro, “Implementing Strategies to Recruit and Retain a Diverse Sample of Heart Failure Patients,” Hisp. Heal. Care Int., vol. 23, no. 1, pp. 9–17, Mar. 2025, doi: 10.1177/15404153241248144.
[64] C.-C. Chiu, C.-M. Wu, T.-N. Chien, L.-J. Kao, C. Li, and H.-L. Jiang, “Applying an Improved Stacking Ensemble Model to Predict the Mortality of ICU Patients with Heart Failure,” J. Clin. Med., vol. 11, no. 21, p. 6460, Oct. 2022, doi: 10.3390/jcm11216460.
Refbacks
- There are currently no refbacks.
Copyright (c) 2025 Nadindra Dwi Ariyanta, Anik Nur Handayani, Jevri Tri Ardiansah, Kohei Arai
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Applied Engineering and Technology
ISSN: 2829-4998
Email: aet@ascee.org | andri.pranolo.id@ieee.org
Published by: Association for Scientic Computing Electronics and Engineering (ASCEE)
Organized by: Association for Scientic Computing Electronics and Engineering (ASCEE), Universitas Negeri Malang, Universitas Ahmad Dahlan
View My Stats AET
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.