A Multimetric Evaluation of Ensemble and Neural Models for Imbalanced Credit Scoring on the German Credit Dataset
DOI:
https://doi.org/10.63158/journalisi.v8i5.1825Keywords:
Brier Score, Credit Scoring, Ensemble Learning, Imbalanced Data, PR-AUC, Tabular Credit RiskAbstract
Credit scoring models must balance global discrimination, minority-class detection, probability quality, and computational efficiency, particularly when default cases are less frequent than non-default cases. This study benchmarks Logistic Regression, Random Forest, XGBoost, and a Multilayer Perceptron using the German Credit Dataset, which contains 1,000 borrower records. All models were evaluated using a consistent preprocessing, partitioning, and imbalance-aware training protocol. Performance was assessed through ROC-AUC, PR-AUC, Balanced Accuracy, precision, recall, F1-score, Brier Score, and computational time. The results indicate moderate predictive performance across the evaluated models. XGBoost achieved the highest global discrimination with a ROC-AUC of 0.764, whereas the MLP obtained the highest PR-AUC of 0.378 and the lowest Brier Score of 0.123, indicating the best native probability quality among the evaluated models. No post-hoc probability calibration was applied. However, the MLP produced lower recall at the selected classification threshold. Logistic Regression required the least computation but recorded the lowest ROC-AUC and PR-AUC. Exploratory SHAP analysis was used to identify influential borrower attributes. These benchmark-specific findings demonstrate that model selection depends on the intended decision objective rather than architectural complexity alone.
Downloads
References
[1] L. I. Souadda, A. R. Halitim, B. Benilles, J. M. Oliveira, and P. Ramos, “Optimizing Credit Risk Prediction for Peer-to-Peer Lending Using Machine Learning,” Forecasting, vol. 7, no. 3, p. 35, Jun. 2025, doi: 10.3390/forecast7030035.
[2] V. Chang, Q. A. Xu, S. H. Akinloye, V. Benson, and K. Hall, “Prediction of bank credit worthiness through credit risk analysis: an explainable machine learning study,” Ann. Oper. Res., vol. 354, no. 1, pp. 247–271, Nov. 2025, doi: 10.1007/s10479-024-06134-x.
[3] M. Song, H. Ma, Y. Zhu, and M. Zhang, “Credit Risk Prediction Based on Improved ADASYN Sampling and Optimized LightGBM,” Journal of Social Computing, vol. 5, no. 3, pp. 232–241, 2024, doi: 10.23919/JSC.2024.0019.
[4] Y. Fang, “Application of a Grey Wolf Optimization-Enhanced Convolutional Neural Network and Bidirectional Gated Recurrent Unit Model for Credit Scoring Prediction,” PLoS One, vol. 20, no. 5, p. e0322225, 2025, doi: 10.1371/journal.pone.0322225.
[5] A. E. Khandani, A. J. Kim, and A. W. Lo, “Consumer Credit-Risk Models via Machine-Learning Algorithms,” J. Bank. Financ., vol. 34, no. 11, pp. 2767–2787, 2010, doi: 10.1016/j.jbankfin.2010.06.001.
[6] S. Kokate and M. S. R. Chetty, “Credit Risk Assessment of Loan Defaulters in Commercial Banks Using Voting Classifier Ensemble Learner Machine Learning Model,” International Journal of Safety and Security Engineering, vol. 11, no. 5, pp. 565–572, 2021, doi: 10.18280/ijsse.110508.
[7] M. M. Seliem, M. Amin, M. M. A. El Nasr, E. A. Elnaggar, H. A. M. Khalifa, and M. A. A. Arab, “Performance of Machine Learning Algorithms for Credit Risk Prediction with Feature Selection,” Stat. Optim. Inf. Comput, vol. 14, no. 1, pp. 311–328, doi: 10.19139/soic-2310-5070-2392.
[8] C. X. Pham, H. N. Trinh, and L. Q. Tran, “A Robust Approach to Credit Scoring with Deep Learning and Embedded Methods,” Engineering, Technology & Applied Science Research, vol. 15, no. 6, pp. 29284–29291, 2025, doi: 10.48084/etasr.12649.
[9] A. Shmuel, O. Glickman, and T. Lazebnik, “A Comprehensive Benchmark of Machine and Deep Learning Models on Structured Data for Regression and Classification,” Neurocomputing, vol. 655, p. 131337, 2025, doi: 10.1016/j.neucom.2025.131337.
[10] B. R. Gunnarsson, S. V. Broucke, B. Baesens, M. Óskarsdóttir, and W. Lemahieu, “Deep learning for credit scoring: Do or don’t?,” Eur. J. Oper. Res, vol. 295, no. 1, pp. 292–305, doi: 10.1016/j.ejor.2021.03.006.
[11] I. M. B. Vera, J. M. Restrepo, and D. F. Manotas-Duque, “Data Mining for the Adjustment of Credit Scoring Models in Solidarity Economy Entities: A Methodology for Addressing Class Imbalances,” Risks, vol. 13, no. 2, p. 20, 2025, doi: 10.3390/risks13020020.
[12] J. T. Hancock, T. M. Khoshgoftaar, and J. M. Johnson, “Evaluating Classifier Performance with Highly Imbalanced Big Data,” J. Big Data, vol. 10, no. 1, p. 42, 2023, doi: 10.1186/s40537-023-00724-5.
[13] P. Thölke, “Class Imbalance Should Not Throw You Off Balance: Choosing the Right Classifiers and Performance Metrics for Brain Decoding with Imbalanced Data,” Neuroimage, vol. 277, p. 120253, 2023, doi: 10.1016/j.neuroimage.2023.120253.
[14] J. T. Hancock, T. M. Khoshgoftaar, and J. M. Johnson, “Evaluating classifier performance with highly imbalanced Big Data,” J. Big Data, vol. 10, no. 1, p. 42, doi: 10.1186/s40537-023-00724-5.
[15] F. Shen, X. Zhao, G. Kou, and F. E. Alsaadi, “A New Deep Learning Ensemble Credit Risk Evaluation Model with an Improved Synthetic Minority Oversampling Technique,” Appl. Soft Comput., vol. 98, p. 106852, 2021, doi: 10.1016/j.asoc.2020.106852.
[16] L. O. Hjelkrem and P. E. D. Lange, “Explaining Deep Learning Models for Credit Scoring with SHAP: A Case Study Using Open Banking Data,” Journal of Risk and Financial Management, vol. 16, no. 4, p. 221, 2023, doi: 10.3390/jrfm16040221.
[17] C. X. Pham, H. N. Trinh, and L. Q. Tran, “A Robust Approach to Credit Scoring with Deep Learning and Embedded Methods,” Eng. Technol. Appl. Sci. Res, vol. 15, no. 6, pp. 29284–29291, doi: 10.48084/etasr.12649.
[18] J. Xiao, S. Li, Y. Tian, J. Huang, X. Jiang, and S. Wang, “Example Dependent Cost Sensitive Learning Based Selective Deep Ensemble Model for Customer Credit Scoring,” Sci. Rep., vol. 15, no. 1, p. 6000, 2025, doi: 10.1038/s41598-025-89880-7.
[19] G. James, D. Witten, T. Hastie, and R. Tibshirani, An Introduction to Statistical Learning, 2nd ed. in Springer Texts in Statistics. New York, NY: Springer US, 2021. doi: 10.1007/978-1-0716-1418-1.
[20] M. Sivakumar, S. Parthasarathy, and T. Padmapriya, “Trade-off between training and testing ratio in machine learning for medical image processing,” PeerJ Comput. Sci, vol. 10, p. 2245, 2024. doi: 10.7717/peerj-cs.2245
[21] Y. Chen, R. Calabrese, and B. Martin-Barragan, “Interpretable machine learning for imbalanced credit scoring datasets,” Eur. J. Oper. Res, vol. 312, no. 1, pp. 357–372, 2024. doi: 10.1016/j.ejor.2023.06.036.
[22] P. Thölke, “Class imbalance should not throw you off balance: Choosing the right classifiers and performance metrics for brain decoding with imbalanced data,” Neuroimage, vol. 277, p. 120253, 2023. doi: 10.1016/j.neuroimage.2023.120253.
[23] W. Liu, H. Fan, M. Xia, and M. Xia, “A focal-aware cost-sensitive boosted tree for imbalanced credit scoring,” Expert Syst. Appl, vol. 208, p. 118158, 2022. doi: 10.1016/j.eswa.2022.118158.
[24] A. Aghbalou, A. Sabourin, and F. Portier, “On the Bias of K-Fold Cross Validation with Stable Learners,” in Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS), vol. 206, pp. 3775–3794, 2023
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Information Systems and Informatics
This work is licensed under a Creative Commons Attribution 4.0 International License.
Author's Declaration
- The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
- The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
- The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
- The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
- The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
- The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
- If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
- The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights
