Software Defect Prediction: A Systematic Review and a Conceptual Framework for Stable, Actionable Explanations (AISD-F)
DOI:
https://doi.org/10.63158/journalisi.v8i5.1823Keywords:
Software defect prediction, Explainable Artificial Intelligence, explanation stability, class imbalance, SLRAbstract
Machine learning-based Software Defect Prediction (SDP) has largely focused on improving prediction accuracy, but recent research has placed greater emphasis on making predictions understandable and useful to developers. This systematic review examines the use of Explainable Artificial Intelligence (XAI) in SDP and the evidence supporting its reliability in practice. Following PRISMA guidelines, 40 peer-reviewed studies, primarily published between 2020 and 2025, were selected from IEEE Xplore, SpringerLink, ScienceDirect, Google Scholar, and Scopus. The studies were examined in terms of XAI methods, explanation reliability and stability, dataset quality, and model selection. LIME and SHAP were the most frequently used methods, yet multi-run evaluations showed changes in their feature rankings across configurations, in contrast to the reliability reported by many single-run studies. Data-quality problems, particularly SZZ-related label noise, class imbalance, and multicollinearity, often had a greater effect on prediction reliability than model architecture. The review also found growing use of line-level and just-in-time prediction, although consistent validation at industrial scale remains limited. Within the reviewed corpus, no study provides controlled evidence that XAI explanations improve developers' ability to resolve software defects, and the field currently lacks industrial-scale validation and controlled developer studies of XAI-assisted defect resolution. Based on these findings, this paper proposes the Actionable and Interpretable Software Defect Framework (AISD-F), a conceptual, five-stage modular architecture (not yet empirically validated) that is intended to integrate data quality assurance, hybrid model selection, dual-layer explanation, and CI/CD into a unified research agenda for explainable SDP, rather than a deployment-ready solution.
Downloads
References
[1]. M. Mustaqeem, M. Alam, S. Mustajab, F. Alshanketi, S. Alam, and M. Shuaib, “Comprehensive Bibliographic Survey and Forward-Looking Recommendations for Software Defect Prediction: Datasets, Validation Methodologies, Prediction Approaches, and Tools,” IEEE Access, Dec. 2024, doi: 10.1109/ACCESS.2024.3517419.
[2]. K. Bhandari, K. Kumar, and A. L. Sangal, “Data Quality Issues in Software Fault Prediction: A Systematic Literature Review,” Artif. Intell. Rev., vol. 56, pp. 7839–7908, 2023, doi: 10.1007/s10462-022-10371-6.
[3]. B. Isong and E. Igo, “Ensemble Learning for Software Defect Prediction: Performance, Practicality and Future Directions,” J. Inf. Syst. Inform., vol. 7, no. 3, pp. 2245–2291, 2025, doi: 10.51519/journalisi.v7i3.1171.
[4]. J. Pachouly, S. Ahirrao, K. Kotecha, G. Selvachandran, and A. Abraham, “A Systematic Literature Review on Software Defect Prediction Using Artificial Intelligence: Datasets, Data Validation Methods, Approaches, and Tools,” Eng. Appl. Artif. Intell., vol. 111, Art. no. 104773, 2022.
[5]. T. Hasan and S. M. Mohi-Aldeen, “Software Defect Prediction Based on Deep Learning Algorithms: A Systematic Literature Review,” Al-Rafidain J. Comput. Sci. Math. (RJCSM), vol. 19, no. 1, pp. 67–79, Jun. 2025, doi: 10.33899/csmj.2025.156086.1160.
[6]. A. H. Mohammadkhani, N. S. Bommi, M. Daboussi, O. Sabnis, C. Tantithamthavorn, and H. Hemmati, “A Systematic Literature Review of Explainable AI for Software Engineering,” Inf. Softw. Technol., 2023.
[7]. E. S. Ortigossa, T. Gonçalves, and L. G. Nonato, “Explainable Artificial Intelligence (XAI)—From Theory to Methods and Applications,” IEEE Access, vol. 12, pp. 83661–83733, 2024, doi: 10.1109/ACCESS.2024.3409843.
[8]. Gezici and A. K. Tarhan, “Explainable AI for Software Defect Prediction with Gradient Boosting Classifier,” in Proc. 7th Int. Conf. Comput. Sci. Eng. (UBMK), Diyarbakir, Turkey, 2022, pp. 1–6, doi: 10.1109/UBMK55850.2022.9919490.
[9]. M. A. Awal and C. K. Roy, “EvaluateXAI: A Framework to Evaluate the Reliability and Consistency of Rule-Based XAI Techniques for Software Analytics Tasks,” J. Syst. Softw., vol. 217, p. 112159, 2024.
[10]. S. Stradowski and L. Madeyski, “Interpretability/Explainability Applied to Machine Learning Software Defect Prediction: An Industrial Perspective,” IEEE Softw., Nov. 2024, doi: 10.1109/MS.2024.3505544.
[11]. O. R. Polu, “Machine Learning for Predicting Software Project Failure Risks,” Int. J. Comput. Eng. Technol, vol. 15, no. 4, pp. 950–959, 2024.
[12]. J. Jiarpakdee, C. Tantithamthavorn, H. K. Dam, and J. Grundy, “An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models,” IEEE Trans. Softw. Eng., Mar. 2020, doi: 10.1109/TSE.2020.2982385.
[13]. B. Asal and M. Ö. Demir, “Enhancing Software Defect Prediction Through Explainable AI: Integrating SHAP and LIME in a Voting Classifier Framework,” in Proc. 8th Int. Artif. Intell. Data Process. Symp. (IDAP), Malatya, Turkiye, 2024, pp. 1–7, doi: 10.1109/IDAP64064.2024.10710700.
[14]. A. Khan, A. Ali, J. Khan, F. Ullah, and M. A. Khan, “A Systematic Literature Review of Explainable Artificial Intelligence (XAI) in Software Engineering (SE),” Research Square, Sep. 8, 2023, doi: 10.21203/rs.3.rs-3209115/v1.
[15]. M. Mersha, K. Lam, J. Wood, A. AlShami, and J. Kalita, “Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction,” Neurocomputing, vol. 599, Art. no. 128111, 2024, doi: 10.1016/j.neucom.2024.128111.
[16]. J. Shin, R. Aleithan, J. Nam, J. Wang, N. S. Harzevili, and S. Wang, “An Empirical Study on the Stability of Explainable Software Defect Prediction,” in Proc. 30th Asia–Pacific Softw. Eng. Conf. (APSEC), Seoul, Republic of Korea, 2023, pp. 141–150, doi: 10.1109/APSEC60848.2023.00024.
[17]. G. Xu, C. Feng, X. Guo, Z. Zhu, and W. Wang, “A Joint Learning Framework for Bridging Defect Prediction and Interpretation,” IEEE Trans. Rel., 2025, doi: 10.1109/TR.2025.3586671.
[18]. Gezici Geçer and A. Kolukısa Tarhan, “Explainable AI Framework for Software Defect Prediction,” J. Softw.: Evol. Process, vol. 37, no. 4, p. e70018, 2025.
[19]. G. Esteves, E. Figueiredo, A. Veloso, et al., “Understanding Machine Learning Software Defect Predictions,” Autom. Softw. Eng., vol. 27, pp. 369–392, 2020.
[20]. L. Arora, S. S. Girija, S. Kapoor, A. Raj, D. Pradhan and A. Shetgaonkar, “Explainable Artificial Intelligence Techniques for Software Development Lifecycle: A Phase-specific Survey,” in Proc. 2025 IEEE 49th Annu. Comput., Softw., Appl. Conf. (COMPSAC), Toronto, ON, Canada, 2025, pp. 2281-2288, doi: 10.1109/COMPSAC65507.2025.00321.
[21]. G. Lee and S. U. -J. Lee, “An Empirical Comparison of Model-Agnostic Techniques for Defect Prediction Models,” in Proc. 2023 IEEE Int. Conf. Softw. Anal., Evol. Reeng. (SANER), Taipa, Macao, 2023, pp. 179-189, doi: 10.1109/SANER56733.2023.00026.
[22]. R. Jia, “A Brief Analysis of the Progress and Trends in Software Defect Prediction Methods,” in Proc. 5th Int. Conf. Signal Process. Mach. Learn., 2025, doi: 10.54254/2755-2721/99/20251769.
[23]. Q. Song, Y. Guo, and M. Shepperd, “A Comprehensive Investigation of the Role of Imbalanced Learning for Software Defect Prediction,” IEEE Trans. Softw. Eng., vol. 45, no. 12, pp. 1253–1269, Dec. 2019, doi: 10.1109/TSE.2018.2836442.
[24]. L. Gong, H. Zhang, J. Zhang, M. Wei, and Z. Huang, “A Comprehensive Investigation of the Impact of Class Overlap on Software Defect Prediction,” IEEE Trans. Softw. Eng., Apr. 2023, doi: 10.1109/TSE.2022.3220740.
[25]. I. Mehmood, S. Shahid, H. Hussain, I. Khan, S. Ahmad, S. Rahman, N. Ullah, and S. Huda, “A Novel Approach to Improve Software Defect Prediction Accuracy Using Machine Learning,” IEEE Access, vol. 11, pp. 60710–60726, 2023, doi: 10.1109/ACCESS.2023.3287326.
[26]. S. Cao, X. Sun, R. Widyasari, D. Lo, X. Wu, L. Bo, J. Zhang, B. Li, W. Liu, D. Wu, and Y. Chen, “A Systematic Literature Review on Explainability for Machine/Deep Learning-Based Software Engineering Research,” arXiv, arXiv:2401.14617, Jan. 2024.
[27]. S. Wattanakriengkrai, P. Thongtanunam, C. Tantithamthavorn, H. Hata, and K. Matsumoto, “Predicting Defective Lines Using a Model-Agnostic Technique,” IEEE Trans. Softw. Eng., Sep. 2020, doi: 10.1109/TSE.2020.3023177
[28]. C. Chai, G. Fan, H. Yu, et al., “Exploring Better Alternatives to Size Metrics for Explainable Software Defect Prediction,” Softw. Qual. J., vol. 32, pp. 459–486, 2024.
[29]. L. Schulte, B. Ledel, and S. Herbold, “Studying the Explanations for the Automated Prediction of Bug and Non-Bug Issues Using LIME and SHAP,” Empir. Softw. Eng., vol. 29, no. 93, 2024, doi: 10.1007/s10664-024-10469-1.
[30]. A. Jude and J. Uddin, “Explainable Software Defects Classification Using SMOTE and Machine Learning,” Ann. Emerg. Technol. Comput. (AETiC), vol. 8, no. 1, pp. 36–49, 2024.
[31]. T. O. Olaleye, D. A. Aborishade, O. Arogundade, A. Abayomi-Alli, and O. J. Adeniran, “Multilayer Perceptron of Software Complexity Metrics for Explainable Multicollinearity Mitigation and Defect Localisation,” Cureus J. Comput. Sci., vol. 2, no. 1, 2025, doi: 10.7759/s44389-024-02871-z.
[32]. X. Wu, W. Zheng, X. Xia, and D. Lo, “Data Quality Matters: A Case Study on Data Label Correctness for Security Bug Report Prediction,” IEEE Trans. Softw. Eng., Jul. 2022, doi: 10.1109/TSE.2021.3063727.
[33]. R. Malhotra and K. Khan, “OpTunedSMOTE: A Novel Model for Automated Hyperparameter Tuning of SMOTE in Software Defect Prediction,” Intell. Data Anal., vol. 29, no. 3, pp. 787–807, 2025, doi: 10.1177/1088467X241301390.
[34]. R. He, Y. Li, and C. Sun, “Understanding Software Defect Prediction Through eXplainable Neural Additive Models,” IEEE Access, vol. 13, pp. 82860–82873, 2025, doi: 10.1109/ACCESS.2025.3567140.
[35]. S. Kabir, M. S. Hossain, and K. Andersson, “A Review of Explainable Artificial Intelligence from the Perspectives of Challenges and Opportunities,” Algorithms, vol. 18, no. 9, Art. no. 556, Sep. 2025, doi: 10.3390/a18090556
[36]. V. Chamola, V. Hassija, A. R. Sulthana, D. Ghosh, D. Dhingra, and B. Sikdar, “A Review of Trustworthy and Explainable Artificial Intelligence (XAI),” IEEE Access, vol. 11, pp. 78994-79012, 2023, doi: 10.1109/ACCESS.2023.3294569
[37]. M. J. Page et al., “PRISMA 2020 Explanation and Elaboration: Updated Guidance and Exemplars for Reporting Systematic Reviews,” BMJ, vol. 372, p. n160, 2021, doi: 10.1136/bmj. n160.
[38]. M. Begum et al., “LCNN: Lightweight CNN Architecture for Software Defect Feature Identification Using Explainable AI,” IEEE Access, vol. 12, pp. 55744–55756, 2024, doi: 10.1109/ACCESS.2024.3388489.
[39]. J. Humphreys and H. K. Dam, “An Explainable Deep Model for Defect Prediction,” in Proc. IEEE/ACM 7th Int. Workshop Realising Artificial Intelligence Synergies in Software Engineering (RAISE), Montreal, QC, Canada, 2019, pp. 49–55, doi: 10.1109/RAISE.2019.00016.
[40]. D. Li, Y. Liu, and J. Huang, “Assessment of Software Vulnerability Contributing Factors by Model-Agnostic Explainable AI,” Mach. Learn. Knowl. Extr., vol. 6, no. 2, pp. 1087–1113, May 2024, doi: 10.3390/make6020050.
[41]. T. Ganz, Software Defect Localization Using Explainable Deep Learning, Ph.D. dissertation, Fakultät IV – Elektrotechnik und Informatik, Technische Universität Berlin, Berlin, Germany, 2024.
[42]. I. Siachos, N. Kanakaris, and N. Karacapilidis, “Software Bug Prediction Using Graph Neural Networks and Graph-Based Text Representations,” Expert Syst. Appl., vol. 259, Art. no. 125290, 2025, doi: 10.1016/j.eswa.2024.125290.
[43]. N. S. Bommi and A. Negi, “A Standard Baseline for Software Defect Prediction: Using Machine Learning and Explainable AI,” in Proc. IEEE 47th Annu. Comput., Softw., Appl. Conf. (COMPSAC), Torino, Italy, 2023, pp. 1798–1803, doi: 10.1109/COMPSAC57700.2023.00278.
[44]. M. Mustaqeem, S. Mustajab, M. Alam, F. Jeribi, S. Alam, and M. Shuaib, “A Trustworthy Hybrid Model for Transparent Software Defect Prediction: SPAM-XAI,” PLoS ONE, vol. 19, no. 7, Art. no. e0307112, Jul. 2024, doi: 10.1371/journal.pone.0307112.
[45]. A. P. Meyer, A. Albarghouthi, and L. D'Antoni, “The Dataset Multiplicity Problem: How Unreliable Data Impacts Predictions,” in Proc. 2023 ACM Conf. Fairness, Accountability, Transpar. (FAccT), Chicago, IL, USA, Jun. 2023, pp. 320–331, doi: 10.1145/3593013.3593988.
[46]. Ł. Radliński, “The Impact of Data Quality on Software Testing Effort Prediction,” Electronics, vol. 12, no. 7, Art. no. 1656, Mar. 2023, doi: 10.3390/electronics12071656.
[47]. T. N. Al-Hadidi and S. O. Hasoon, “Software Defect Prediction Using Extreme Gradient Boosting (XGBoost) with Optimisation Hyperparameter,” Al-Rafidain J. Comput. Sci. Math., Jan. 2024, doi: 10.33899/CSMJ.2023.142739.108.
[48]. D. Al-Fraihat, Y. Sharrab, A.-R. Al-Ghuwairi, H. Alshishani, and A. Algarni, “Hyperparameter Optimisation for Software Bug Prediction Using Ensemble Learning,” IEEE Access, Mar. 2024, doi: 10.1109/ACCESS.2024.3380024.
[49]. M. Das, B. R. Mohan, R. M. R. Guddeti, and N. Prasad, “Hybrid Bio-Optimized Algorithms for Hyperparameter Tuning in Machine Learning Models: A Software Defect Prediction Case Study,” Mathematics, vol. 12, no. 16, p. 2521, Aug. 2024, doi: 10.3390/math12162521.
[50]. I. D. Raji, H. Bello-Salau, I. J. Umoh, A. J. Onumanyi, M. A. Adegboye, and A. T. Salawudeen, “Simple Deterministic Selection-Based Genetic Algorithm for Hyperparameter Tuning of Machine Learning Models,” Appl. Sci., vol. 12, no. 3, Art. no. 1186, Jan. 2022, doi: 10.3390/app12031186.
[51]. C. Antal-Vaida, “Basic Hyperparameters Tuning Methods for Classification Algorithms,” Inform. Econ., vol. 25, no. 2, pp. 64–74, 2021, doi: 10.24818/issn14531305/25.2.2021.06.
[52]. S. Haldar and L. F. Capretz, “Explainable Software Defect Prediction from Cross-Company Project Metrics Using Machine Learning,” in Proc. 7th Int. Conf. Intell. Comput. Control Syst. (ICICCS), Madurai, India, 2023, pp. 150–157, doi: 10.1109/ICICCS56967.2023.10142534.
[53]. L. Madeyski and S. Stradowski, “Predicting Test Failures Induced by Software Defects: A Lightweight Alternative to Software Defect Prediction and Its Industrial Application,” J. Syst. Softw., vol. 222, Art. no. 112360, 2025, doi: 10.1016/j.jss.2024.112360.
[54]. P. Tadapaneni, N. C. Nadella, M. Divyanjali, and Y. Sangeetha, “Software Defect Prediction Based on Machine Learning and Deep Learning,” Proc. 2022 Int. Conf. Inventive Comput. Technol. (ICICT), Nepal, 2022, pp. 116–122, doi: 10.1109/ICICT54344.2022.9850643.
[55]. W. Albattah and M. Alzahrani, “Software Defect Prediction Based on Machine Learning and Deep Learning Techniques: An Empirical Approach,” AI, vol. 5, no. 4, pp. 1743–1758, Sep. 2024, doi: 10.3390/ai5040086.
[56]. L. Chmielowski, M. Kucharzak, and R. Burduk, “Application of Explainable Artificial Intelligence in Software Bug Classification,” IAPGOŚ, no. 1, pp. 14–17, 2023, doi: 10.35784/iapgos.3396.
[57]. M. Begum, M. H. Shuvo, I. Ashraf, A. A. Mamun, J. Uddin, and M. A. Samad, “Software Defects Identification: Results Using Machine Learning and Explainable Artificial Intelligence Techniques,” IEEE Access, vol. 11, pp. 132750–132765, 2023.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Information Systems and Informatics
This work is licensed under a Creative Commons Attribution 4.0 International License.
Author's Declaration
- The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
- The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
- The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
- The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
- The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
- The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
- If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
- The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights
