An Empirical Study of Gradient-based Explainability Techniques for Self-admitted Technical Debt Detection

Abstract

Self-Admitted Technical Debt (SATD) is an intentionally introduced software code comment describing potential defects or other technical debt. Currently, deep learning is widely used in fields such as Natural Language Processing. Deep learning can be used for SATD detection, but there is a class imbalance problem and a large number of easily classified SATD instances that may potentially affect the loss value. As a result, we proposed a weighted focal loss function based on particle swarm to address the problem. Meanwhile, there is no empirical research based on local explanations for SATD detection. We have investigated local interpretation models such as Saliency Maps, Integrated Gradients, which are currently widely used in deep learning, and conducted empirical research for shared data sets. The research results show that our proposed weighted focal loss function can achieve the best performance for SATD detection; our model achieves 12.27%, 5.97%, and 5.62% improvement in Precision, Recall, and AUC compared to the baseline model, respectively; Local explanation models, including Saliency Maps and Integrated Gradients can cover nearly half of the manually labeled paradigms; these two interpretation models can also discover potential new paradigms.