Hierarchical Feature Selection with Orthogonal Transfer

Abstract

Feature selection is an indispensable preprocessing step in high-dimensional data classification, which has an effect on both the running time and the result quality of the subsequent classification processing steps. Most existing approaches use flat strategies, which treat each category or class separately and ignore hierarchical structure. In this paper, we propose a hierarchical feature selection algorithm with orthogonal transfer. We first compute the weight of the feature to the category by hierarchical SVM with orthogonal transfer. More specifically, we use an objective that is a convex function of the normal vectors to compute the weight. Then, we select features using the weight and predict the class label for a test sample according to classifier. Finally, extensive experimental results on various real-life datasets have demonstrated the superiority of the proposed algorithm.