Image Sequence Facial Expression Recognition Based on Deep Residual Network

Abstract

A sequence of facial expression images can provide rich texture information and motion information about facial expression changes. Combining traditional manual designed feature extraction methods with learning-based methods, this paper proposes an image sequence facial expression recognition algorithm based on deep residual network. Feature extraction is performed for each frame image, where the local binary pattern (LBP) map of the facial expression image is used as the input of the network, and the deep residual network model is used as the feature extractor for the image sequence. Then, each frame image feature is connected to a feature vector as the feature representation of the image sequence. Further, the image sequence is used as the input of the long short-term memory (LSTM) network, and the classification result is obtained through model training. Experimental results demonstrate the effectiveness of the proposed algorithm, where high recognition rates are observed based on both FER-2013 and AFEW6 datasets.