Performance Predict Method Based on Neural Architecture Search

Abstract

Deep learning has granted remarkable breakthroughs on various tasks over the past few years, such as image segmentation, speech recognition, and nature language processing. One vital aspect of progress is the emergence of advanced neural architectures. However, Currently used architectures have frequently been developed manually by human experts, which is a time-consuming and laborious process. Because of this, more and more research is now involved in automated neural architecture search techniques.
This paper studies the process of Neural Architecture Search (NAS) technology, summarizes the previous work in this research field and classifies it according to three aspects: search space, search strategy, and acceleration method. In addition, this study selects the performance prediction method in the NAS acceleration strategy as a breakthrough direction and inherits the works of MetaQNN network structure and the sequential regression prediction model (SRMs), which were proposed by the previous research. Firstly, based on our hypothesis, we successfully use the idea of the N-grams model of natural language processing to extract the sequence features belonging to the chain neural network. Then, based on the extracted network structure features, referring to the steps of SRMs, we give a new recipe for predicting the accuracy score of neural network models on the training set. Finally, through experiments and comparison, we prove the accuracy of this prediction model and the effectiveness of accelerating the neural architecture search process.