ACO-HCO: Heuristic Performance Tuning Scheme for the Hadoop MapReduce Architecture

Chiang-Lung Liu,
Hsiang-Fu Lo,
Wei-Tsong Lee,


Hadoop MapReduce is a widely-used cloud computing technology for big data processing. However, the Hadoop configuration parameters settings can significantly change the execution performance. Manual adjustment of the Hadoop parameters will be a time consuming and difficult task. In this paper, we propose ACO-HCO, a Hadoop configuration tuning scheme for MapReduce applications. We use MapReduce applications job history records to generate specific job profiles. Based on these profiles, an objective function for execution time is constructed with gene expression programming algorithm by mining the correlation among the core Hadoop configuration parameters and input data size. Leveraging the objective function, an ACO-based configuration optimizer is able to heuristically search for the optimal configuration for a given application. Experimental results show that ACO-HCO enhances the performance of Hadoop significantly compared with the default configuration. Moreover, ACO-HCO performs better than heuristic approach and the cost-based model in Hadoop performance tuning.

Citation Format:
Chiang-Lung Liu, Hsiang-Fu Lo, Wei-Tsong Lee, "ACO-HCO: Heuristic Performance Tuning Scheme for the Hadoop MapReduce Architecture," Journal of Internet Technology, vol. 21, no. 4 , pp. 1151-1159, Jul. 2020.

Full Text:



  • There are currently no refbacks.

Published by Executive Committee, Taiwan Academic Network, Ministry of Education, Taipei, Taiwan, R.O.C
JIT Editorial Office, Office of Library and Information Services, National Dong Hwa University
No. 1, Sec. 2, Da Hsueh Rd., Shoufeng, Hualien 974301, Taiwan, R.O.C.
Tel: +886-3-931-7314  E-mail: