Efficient Integration Method of Large-Scale Heterogeneous Security Logs Using NoSQL in Cloud Computing Environment

Abstract

The cloud computing environment has expanded considerably with the rapid advancement of related technologies. Although cloud computing is convenient for users, detecting and preventing possible security breaches remains an unsolved problem. Security logs are critical data that indicate events in an operating system or other software, and these data are stored through heterogeneous machines such as network security devices, server systems, and database management systems (DBMS). However, existing methods can create problems for efficient analysis because of large-scale heterogeneous security logs in the cloud-computing environment. Therefore, because cloud computing provides various services to users, an efficient integration method of security logs must be developed. This study proposes a NoSQL-based method to collect and integrate security logs using MapReduce. Our study shows that log data were reduced by more than 87% when integrating duplicate large-scale security logs. This proposed method provides faster data storage than conventional DBMS and is more effective.