AI-Based Multimodal Anomaly Detection for Industrial Machine Operations

Abstract

In the manufacturing process involving grinding wheels, challenges in fine-tuning grinding machines are typically addressed by craftsmen through subjective observations of sparks and sounds. However, most current anomaly detection methods mainly aim at a single modality, whereas existing multimodal methods cannot effectively cope with a common issue. To address this, this paper introduces an innovative mechanism, AI-Based Multimodal Anomaly Detection (AMAD), designed to optimize the efficiency and accuracy of grinding wheel production lines. The proposed AMAD includes data preprocessing and multimodal anomaly detection, accurately identifying anomalies in grinding wheel operation videos. In the data preprocessing phase, the proposed AMAD utilizes Mel Frequency Cepstral Coefficients (MFCC) and AutoEncoder for audio processing and segmentation for video processing. In the multimodal anomaly detection phase, the proposed AMAD employs Convolutional Neural Networks (CNN) for audio analysis and Convolutional Long Short-Term Memory (ConvLSTM) for video analysis. By combining both audio and video modalities, the proposed AMAD effectively predicts whether the input video represents normal or abnormal grinding wheel operations. This multimodal approach not only improves the accuracy of anomaly detection but also enhances the robustness of the system. Simulation results demonstrate that the proposed AMAD significantly improves performance in anomaly detection in terms of precision, recall, and F1-Score.