A Simple Adversarial Attack against Code Completion Engines Based on Large Language Models

Abstract

The large language model-driven code completion engines have demonstrated an significant capability to generate functionally correct code based on context. However, these code-completion engines risk being exploited through black-box attacks. We propose a simple yet practical Adversarial attack against Black-box Code Completion engines (ABCC). This novel attack method aims to steer code completion engines to generate vulnerable code. Consistent with most commercial completion engines, ABCC assumes only black-box query access to the target engine without needing knowledge of the engine’s internal structure. Our attack is executed by inserting malicious attack strings as brief comments within the completion input. Firstly, we generate attack strings using large language models based on the expected malicious code. Then, using these attack strings, we guide the code completion engine to produce the desired malicious code. We validated our approach on the state-of-the-art black-box commercial service OpenAI API. In security-critical test cases covering 12 types of CWEs, ABCC significantly increased the likelihood of the targeted completion engine generating unsafe code, with an absolute increase exceeding with a success rate of 277.7%, which is significantly higher than the baseline model GPT-3.5-Turbo-Instruct when it completes code without using prompts.