Remote Prompt Injection in GitLab Duo Leads to Source Code Theft
Source: https://www.promptarmor.com/gitlabduo Author: Omer Mayraz (PromptArmor) Date: 2025-05-28
Summary
Security research showing a remote prompt injection vulnerability in GitLab Duo (AI coding assistant). Malicious instructions embedded in repository content could redirect Duo to exfiltrate source code. Real-world demonstration of the prompt injection threat.
Key Claims
- Attack vector: attacker adds malicious instructions to a file or comment in the repository. When a developer asks GitLab Duo about the codebase, Duo reads the malicious instructions and follows them.
- Impact: Duo exfiltrated source code from private repositories to attacker-controlled endpoint.
- The fundamental vulnerability: any AI assistant that reads user-controlled content (documents, code, emails) as part of its task is vulnerable to prompt injection.
- Patch: GitLab implemented output filtering and instruction-following restrictions, but acknowledged this is a partial mitigation.
- Broader lesson: this is not a GitLab-specific bug — any LLM-integrated tool that reads external content has this attack surface.
Connection to Other Sources
Concrete example of the threat Steve Newman warned about (Windows 95 era of AI security). Real-world instantiation of context pollution concerns.
Concepts
- Coding Agents — coding assistants as attack surface
- AI Alignment — prompt injection as alignment failure (agent follows wrong instructions)