In the context of Adversarial Machine Learning, Jailbreaking is technically distinct from Direct Prompt Injection because Jailbreaking specifically…

The statement is true. It accurately distinguishes between two types of adversarial attacks on Large Language Models (LLMs). Jailbreaking uses carefully crafted prompts to bypass the model's safety and ethics alignment training, tricking it into generating content that violates its established policies. Direct Prompt Injection, on the other hand, involves providing malicious instructions that override or poison the original system prompt, causing the model to execute an unintended task, such as revealing sensitive information or performing an unauthorized action.

False: This option is incorrect because the described distinction is a widely accepted technical differentiation in the field of adversarial machine learning for LLMs.

1. Wei, A., et al. (2023). Jailbroken: How Does LLM Safety Training Fail? arXiv preprint arXiv:2307.02483. (This academic paper specifically analyzes the mechanisms of jailbreaking as a failure of safety alignment). https://doi.org/10.48550/arXiv.2307.02483

2. Perez, F., & Ribeiro, I. (2022). Ignore Previous Prompt: Attack Techniques For Language Models. (This work is foundational in defining and demonstrating prompt injection attacks that overwrite original instructions).

3. OWASP Foundation. (2023). OWASP Top 10 for Large Language Model Applications. LLM01: Prompt Injection. (The official OWASP documentation distinguishes between direct and indirect prompt injection as a primary vulnerability class, separate from alignment-breaking attacks).