Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations

Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations

NIST - the National Institute of Standards and Technology, a US government agency, released a 106 page report on attacks against modern machine learning models, mostly covering LLMs.

Prompt injection gets two whole sections, one on direct prompt injection (which incorporates jailbreaking as well, which they misclassify as a subset of prompt injection) and one on indirect prompt injection.

They talk a little bit about mitigations, but for both …

adversarial adversarial machine learning agency ai attacks generativeai government institute llms machine machine learning machine learning models modern nist page prompt prompt injection promptinjection report standards taxonomy technology terminology