The Filter AI / Sources

Source desk.

Everything on this site is built on work by other people. This is that work, graded by what kind of evidence it actually is, and each entry says what it does not prove as well as what it does. If something here contradicts something elsewhere on the site, this page wins.

19 sources 3 evidence grades Every link resolved 31 Aug 2026
How grading works

Three grades, and what they mean.

A grade describes the kind of evidence, not how good the work is. Some of the most useful writing in this field is grade C, and some grade A papers have been superseded. The grade tells you how much weight a claim can carry on its own.

GradeMeansUse it for
APeer-reviewed research, a published standard, or a reproducible measurement with a public method.Load-bearing claims. Cite it in a design review.
BFirst-party documentation from the people who built the thing. Accurate about behaviour, interested in the outcome.How something works. Not how well it works.
CPractitioner commentary, taxonomies and framing. Often the clearest explanation available, and not a measurement.Vocabulary and intuition. Never as proof.
Standards

Standards and taxonomies.

Shared vocabulary. Their value is less in the content than in the fact that other people have read the same document.

A

OWASP Top 10 for LLM Applications, 2026

OWASP GenAI Security Project · 2026 edition

The reference list of risks in LLM applications, and the naming scheme used throughout this site. Prompt injection has held the top position across every edition. The 2026 revision renumbered eight of the ten entries, moved excessive agency up to third, dropped improper output handling to tenth, and renamed system prompt leakage to hidden context exposure to cover everything an application holds and never shows you. Where a number changed, this site prints the old one beside it.

Does not prove: that these are the ten most frequent risks in the wild — it is expert consensus weighted with incident data, not telemetry, and it is a list of risks rather than a set of controls. Also note: the 2026 ordering used here was taken from the published edition summary and cross-checked against two independent write-ups that agree exactly. OWASP’s own per-risk pages under genai.owasp.org/llm-top-10 still carried 2025 numbering when this was checked on 31 August 2026, so deep links to individual risks point at the 2025 slugs.

A

NIST AI 100-2e2025 · Adversarial Machine Learning

NIST · 2025

The most rigorous taxonomy of attacks and mitigations available, spanning predictive and generative systems, with a literature base of several hundred references. The 2025 edition adds explicit treatment of AI agents and of misuse violations. PDF.

Does not prove: that any listed mitigation works in your system. It is a taxonomy and a survey — deliberately descriptive, not prescriptive.

A

MITRE ATLAS

MITRE · continuously updated

ATT&CK-style tactics and techniques for attacks on AI systems, with case studies drawn from real incidents. Useful when you need to describe an attack chain to people who already think in ATT&CK.

Does not prove: prevalence. Case study coverage reflects what was published and disclosed, which is a small and unrepresentative slice.

C

The Arcanum Prompt Injection Taxonomy

Jason Haddix, Arcanum Information Security · v1.6

The most granular open classification of prompt injection specifically: intents, techniques and evasions, each with a citable code and cross-references to how OWASP, ATLAS and NIST name the same thing. Repository, with the data as plain JSON.

Does not prove: effectiveness or frequency. It is a naming system built from practitioner experience, which is exactly what it claims to be — and what makes it useful when OWASP's ten categories are too coarse.

Foundations

Foundational papers.

A

Universal and transferable adversarial attacks on aligned language models

Zou et al. · 2023

Demonstrated automatically generated suffixes that transfer across models, including ones the attack was never optimised against. The reason "we fixed the jailbreaks we knew about" is not a security posture.

Does not prove: that current models are equally vulnerable. Specific published strings get trained against; the search procedure that produced them does not go away.

Defences

Defences with published methods.

A

Defending against indirect prompt injection attacks with spotlighting

Hines et al., Microsoft · 2024

Formalises marking untrusted content so the model can distinguish it from instructions: delimiting, datamarking and encoding. The basis of DP-05, and the cheapest structural improvement most teams can make.

Does not prove: robustness against an adaptive attacker. It raises the cost of an attack. Do not make it load-bearing.

A

StruQ: defending against prompt injection with structured queries

Chen et al. · 2024

Separates instructions from data at the model level by training on a structured query format, rather than asking a general model to respect a convention at inference time.

Does not prove: applicability to hosted models. The approach requires influence over training, which most application teams do not have.

A

Defeating prompt injections by design (CaMeL)

Debenedetti et al. · 2025

Extracts control flow from the trusted query and runs it in an interpreter, with capabilities attached to data so untrusted values cannot redirect execution. The strongest containment story published so far, and honest about the capability it costs. Basis of DP-12. Code.

Does not prove: practicality at product scale. The evaluation is on a benchmark, the engineering cost is substantial, and the paper documents residual channels rather than claiming a total solution.

C

The dual LLM pattern

Simon Willison · 2023

The clearest short description of splitting a system into a privileged model that holds tools and a quarantined model that reads untrusted content. Basis of DP-11.

Does not prove: anything empirically. It is a design proposal, and the author says so; its value is that it makes the architecture obvious once you have read it.

Evaluation

Evaluation and skepticism.

The papers that stop you over-trusting the section above. Every one of these is a reason to prefer containment over detection.

Tooling

Tooling you can actually run.

Grade B throughout: first-party documentation from the people who built it. Accurate about behaviour, interested in the outcome.

B

LLM Guard

Protect AI · open source

Composable input and output scanners — injection detection, secret detection, PII, topic limits — with a readable enough codebase to learn from even if you do not deploy it.

Does not prove: its own detection quality. Measure on your traffic, and log the allows as well as the blocks or you cannot compute precision at all.

B

PurpleLlama (Llama Guard, Prompt Guard, CyberSecEval)

Meta · open weights and tools

Open classifier models for input and output moderation, plus a security evaluation suite. Useful as a self-hosted baseline when sending traffic to a third-party scoring API is not acceptable.

Does not prove: coverage outside its training distribution — notably in lower-resource languages, where a detection gap becomes a fairness problem.

B

NeMo Guardrails

NVIDIA · open source

Programmable rails for conversational flow, topic boundaries and tool use, expressed as configuration rather than as prompt text. Sits closer to the constrain shape than the filter shape.

Does not prove: containment. Rails constrain a cooperative model; they are not an authorisation boundary and should not be mistaken for one.

B

Azure AI Content Safety · Prompt Shields

Microsoft · product documentation

A managed classifier for direct and indirect injection. The documentation is worth reading even on another platform, because it distinguishes the two delivery routes clearly at the API surface.

Does not prove: published detection figures on your data. Vendor evaluation sets are not your traffic distribution.

C

Gandalf

Lakera · free browser game

The best known attacker-seat game: extract a password through escalating defences. Genuinely instructive about how brittle prompt-level protection is, and it is where most people's intuition about this comes from.

Does not prove: defensive skill. It trains the offensive seat — which is precisely the gap The Filter AI exists to fill.

Commentary

Ongoing commentary worth following.

C

Simon Willison on prompt injection

Ongoing since 2022

The longest-running practitioner record of the problem, including the coining of the term. If you want to understand why the field has not solved this, reading it in order is the fastest route.

Does not prove: anything on its own. It is commentary, consistently well-sourced, and it links to the primary work.

C

The lethal trifecta

Simon Willison · 2025

The framing that an agent is dangerous when it combines access to private data, exposure to untrusted content, and a way to communicate externally. The most useful triage question available, and the reasoning behind DP-04.

Does not prove: completeness. Destructive actions need no outbound channel at all, so removing the third leg reduces exfiltration rather than risk in general.

Method

Method, and what is missing.

Every link on this page was resolved on 31 August 2026. Sources are included when they are primary — the standard itself, the paper itself, the maintainer's own documentation — rather than a summary of one. Where this site makes a claim that rests on a single source, that source is linked at the point of the claim, not only here.

Gaps worth naming, because a source list that looks complete is lying:

  • No incident telemetry. There is no public dataset of real prompt injection incidents with frequencies and outcomes. Every prevalence statement in this field, including any you find here, is an estimate.
  • Detection quality is under-measured outside English. The gap is widely acknowledged and thinly quantified, which is why the language switching entry treats it as a fairness problem rather than only a coverage one.
  • False positive rates are almost never published. Vendors report detection; the second number is left to you. That absence is the reason the base rate calculator exists.
  • Agentic defences are young. AgentDojo and CaMeL are recent, and the adaptive-attack literature is younger still. Expect this section to age fastest.

Spotted something wrong, out of date, or missing? Get in touch — corrections to this page are more welcome than compliments about the others.

The sources explain it. The exercises make it stick.

The Filter AI turns this material into 35 scored decisions across seven units, every one anchored to a risk in the OWASP LLM Top 10. Unit one is free.

Get the full training $39.90 Start with the field guide