The Filter AI / Build
How a model is trained, how an application is assembled around it, and what the ecosystem is made of — written for the person who has to defend the result. Not a tutorial for shipping one. A machine you cannot name the parts of is a machine you cannot secure.
Most security material about AI starts at the chat box. That is roughly like learning web security by studying the login form: you will recognise the obvious problems and miss every structural one.
Three things change once you know how the machine is put together.
The one thing to carry through the whole page
Nothing in the training pipeline creates a channel that separates instructions from data, and nothing in the application stack adds one either. Every layer below is built on a component that cannot tell your rules apart from the document it was handed. The rest is engineering around that fact.
Six stages, in order. The security-relevant point is which stage produces which behaviour, and who controls it — because you inherit the first five and only really operate the sixth.
Assembling the training data: crawled web text, books, code, licensed sets, filtered and deduplicated. The largest and least visible stage.
web crawlcode reposlicensed datafilteringdedup
You inherit: whatever was in there. Memorised secrets, licence problems, and the raw material for data poisoning. Maps to: LLM05, LLM04.
Next-token prediction across the corpus, at enormous scale. The result is a base model: a system that continues text plausibly and has no concept of a request, a rule, or a user.
next-token objectivetransformerweeks of computebase model
Where the defect is born. A base model has exactly one input: a sequence. Not a privileged sequence and a subordinate one. One. Everything downstream is an attempt to impose a hierarchy on a component that has no notion of one.
Training on demonstrations of instruction-following — prompt and ideal response pairs — so the base model behaves like an assistant rather than an autocomplete.
instruction datademonstrationschat templaterole tokens
You inherit: the appearance of an instruction channel. Role markers like system and user are learned conventions in the token stream, not enforced boundaries — which is precisely why forging them works. Maps to: LLM01.
Optimising against human or model judgements of which response is better — reward modelling with reinforcement learning (InstructGPT), direct methods such as DPO, or AI feedback against a written policy (Constitutional AI).
reward modelRLHF / DPOrefusal behaviourhelpfulness
You inherit: the refusal behaviour, and the tension inside it. This stage is trained to be both helpful and harmless, and a jailbreak is an argument that the helpful reading should win. Maps to: the whole jailbreak literature.
Explicitly training the model to rank sources of instruction: platform above developer, developer above user, user above tool output. OpenAI's instruction hierarchy paper is the clearest public statement of the approach.
privilege rankingadversarial trainingred team datarefusal tuning
You inherit: a real, measurable improvement and not a guarantee. It is a learned preference, so it holds probabilistically. Treat it as defence in depth, never as an authorisation boundary.
What happens after the lab ships it: fine-tunes, LoRA adapters, distillation, quantisation, serialisation into a file, publication to a hub, and versioning behind an API.
fine-tuneLoRAquantisationsafetensorsmodel version
Your first real decision point. A community adapter is executable dependency. Prefer safetensors over pickle formats, pin versions, and review provenance. Maps to: LLM04.
The system prompt, tool schemas, sampling settings, context window, and any structured-output constraints. Applied per request, at run time.
system prompttool schemastemperaturecontext window
The only stage you fully control — and the weakest one, because everything here is text or configuration handed to a component that treats text as text. It is where most teams put their security policy, and it is the wrong place for it.
The consequence people skip
Stages 02 to 04 all teach the model to prefer some instructions over others. None of them build a mechanism that enforces it. A preference expressed by a probability distribution can be argued with, and an attacker gets unlimited attempts while you get one deployment. That is the entire reason the pattern library builds isolation before detection.
Whatever framework a team has chosen, and whatever they call it, every LLM application is assembled from the same six parts. Learn these and you can read any architecture diagram in this field, including one drawn by someone using entirely different words.
| Primitive | What it is | The question to ask | Layer |
|---|---|---|---|
| 01 Model |
The component that predicts. A hosted endpoint or weights you run yourself. | Which version, pinned how, and what did stage 05 do to it? | 01 |
| 02 Context |
Everything assembled into the prompt for this one turn. System text, user text, retrieved documents, tool results. | Which bytes in here did we author, and which arrived from outside? | 02 · 03 |
| 03 Memory |
What survives between turns and between sessions. History, summaries, saved profiles, vector stores. | Who can write into it, and does an injection written today fire next week? | 03 |
| 04 Tools |
The functions the model can invoke. Search, database reads, email, shell, another agent, an MCP server. | Whose credentials, what scope, and which of these cannot be undone? | 04 · 06 |
| 05 Loop |
The controller that decides whether to call the model again. Turns a single completion into an agent. | What stops it, who pays for it, and can retrieved content extend it? | 04 |
| 06 Deploy |
Where it runs and what it is attached to. Identity, network egress, logging, the surface a user sees. | What can this process reach on the network, and what did we record about the decision? | 05 · 06 · 07 |
A useful review habit: for any system someone shows you, name all six out loud before asking a single security question. Two thirds of the findings fall out of the naming exercise, because the parts nobody can describe are the parts nobody owns.
Here is what actually happens when someone types a sentence into an agentic assistant. Colour marks provenance: green is content you authored, red is content from outside, amber is mixed and needs a decision.
Five things worth noticing in that trace, all of which are invisible in a product demo.
Almost every product in this space is one of these four, or a composition of them. Each entry gives the shape, where untrusted content enters, how far a successful injection reaches, and the minimum set of patterns worth arguing for before it ships.
A system prompt, the user's message, and a response. No retrieval, no tools, no memory beyond the conversation. Support chat widgets and writing helpers usually start here.
The user's own message, and nowhere else. This is the only architecture where that statement is true.
Small. A successful injection gets the model to say something. Reputational and policy risk, rarely data loss — unless the system prompt held a secret.
DP-02 on the render surface, and nothing confidential in the prompt. That is genuinely most of it.
The user's question is embedded, similar chunks are fetched from a vector store, and those chunks are pasted into the context. The original RAG paper is from 2020; the pattern is now the default shape of an enterprise AI product.
The ingest path, which is usually wide open — anywhere a customer, partner or crawler can write is an input to your prompt. Also the retrieval path, if tenancy filtering is done in the prompt rather than in the query.
Everyone who asks a question that retrieves the poisoned chunk. One insertion, persistent, and it survives until someone cleans the index.
The model is offered a set of functions and a loop runs it repeatedly: think, call a tool, read the result, decide again. ReAct is the canonical description of the pattern.
Every tool result, on every pass of the loop. A search tool means the open web is an input. The door does not close after the first turn.
Everything the agent's credentials can reach. This is the architecture where injection stops being a content problem and becomes remote control of a process holding your tokens.
Several agents delegating to each other, and tools supplied by third parties over a protocol such as MCP. The tool catalogue becomes a dependency graph you did not write.
Tool results, tool descriptions from third-party servers, and every sub-agent summary — which arrives looking like internal output because the provenance label was dropped at the hop.
The union of every agent's permissions, which nobody has written down. Compromise one hop and you inherit the authority of whatever trusts its output.
How products actually move
Almost nobody designs A4. They ship A1, add retrieval because answers were thin, add a tool because users asked, then connect a second agent because one was slow. The architecture changes category without anyone declaring a new design. The useful review question is not "what is this" but "which of the four was it last quarter", because the controls were sized for that one.
The layers a team assembles, what each one is for, and — the column that matters here — what adding it does to your attack surface. Names are representative examples, not recommendations, and this part of the market changes faster than any other section on this site.
| Layer | What it does | Representative names | What it adds to your surface |
|---|---|---|---|
| Model providers | Train and serve frontier models behind an API. | Anthropic, OpenAI, Google, Meta, Mistral | A dependency whose behaviour changes on their schedule. Pin versions and read the deprecation notes. |
| Self-hosted serving | Run open weights on your own hardware. | vLLM, Ollama, TGI, llama.cpp | You now own the supply chain: weight provenance, serialisation format, and the inference server's own CVEs. |
| Orchestration | Frameworks that assemble context, run the loop and call tools. | LangChain, LlamaIndex, agent SDKs | The prompt-assembly layer becomes someone else's code. Know where it interpolates untrusted text, because it will not tell you. |
| Protocols | Standard interfaces between models and external tools. | MCP, function calling schemas | Tool descriptions written by third parties, injected into your context. Review manifests like dependencies. |
| Retrieval and stores | Embedding, indexing and similarity search. | pgvector, Pinecone, Weaviate, Elastic | A writable input to your prompt, plus a tenancy boundary that is usually enforced by a query filter you have not audited. |
| Gateways and proxies | One egress point for model traffic: keys, routing, quotas, logging. | LiteLLM, cloud AI gateways | Mostly reduces surface — it is the natural home for budgets, logging and egress rules. Also a single point of failure holding every key. |
| Guardrails | Classify input and output, enforce topic and format rules. | Llama Guard, NeMo Guardrails, LLM Guard, managed shields | Latency, spend, and a false positive rate that scales with traffic. Useful. Not an authorisation boundary — see DP-09. |
| Evaluation | Measure task quality and security regressions before shipping. | AgentDojo, promptfoo, in-house suites | Reduces surface, and is the only way you will notice a defence degrading after a model version bump. |
| Observability | Trace prompts, tool calls, costs and guardrail decisions. | OpenTelemetry-based tracing, LLM-specific tools | Reduces surface, with one caveat: traces contain prompts, and prompts contain customer data. Treat the trace store like the database. |
Reading the column that matters
Six of those nine layers add attack surface, and three reduce it. Notice which three: the gateway, the evals and the tracing — the unglamorous operational layers, the ones cut first when a launch date moves. That is not a coincidence, and it is worth saying out loud in a planning meeting.
Having walked the whole stack, here is the honest division. It is shorter than people expect on one side and longer on the other.
Training data. Model behaviour. Refusal tuning. How well the instruction hierarchy holds. Detection quality in each language. When the vendor deprecates a version.
Model and version choice. System prompt. Which tools exist. Whether output is structured. Which guardrail runs and at what threshold.
Provenance labelling. Credential scope. Tenancy filtering. Egress rules. Loop ceilings. Output encoding. What gets logged. Which actions need a human. Whether a wrongly blocked user can appeal.
Every item in the third column is engineering you can do this quarter without a model vendor's cooperation, and every one of them holds when the injection works. That is the argument this whole page has been building toward, and it is why the pattern library is ordered the way it is.
No. Nothing on this page requires you to read a loss function. What it requires is that you can name the six primitives in a system someone shows you, and say which of the four architectures it currently is. That is a vocabulary skill, not a mathematics one.
Only incidentally. It is the mental model a defender needs to review one, threat model it, and argue for controls in the right place. If you want to actually ship an agent, the frameworks in the ecosystem table have their own documentation and it is better than anything I would write here.
Because it tells you what you are allowed to rely on. Knowing that the instruction hierarchy is trained preference rather than enforced boundary is the difference between putting your authorisation rules in a prompt and putting them in code. That single distinction is worth the whole section.
The names will. The layers will not, and the surface column will not either — a retrieval store has been a writable input to your prompt since 2020 and will be in 2030. Read the table by column, not by row.
Whichever one has the most components you can actually name. If retrieval exists, it is at least A2. If anything the model emits causes a side effect, it is A3, regardless of what the team calls it. If a third party supplies a tool description, it is A4.
You now have the machine. The rest of the site is about defending it.
The same system viewed as seven security boundaries, with the OWASP LLM Top 10 mapped onto them and the arithmetic that decides whether a guardrail is worth deploying.
ThenTwelve controls, each one placed at a specific component from section 03. What it stops, what it does not stop, what it costs.
In parallelTwenty-four techniques, each entering at one of the points marked untrusted in the diagrams above.
Check the claimsThe papers behind section 02 — instruction hierarchy, preference tuning, RAG, ReAct — graded, dated, with what each one does not prove.
The Filter AI puts you on the receiving end of the architectures above: 35 exercises, seven units, every one scored on both numbers — the attacks you missed and the real people you blocked. Unit one is free.