LLM Red Teaming Methodology

LLM red teaming starts with capability, not prompts. Map what the model can read, what tools it can call and where its output lands, because that determines every possible impact. Then attack in order: direct injection, indirect injection through each untrusted content path, exfiltration, then unauthorised action. Report impact, never cleverness.

Most LLM testing you see online is prompt tricks with no impact analysis. That produces screenshots, not findings. This is the version that produces findings a client acts on.

Phase 1: Capability mapping

Before sending a single payload, answer these. The answers define the entire test.

A text only chatbot with no tools has a low ceiling no matter how injectable it is. An agent with email read and send is a complete exfiltration primitive on its own.

Phase 2: Direct injection baseline

Establish what the model will do when you ask it directly. Instruction override, role reassignment, system prompt extraction, encoding bypasses, payload splitting across turns.

This is the baseline, not the finding. Background in prompt injection explained.

Phase 3: Indirect injection, where the real findings are

Enumerate every path by which content you do not control enters the context, then plant a payload in each one.

• Uploaded documents and PDFs
• Web pages the model browses
• Emails and support tickets it reads
• Retrieved chunks from a vector store
• Code comments in a repository
• Calendar invites, product reviews, user profile fields

The victim here does nothing wrong. They asked their assistant to summarise a document, and the document told the assistant to do something else. That framing is what makes the finding land with a client.

Phase 4: Exfiltration

Getting data out is usually more valuable than getting the model to misbehave.

1. Rendered markdown. Instruct the model to embed conversation content in an image URL pointing at your server. The client renders it, the browser makes the request, the data leaves. No click needed.
2. Tool chaining. Read plus send in the same agent is an exfiltration path with no extra work.
3. Response channel. Simply getting the model to output data the user should not see.
4. Browsing. The model fetches a URL containing the data as a parameter.

Phase 5: Unauthorised action

Can an injected instruction make the agent do something the user would not authorise. Send a message, move money, change a setting, delete a record, call an internal API with the user's privileges.

Check authorisation at the tool layer too. An agent that can call an API is subject to the same object level authorisation problems as any other client.

Phase 6: Classical vulnerabilities around the model

The phase that produces the findings clients actually pay for, and the one prompt-only testers never reach.

• Model output flowing unsanitised into a shell, eval, SQL query or HTML context
• SSRF through browsing capability
• Broken access control on the AI feature endpoints themselves
• Authorisation gaps between the AI layer and the underlying data store

Evidence capture for a non deterministic system

LLMs do not reproduce identically, which makes evidence harder than in normal testing. What works:

• Record the full context, not just your prompt
• Run each successful payload several times and report the success rate honestly
• Capture the resulting tool call or network request, not only the model's text
• State the model version and settings, since behaviour changes between them

A finding that reproduces three times out of ten is still a finding. Say so rather than implying it is deterministic.

Reporting

Severity follows capability. Rate by what the attacker achieved, not by how clever the prompt was.

Weak: the model ignored its instructions. Strong: an email from an external sender caused the assistant to forward the user's inbox to an attacker controlled address, with no user interaction. Structure in how to write a VAPT report.

Practise this

Build a small agent with a tool, then attack your own agent. The gap between injecting successfully and achieving impact is the whole discipline. Covered end to end in the AI Security program.