Follow the action, not only the prompt
Prompt injection and jailbreaks
Test whether hostile content changes authority, task or refusal behaviour.
Unsafe tool use and data leakage
Inspect attempted actions and information flows across permission and data boundaries.
Authority impersonation
Check whether apparent instructions from an executive, administrator or tool response receive authority they do not have.
Delegation failures
Examine handoffs of work, context and permissions between agents.
Boundary violations and multi-step attack paths
Look beyond one reply to sequences combining retrieval, memory, tool calls and downstream effects.
From threat hypothesis to reviewable evidence
Agree the boundary
Define permitted targets, test data, environments and stop conditions.
Map a misuse path
Describe the potential business harm, attacker influence and control that should interrupt it.
Run a controlled case
Capture inputs, actions and observations through approved test access.
Investigate the finding
Distinguish a model response from an actual tool action or confirmed disclosure.
Create a regression case
Preserve a reproducible version of the meaningful failure and expected corrected behaviour.
Review and repeat
Record limitations and repeat relevant cases after material change.
Three outputs that should not be confused
| Work | Contribution | Limit |
|---|---|---|
| One-off security probing | Finds possible weaknesses and unexpected paths. | Does not establish repeatable coverage. |
| Repeatable evaluation cases | Specify setup, expected behaviour and observable results. | Do not cover every threat or context. |
| Enterprise review evidence | Connects findings, criteria, exact versions and decisions. | Does not automatically establish legal conformity. |
Make the test safe enough to be useful
Use approved environments and test identities. Confirm which actions can execute, which must be simulated and who can stop the run. An assessment that creates avoidable operational harm has not respected the enterprise boundary.
The workflow matters. A prompt may look harmless while a later tool call crosses a boundary. A concerning reply may not prove that a protected action occurred. Evidence should make that distinction explicit.
Prioritize findings in the context of intended use
Review the affected boundary, potential consequence and reliability of reproduction. Record untested paths and unresolved dependencies. Remediation needs an owner, a changed control and an evaluation that justifies closing the issue. Use broader red teaming alongside focused prompt injection testing; they answer different questions.
Questions about ai agent red teaming
Does red teaming prove there are no vulnerabilities?
No. It provides observations from an agreed scope and method. Unknown paths and untested conditions remain limitations.
When should cases be repeated?
Repeat relevant cases after material changes to prompts, tools, permissions, dependencies or operating scope, and when a credible new threat is identified.