An assistant asked to generate a valid AWS IAM policy may consult external examples. IAMRAGBench tests the effect of placing attacker-chosen guidance in the retrieved document context while retaining a legitimate policy-writing objective. In the included-context evaluation, the authors report 46 exact attacker goals achieved among 292 eligible attempts,…
Inspect the ClaimsCase · DiggingBeagle record
Poisoned reference text steered AI-generated AWS policies toward excess rights
IAMRAGBench demonstrates a permission-escalation route through retrieved documentation rather than invalid syntax or a direct user request.
Context privilege escalationIndirect prompt injectionLive authorization at the point of agent side effectEffect-bound approval
Public disclosure: Nov 3, 2026
Scope
Controlled AWS IAM generation benchmark with injected documents; no verified live account deployment or exfiltration.
Assessment profile
Assessment method- Setting
- evaluation
- Exploitation
- demonstrated
- Evidence
- Grade B
- Remediation
- mitigation available
- AI role
- AGAINST AI
Basis and provenance
The study specifies authorization envelopes, positive and forbidden probes, provider validity, exact deltas, and separate retrieval regimes. The evidence supports controlled policy artifact drift, not observed AWS account compromise. Independent policy-diff enforcement is a plausible mitigation but not a universally tested product patch.. Impact is ungraded because no independently measured production-target impact or victim scope is established by this record.
Assessed Nov 4, 2026 using diggingbeagle.assessment/1.
Evidence basis: CLM-NOV04-IAM-RAG-POLICY-1 · CLM-NOV04-IAM-RAG-POLICY-2 · CLM-NOV04-IAM-RAG-POLICY-3 · IAMRAGBench: poisoned retrieval documents in AI-generated AWS IAM policies
At a glance
Schema validation cannot replace semantic least-privilege verification. A generated IAM document needs independent policy analysis, diff to an approved privilege envelope and human authorization before application.
Read the implicationsProduction effectiveness of the attacker snippets and downstream cloud incidents are unmeasured in the cited benchmark. Some tested cases may depend on whether the malicious snippet is included in retrieval context.
Limits and uncertaintyFull account
An assistant asked to generate a valid AWS IAM policy may consult external examples. IAMRAGBench tests the effect of placing attacker-chosen guidance in the retrieved document context while retaining a legitimate policy-writing objective. In the included-context evaluation, the authors report 46 exact attacker goals achieved among 292 eligible attempts, around 15.75%. The resulting policies may be syntactically and provider-valid even when authorization scope has been silently widened.
The security boundary is not only whether a model emits well-formed JSON. The important comparison is the human-approved minimum privilege specification against the actual actions permitted by the generated policy. This controlled test does not show a production cloud compromise or that every retrieved page is equally effective. Paper.
Retrieval changes the result materially. The direct included-context test produced 49 exact authorization deltas among 292 eligible controls, of which 46 matched the attacker's declared permission goal. In the end-to-end BM25 retrieval configuration, the task-specific poison entered the top-five context for all 60 tasks but only seven of 294 eligible runs produced an exact goal-matched expansion. Under a separate authority-spoofing presentation, 115 of 294 matched across five endpoints; the authors describe the all-five extension as post hoc, not a frozen preregistered census. These regimes must not be merged into one attack rate. Benchmark, Table 2 and sections 6.1 to 6.3. The secure check is a request-level authorization delta against the user's intended envelope: a provider-valid JSON document may still grant the forbidden tenant, action or resource. This is a policy-artifact boundary rather than the tool-call approval gap, which concerns whether the executor dispatches an action.
Mechanism and trust boundary
- 01
Insert policy advice into retrieval
The attacker controls a lower-trust reference snippet encountered during the agent’s legitimate cloud-policy generation task.
Boundary: Retrieved documentation to model context
- 02
Steer policy semantics
The model adopts broader authorization suggestions while still constructing an IAM policy accepted by the provider grammar.
Boundary: Untrusted examples to security-critical authorization design
- 03
Deliver an overbroad permission artifact
A valid policy grants actions or resources beyond the legitimate task even if no one executes the policy during the experiment.
Boundary: Generated configuration artifact to real cloud authority on deployment
Claims & evidence
CLM-NOV04-IAM-RAG-POLICY-1The researchers report that 46 of 292 eligible included-context attempts reached an attacker-selected permission-expansion goal.supported
Basis: reported finding
- supportsIAMRAGBench: poisoned retrieval documents in AI-generated AWS IAM policiesprimary disclosure
Attack evaluation results, eligible attempts
CLM-NOV04-IAM-RAG-POLICY-2The attack targets excessive permissions in provider-valid generated IAM policy output rather than relying on malformed policy syntax.supported
Basis: reported finding
- supportsIAMRAGBench: poisoned retrieval documents in AI-generated AWS IAM policiesprimary disclosure
Threat model and IAM policy validity criteria
CLM-NOV04-IAM-RAG-POLICY-3With end-to-end BM25 retrieval, the designated poison was retrieved for all 60 tasks but only 7 of 294 eligible endpoint-case runs met the exact attacker-goal authorization test, versus 46 of 292 in the direct included-context condition.supported
Basis: reported finding
- supportsIAMRAGBench: poisoned retrieval documents in AI-generated AWS IAM policiesprimary disclosure
Section 6.1, Table 2, and Section 6.2, end-to-end BM25 regime
Implications
Schema validation cannot replace semantic least-privilege verification. A generated IAM document needs independent policy analysis, diff to an approved privilege envelope and human authorization before application.
Controls and mitigations
- Compare generated actions/resources to a machine-readable minimum-privilege specification.
- Run cloud IAM simulation and prohibit wildcard or privilege-escalation grants unless explicitly approved.
- Treat retrieved examples as untrusted data and retain exact document provenance.
- Compare generated IAM policies against a separately specified authorization envelope using allow and deny probes before deploying any provider-valid policy.
Unknowns and contradictions
- Production effectiveness of the attacker snippets and downstream cloud incidents are unmeasured in the cited benchmark.
- Some tested cases may depend on whether the malicious snippet is included in retrieval context.
Sources and citation
Material revision history
- Nov 4, 2026 · Published version · first publication · revision 217