AgentShield researchers compared web agents that read raw HTML, the accessibility tree or the rendered screenshot. DOM-hidden instructions and canvas-only pixels travel through different perception channels. In a controlled product-information task, the authors drew a faint extra product claim onto a canvas. Some screenshot-first agents reproduced it,…
Inspect the ClaimsCase · DiggingBeagle record
Canvas-only page text contaminated screenshot-first agents while DOM readers missed it
AgentShield shows that defensive coverage depends on whether browser agents consume DOM text, accessibility nodes or rendered pixels.
Scope
Controlled answer-contamination research on a researcher-built web page. No observed account takeover, payment transfer or production infection.
Assessment profile
Assessment method- Setting
- evaluation
- Exploitation
- demonstrated
- Evidence
- Grade B
- Remediation
- not applicable
- AI role
- WITH AI
Basis and provenance
Controlled paper with explicit model-by-modality runs and a fully visible canvas control establishes representation-specific prompt influence in the selected experimental agents. No production exploitation, population prevalence or universal mitigation efficacy measured; impact left unbanded.
Assessed Nov 5, 2026 using diggingbeagle.assessment/1.
Evidence basis: CLM-VIS-CANVAS-CONTRAST · CLM-VIS-CANVAS-PERCEPTION-COVERAGE · AgentShield: defensive browser fingerprinting, hidden-content effects and extinction-induced policy drift
At a glance
The study isolates a representation mismatch: canvas pixels were available to screenshot-first agents while the same text was absent from DOM and accessibility outputs. A defensive test that scans only markup cannot establish what an OCR-based agent may ingest, and a screenshot test cannot inspect concealed DOM text that was never drawn. The necessary…
Read the implicationsNo general prevalence of canvas-only manipulation is established. Results depend on the screenshot and DOM input representations used by each agent.
Limits and uncertaintyFull account
AgentShield researchers compared web agents that read raw HTML, the accessibility tree or the rendered screenshot. DOM-hidden instructions and canvas-only pixels travel through different perception channels. In a controlled product-information task, the authors drew a faint extra product claim onto a canvas. Some screenshot-first agents reproduced it, including Amazon Nova in eight of ten runs. Clients consuming only DOM or accessibility text did not receive it because the text had no corresponding DOM node.
This does not prove that a model is generally immune to prompt injection or that the website has been compromised. It identifies a mismatch between the layer a defender inspects and the layer the agent perceives. A sanitizer covering HTML and accessibility nodes can miss attacker-controlled pixels in screenshots, while a screenshot-only agent may ignore malicious DOM strings. The reported experiment tests answer contamination rather than unauthorized transfer or code execution.
The researchers separated the reader channels rather than treating a page as one string. Their normal page retained readable DOM text but drew the manipulated claim as faint canvas pixels. The content therefore existed in screenshots but not in the DOM or accessibility tree. When they moved the whole visible article into canvas as a control, the DOM-reading clients also lost the ordinary page text. That result makes a specific point: a DOM-only sanitizer can truthfully find no malicious string while a screenshot-first agent receives one. Conversely, screenshot-only inspection can overlook nonvisible DOM strings. The experiment concerns answer contamination in a controlled page, not a real merchant or observed money theft.
Mechanism and trust boundary
- 01
Render a low-opacity claim on canvas
Test page contains pixels encoding a factual-looking claim absent from source DOM.
Boundary: Untrusted web graphics into screenshot input
- 02
Screenshot-first agent interprets claim
Vision-based agent repeats image-only content; HTML-only agents never see it.
Boundary: Rendered pixels into answer authority
Claims & evidence
CLM-VIS-CANVAS-CONTRASTIn AgentShield tests a low-opacity canvas-rendered product claim was copied by some screenshot-first agents, including 8/10 Amazon Nova runs, while DOM-only clients failed to perceive that canvas text.supported
Basis: reported finding
- supportsAgentShield: defensive browser fingerprinting, hidden-content effects and extinction-induced policy driftprimary disclosure
Section IV-D Content Modification, Visual-Layer (OCR) Injection; Figure 9
CLM-VIS-CANVAS-PERCEPTION-COVERAGEAgentShield reports that visually rendered but DOM-absent low-opacity canvas text was incorporated by Amazon Nova in eight of ten trials and by Gemini in Chrome and Director.ai in all tested trials; agents restricted to the DOM or accessibility tree failed to retrieve even visible canvas text in a follow-up control.supported
Basis: reported finding
- supportsAgentShield: defensive browser fingerprinting, hidden-content effects and extinction-induced policy driftprimary disclosure
Section IV-D Content Modification, Visual-Layer (OCR) Injection and control that rendered the full article to canvas
Implications
The study isolates a representation mismatch: canvas pixels were available to screenshot-first agents while the same text was absent from DOM and accessibility outputs. A defensive test that scans only markup cannot establish what an OCR-based agent may ingest, and a screenshot test cannot inspect concealed DOM text that was never drawn. The necessary control is a trusted distinction between user instruction and untrusted page observation after every conversion step, not a promise that one type of model is inherently safe. See perception-channel injection. This Case measures selected answer contamination in a synthetic product task, not an executed unauthorized transaction, and it should not inherit the separate stalled-checkout Case's outcome counts.
Controls and mitigations
- Assess both screenshot pixels and DOM/accessibility representations where the agent may use either.
- Require claims to be grounded in independent trusted product evidence rather than a webpage alone.
- Measure attack success separately for each perception pathway, not as a model-wide attribute.
- Use source-aware provenance for both OCR and DOM observations; treat every page representation as untrusted data, not executable authorization. Test the exact modality of the deployed agent against canvas, CSS-hidden nodes and accessibility-only text.
Unknowns and contradictions
- No general prevalence of canvas-only manipulation is established.
- Results depend on the screenshot and DOM input representations used by each agent.
Sources and citation
Material revision history
- Nov 5, 2026 · Published version · first publication · revision 221