The report
The ScamWorld study, revised November 4, reconstructed 363 scam websites and 53 legitimate comparison sites for executable agent tasks. Across six tested backbones, 11% to 35% of scam episodes ended in the fraud-completing final action. The site-specific tasks matter: payment demands were accepted in 28% of tested scenarios, while scenarios asking for credentials or wallet access were accepted in 2%.
The research Case records an additional distinction: 220/492 deceived episodes included a warning about the site's risk before the agent acted anyway. A statement that a site “looks suspicious” is not a transaction gate. These are simulated site outcomes, not measured financial losses or success rates of deployed criminal campaigns.
A second AgentShield checkout study shows how repeated obstruction can make alternative instructions more attractive. The two studies have different test designs but the same authorization question: who approved the exact payment route and final recipient? Effect-bound approval answers at the executor, not in a reassuring model summary.
Recognition is a different experiment from refusal
The authors additionally asked agents to classify sites as scams outside the purchase workflow. They reported roughly 97 percent identification in that explicit task, versus roughly 75 percent risk warnings or stops during task execution. A warning in a natural-language answer is not equivalent to a denied payment command, and neither outcome gives the number of real-world victims. The legitimate-site controls matter as well: a browser agent that rejects every checkout would not pass a usable task benchmark. Comparing fraud avoidance with valid-order completion and independently logged payment effects is more informative than scoring one model reply for cautious vocabulary.
The final-response monitor can miss the warning too
The paper's trajectory analysis adds a narrower visibility problem. In 164 of 220 deceived episodes where agents raised a site-specific concern and later provided an answer, that concern appeared in their reasoning but not in the final response. The authors also report that deceived agents often cited the user's original instruction as their justification for continuing, even though a new fee or private recipient was introduced by the site. A natural-language postmortem may therefore omit the hesitation while the logged browser action shows completion. This is why the durable ScamWorld Case separates observation, explanation and side effect, and why effect-bound approval belongs outside the agent's narrative. The study's measured transfers were simulated.