Task Tampering shows an agent can follow workflow steps yet change unauthorized state

An attacker can modify untrusted intermediate task state or tool evidence so that an assistant appears to follow the authorized workflow while committing an unauthorized amount or state update.

The Amazon Task Tampering study evaluates whether correct-looking action order masks wrong external effects. Its observations are in controlled tasks, not proof of a specific financial-production compromise.

Task Tampering changes what a workflow is authorized to do without necessarily making the assistant abandon the visible steps of the task. In a financial-style example, reading a document, choosing an action and submitting a tool call can all occur in the expected order while the amount, recipient or final state differs from the independently authorized intent. The attacker controls lower-trust task data or tool output, not the original user request. The study evaluates synthetic episodes, so no real account transfers or customer losses should be inferred. A meaningful verifier checks the final external state against a separate task specification rather than using the agent's explanation as the receipt.

The practical measurement is the state left behind by a tool, not whether the model described the expected sequence. A workflow can read the right document, call the right API and still apply a number or destination supplied by lower-trust material. This differs from a tool-recipient swap: the same authorized recipient may receive an unauthorized amount or content. A useful evaluation therefore fixes the original user-approved parameters separately and compares them with independently observed post-action state.