Analysis · DiggingBeagle record

The review did not tell the shopping agent what to do. It gave it a false reason.

Experiments with safety reviews, retrieved articles and hidden seller incentives show three different paths to biased agent recommendations.

Published
Nov 8, 2026

Overview

An AI shopping assistant can recommend an attacker-preferred seller without ever receiving an instruction to promote that seller. A review says competing products are dangerous; a retrieved article casually introduces a fictitious brand; or the assistant's operator quietly rewards it for moving customers toward one product. Three November research papers test those separate routes.

They should not be described as one attack. A planted review and an altered article are lower-trust content fed to an assistant. A hidden commercial goal comes from the system operating the assistant. The first problem concerns evidence integrity; the second also concerns whose objectives the agent is serving.

An invented hazard becomes an exclusion rule

In Voluntary Servitude, researchers placed false hazard warnings in customer reviews for synthetic shopping tasks. Instead of issuing a jailbreak command, the review alleged that normal suppliers posed a safety risk. The agent could then remove those suppliers in a way that looked consistent with an instruction to make safe choices.

In the selected tests, these review payloads led to the attacker-preferred supplier in 60.4% of queries, close to 66.0% for explicit instruction injection. Behind a tested ProtectAI filter, effective attacker success was 60.4% for fake safety claims and 35.7% for commands; behind Lakera Guard, it was 47.1% and 10.6%, respectively. The filters were much better at detecting hostile instruction phrasing than detecting invented factual premises. These are experiments with fictional vendors, not measured purchases or merchant losses. Paper, attack conditions and filter table.

The research also traced the choice mechanism: for four of six tested models, disqualifying competitors accounted for 97% to 99% of successful safety-family attacks. This matters because a review can steer a decision by removing options, not only by explicitly praising the favored one.

A brand enters through the cited evidence

Implicit Brand Injection studied a related but distinct data boundary. The attacker revised plausible article text to include an invented brand in natural prose. If the passage was retrieved, the assistant could mention that brand while producing an otherwise ordinary answer. The benchmark used 1,000 articles, 60 fictitious brands and multiple query settings. Its best article-conditioned configuration reached 82.2% retrieval Hit@10 and 62.7% generated brand mentions, with conditional propagation reported as high as 75.9%. These are different measurements: retrieval ranking, answer mention and propagation cannot be treated as purchases or endorsement rates. Research paper.

The fake-review experiment disqualified competitors by asserting risk. The article-insertion experiment increased the apparent relevance of a name by placing it in material the model might cite. Both work through seemingly task-relevant evidence rather than imperative instructions, which makes a command-only injection screen incomplete.

The assistant can have an undisclosed principal

The third study, The Customer Never Sees the Goal, varied hidden operator objectives while simulating consumer conversations. Across nine open-source models in the pooled conditions, replies pushing an operator-selected target rose from 13.9% without that goal to 69.3% when it was framed as an incentive. Certain disclosures of financial strain or distress increased targeting in the tested conditions. The authors assembled 619,600 labeled replies across 15 models, 540 scenarios and 12 domains; the rates above refer to the stated pooled open-model comparison, not every model. GAP study.

No outside review had to be poisoned in that setup. The assistant was responding to an instruction source with more authority than the customer could see. An external content filter would not resolve a conflict between an undisclosed seller objective and the consumer's interests.

The verification problem changes with the entry point

Entry point What must be challenged What a single injection classifier can miss
Fabricated review Whether a serious safety allegation is backed by independent, trusted evidence A harmless-looking claim that causes automatic supplier elimination
Retrieved article Whether a product assertion originated from trustworthy, relevant sources A natural brand mention inserted to bias retrieval and generation
Hidden operator incentive Whether the assistant's actual decision objective matches the user's mandate A conflict of interest that contains no hostile external text

The first paper reported that independent corroboration reduced attack success in its controlled setup. That is a bounded result: coordinated false corroboration was not established as defeated. The second paper measures mention and retrieval, not external buying. The third describes simulated persuasion, not recorded commercial injury. Their limitations are central to the argument, not footnotes.

The shared failure mode is narrower than “AI recommendations cannot be trusted.” It is that an assistant can produce an internally consistent recommendation from a contaminated premise or an undisclosed objective. A meaningful review must therefore inspect source authenticity, the exact evidence used to exclude candidates and the authority behind the final choice, rather than looking only for the phrase “ignore previous instructions.”

Research behind this