The report
The MCPVenom authors reported 2,113 vulnerable test outcomes, but only 255 verified synthetic-secret exfiltrations. Those are not interchangeable. Their November 2026 benchmark crossed 12 frameworks, 18 models and 45 attacks, designing 9,720 cells. It recorded 2,113 vulnerable, 7,102 resistant, 237 errored and 268 skipped cells; the reported 22.9% rate uses 2,113 of the 9,215 vulnerable-or-resistant outcomes, not all 9,720 designed cells. Methods and case dossier.
The vulnerability label also covers different observations. Of the 2,113 vulnerable cells, 1,371 reached the dispatched-tool tier, 487 confirmed contact with an attacker-controlled endpoint, and 255 supplied independent canary receipts of synthetic-secret transfer. A tool call that was constructed or dispatched is weaker evidence than an observed transfer. None of these benchmark counts measures real victims or actual stolen credentials. Original results, Table 6.
A contemporaneous IAMRAGBench investigation reached a different endpoint: poisoned retrieval material caused permission expansion in generated AWS IAM policy documents. Its directly included-context evaluation measured 46 exact attacker-goal results among 292 eligible runs; with BM25 retrieval in the loop, the count was seven among 294. The model generated provider-valid syntax that could still exceed the intended authorization envelope. These figures are not MCP exfiltration attempts and must not be combined with them. Original experiment.
The common mistake is asking whether the assistant accepted lower-trust words, then using that answer as a proxy for impact. A more useful test asks what the final component permitted: an HTTP connection, a secret read, an unauthorized file write or an over-privileged IAM policy. Host-enforced tool scopes, egress restrictions and independent authorization-delta checks belong at those actual effect boundaries. The separate MCP approval study tests whether humans were asked before destructive calls, which is another control question, not evidence of theft in either benchmark.