The study normalized source and destination message structures to compare system, user, assistant and tool roles, tool-error flags, arguments and sequence boundaries. Some translations remained valid JSON and passed HTTP checks while dropping the information that determines how a downstream model should interpret or respond to a tool result. Across 685…
Inspect the ClaimsCase · DiggingBeagle record
API gateways sometimes returned success after silently changing the authority or error state of an agent conversation
The study normalized source and destination message structures to compare system, user, assistant and tool roles, tool-error flags, arguments and sequence boundaries. Some translations remained valid JSON and passed HTTP checks while dropping the information that determines how a downstream model should interpret or respond to a tool result.
Public disclosure: Nov 11, 2026
A tool-using agent conversation is converted between provider schemas by an API gateway or compatibility adapter.
At a glance
The adapter may reinterpret a failed tool call as success, omit higher-priority instructions or change the modeled identity of a speaker.
Read the implicationsThe evidence is bounded by the cited source and test setting. Broader incidence, other environments, and actual production consequences are not established merely by these observations.
Limits and uncertaintyFull account
The study normalized source and destination message structures to compare system, user, assistant and tool roles, tool-error flags, arguments and sequence boundaries. Some translations remained valid JSON and passed HTTP checks while dropping the information that determines how a downstream model should interpret or respond to a tool result.
Across 685 completed source-valid translation cells, 127 met the researchers' stricter avoidable, silent role/prompt/tool-loss definition, about 18.5 percent. A separate set of 558 HTTP-200 response streams included 203 outputs with SDK exceptions or silent corruption. These are different denominators and should not be added.
This does not prove every production gateway exhibits the same failure rate or that these specific translations caused external unauthorized writes. The demonstrated control weakness is that schema compatibility is not semantic preservation.
Reported figures and denominator. 127/685 (18.54%) avoidable silent request transformations; 203/558 (36.4%) corrupted or exceptional response streams under a separate evaluation.
The evidence is bounded by the cited source and test setting. Broader incidence, other environments, and actual production consequences are not established merely by these observations.
Mechanism and trust boundary
- 01
Starting condition
A tool-using agent conversation is converted between provider schemas by an API gateway or compatibility adapter.
Boundary: precondition and attacker capability
- 02
Trust boundary crossed or evaluated
Across 685 completed source-valid translation cells, 127 met the researchers' stricter avoidable, silent role/prompt/tool-loss definition, about 18.5 percent. A separate set of 558 HTTP-200 response streams included 203 outputs with SDK exceptions or silent corruption. These are different denominators and should not be added.
Boundary: mechanism
- 03
Observed consequence
The adapter may reinterpret a failed tool call as success, omit higher-priority instructions or change the modeled identity of a speaker.
Boundary: observed effect and limitations
Claims & evidence
CLM-NOV15-ISLOSS-MECHThe study normalized source and destination message structures to compare system, user, assistant and tool roles, tool-error flags, arguments and sequence boundaries. Some translations remained valid JSON and passed HTTP checks while dropping the information that determines how a downstream model should interpret or respond to a tool result. Across 685 completed source-valid translation cells, 127 met the researchers' stricter avoidable, silent role/prompt/tool-loss definition, about 18.5 percent. A separate set of 558 HTTP-200 response streams included 203 outputs with SDK exceptions or silent corruption. These are different denominators and should not be added.supported
Basis: reported finding
- supportsTesting Conversation Preservation Across LLM APIs: islossprimary disclosure
isloss paper, Table of 685 cells, 127 primary findings and 203/558 response stream errors
CLM-NOV15-ISLOSS-SCOPEThis does not prove every production gateway exhibits the same failure rate or that these specific translations caused external unauthorized writes. The demonstrated control weakness is that schema compatibility is not semantic preservation. The evidence is bounded by the cited source and test setting. Broader incidence, other environments, and actual production consequences are not established merely by these observations.supported
Basis: reported finding
- supportsTesting Conversation Preservation Across LLM APIs: islossprimary disclosure
Primary disclosure: results, scope and limitations
Implications
The adapter may reinterpret a failed tool call as success, omit higher-priority instructions or change the modeled identity of a speaker.
Controls and mitigations
- Compare translated conversations against a provider-independent typed semantic representation before dispatch.
- Preserve tool errors, role priority and tool-call identifiers; test actual downstream behavior with synthetic denied operations and treat HTTP 200 as transport success, not semantic correctness.
Unknowns and contradictions
- The evidence is bounded by the cited source and test setting. Broader incidence, other environments, and actual production consequences are not established merely by these observations.
Sources and citation
Material revision history
- Nov 15, 2026 · Canonical change recorded · new in release · revision 250