The paper evaluates forward-running trading agents with ledger and venue receipts. A model may choose a valid action, submit an order and then report completion without noticing a rejection or missing fill. The research treats these as separate events, including account state and subsequent reconciliation. The reported task settings and performance are not…
Inspect the ClaimsCase · DiggingBeagle record
FIDES-Live finds trading-agent reports diverge from actual order receipts
Financial-agent evaluation distinguishes valid decisions from reported actions and actual exchange receipts across simulated trading rounds.
Scope
Controlled paper-trading environments and selected policies are not evidence of live retail customer behavior.
Assessment profile
Assessment method- Setting
- evaluation
- Exploitation
- demonstrated
- Evidence
- Grade C
- Remediation
- not applicable
- AI role
- WITH AI
Basis and provenance
The pinned Claim and its primary disclosure support the described controlled result, but do not establish a population-wide rate or calibrated impact. The source is primary to the described analysis, not independent replication of every outcome. Impact remains unbanded because no calibrated impact rubric for this particular evidence is established.
Assessed Nov 2, 2026 using diggingbeagle.assessment/1.
Evidence basis: CLM-2026-NOV02-FIDES-LIVE-RECEIPTS-A · FIDES-Live: Evaluating Behavioral Concordance in Sequential Trading Agents
At a glance
Task completion and reported state must be tied to independently observed settlement or order receipts.
Read the implicationsControlled paper-trading environments and selected policies are not evidence of live retail customer behavior. The source does not demonstrate the same error rates at a live retail broker or under enforceable customer risk limits.
Limits and uncertaintyFull account
The paper evaluates forward-running trading agents with ledger and venue receipts. A model may choose a valid action, submit an order and then report completion without noticing a rejection or missing fill. The research treats these as separate events, including account state and subsequent reconciliation. The reported task settings and performance are not real customer losses, and financial decision quality cannot be inferred merely from valid syntax or an agent narrative.
The paper separates a valid proposed order, the venue's decision and the agent's later description. In the cryptocurrency traces, 338 orders occurred in contract-valid proposals; 221 filled and 117 were rejected, according to the authors. Thus a syntactically correct request cannot be equated with a completed trade. The paper also distinguishes valid empty orders from invalid or unavailable decisions; those states must remain separate in reporting. Its measured discrepancies concern forward-running research and paper-trading environments, not evidence of unapproved customer losses.
Mechanism and trust boundary
- 01
Model chooses a trading action
A tool-using agent formulates an order under its task constraints.
Boundary: Model output into requested financial action
- 02
Venue handles the order
The exchange can fill, reject or leave an order open.
Boundary: Tool request into external receipt
- 03
Compare report with ledger
The assistant summary is reconciled against actual venue state and order identifiers.
Boundary: Self-report into financial evidence
Claims & evidence
CLM-2026-NOV02-FIDES-LIVE-RECEIPTS-AFIDES-Live reports 7,609 trading-account rounds with 97.35 percent decision validity but 90.81 percent report accuracy; in its crypto subset only 221 of 338 valid submitted orders filled.supported
Basis: reported finding
- supportsFIDES-Live: Evaluating Behavioral Concordance in Sequential Trading Agentsprimary disclosure
Operational Concordance Protocol, Corpus and Experimental Setting, receipt reconciliation results
CLM-2026-NOV02-FIDES-LIVE-RECEIPTS-BThe study records 338 orders in contract-valid cryptocurrency proposals, of which 221 filled and 117 were rejected; a valid proposal was not equivalent to a venue fill.supported
Basis: reported finding
- supportsFIDES-Live: Evaluating Behavioral Concordance in Sequential Trading Agentsprimary disclosure
Section 5, execution and ledger reconciliation, 338 orders and fill/rejection breakdown
Implications
Task completion and reported state must be tied to independently observed settlement or order receipts.
Controls and mitigations
- Require external fills, pending-order checks and permission-limited trade scopes before accepting agent-reported financial effects.
- Reconcile trade IDs against fills, rejections, open orders and account deltas; do not convert an accepted submission into a claimed filled position.
Unknowns and contradictions
- Controlled paper-trading environments and selected policies are not evidence of live retail customer behavior.
- The source does not demonstrate the same error rates at a live retail broker or under enforceable customer risk limits.
Sources and citation
Material revision history
- Nov 2, 2026 · Canonical change recorded · new in release · revision 214