The report
An October 9 preprint reports uneven language-model performance on cross-chain bridge verification flaws. Only two of four tested models exceeded a baseline that simply predicts every vulnerability pattern for every audit report. The study, by Yuying Wang and coauthors, introduces a code-mapped dataset and an 11-pattern taxonomy for this specialized auditing problem.
The collection and evaluation populations differ. Researchers began with 588 audit reports from 36 projects. After identifying verification-related findings and recovering their audited code versions, they retained 830 instances from 209 reports across 27 projects, mapped to 786 source files. The model experiment makes multi-label predictions at report level; it is not 830 separate exploit tests.
| Model or reference | Reported Macro-F1 |
|---|---|
| Composer-2.5 | 51.2% |
| GPT-5.5 | 33.8% |
| Predict-all-patterns baseline | 29.3% |
| DeepSeek-V4-Pro | 25.3% |
| Opus-4.8 | 20.7% |
These author-reported scores average F1 across 11 patterns. They are not the proportion of deployed bridges a model can audit safely. The baseline's result depends on the distribution of labels in this selected corpus.
The error analysis points to both missing cross-component context and mistaken interpretation of delegated checks. An absent gateway can hide evidence of a real flaw, while a valid upstream check can make a missing local check harmless. The replication README is publicly readable and describes the source packages and model outputs; this review did not recompute the results.
The dossier records methods, limitations and researcher identities. The dataset concerns documented audit vulnerabilities, not 830 newly exploited contracts, and the study establishes no AI involvement in historical bridge attacks.