ContractLens finds useful internal model signals for malicious smart-contract screening

ContractLens examines the source of an EVM-compatible token contract before a suspicious transfer has to occur. Its classification signal does not come from asking a model to write an audit report: the authors run code through a frozen pretrained language model, extract internal activations, and fit a separate logistic-regression classifier to selected feed-forward units. The paper, submitted on October 4, 2026 by Shuyi Miao and six coauthors, describes a controlled defensive experiment rather than a deployed scam-blocking product.

The pipeline has two selection stages. First, probes identify transformer layers that distinguish annotated positives from negatives. The authors remove the code responsible for a suspicious feature and compare its change to similarly sized nonsemantic edits, then check whether a layer stays stable when identifiers, comments or formatting change without altering the targeted behavior. Only layers satisfying separability, response to the removed logic and surface stability survive. Second, the method scores MLP units by class-mean activation contrast, their linear-probe weights and relative sensitivity to logic removal versus cosmetic changes. A knee-point cutoff selects the units, called malicious contract-discriminative units (MCDUs). The backbone weights remain frozen, but both selection and decoding still require labeled examples and computation.

What do the labels actually mean? The study reports 32,141 filtered BSC contract sources: 21,166 annotated with at least one of ten target intents and 10,975 designated benign. Its underlying intent and vulnerability dataset covers more than 90,000 Ethereum and BSC contracts and explicitly includes Reflection, Reward, Rebase and Mint as token capabilities, alongside blacklists, honeypots and trading restrictions. A minting function or reflection mechanism can be part of ordinary token design; calling every code-labeled positive a demonstrated scam would be incorrect. The paper itself says the labels represent possible malicious capabilities, not proven exploitation. ContractLens partitions matching creator or source-code groups together, reducing one simple form of leakage, although the precise data lineage and label validity still require independent audit.

The main result on Qwen2.5-Coder-7B is 92.41% mean accuracy and 96.24% mean AUROC across ten separate one-intent-versus-benign tests; Qwen3-8B reports 91.21% mean accuracy and the manuscript's CodeGemma-family model reports 90.94%. These are author-computed macro averages over controlled tasks, not measured precision for an unselected stream of token launches. The two evaluation styles differ: in a leave-one-intent-out experiment on five held-out categories, ContractLens scores 73.50% mean accuracy on balanced sets, against the highest competing mean accuracy of 66.08%. That experiment demonstrates some transfer across the authors' known label taxonomy, not detection of arbitrary unforeseen attack strategies.

The neuron intervention gives a narrower but useful mechanism check. The authors train a full-activation linear probe, then mask the identified units and matched controls. On Qwen3-8B, selected-unit masking moves accuracy from 91.52% to 83.04%, while randomly selected units have no measured accuracy effect and activation-matched nonselected units change accuracy by only 0.25 point. However, AUROC for the selected-neuron intervention falls only from 95.02% to 94.17%, a distinction between accuracy at the experiment's decision rule and overall ranking quality. It supports a task-specific contribution by these units but does not imply that an individual neuron has a stable human-readable meaning, or that removing it erases the model's knowledge of a security problem.

The temporal test should not be described as a live deployment evaluation. The source collection begins with 14,671 BSC token addresses from March 2023 to June 2025. The cited upstream dataset reduces this to 1,495 representative tokens and 1,201 labeled tokens, of which 898 are called scams and 303 benign. ContractLens then imposes a post-Gemma-release deployment filter and evaluates honeypot, blacklist/whitelist and transfer-restriction classes, reporting 77.50% mean accuracy and 84.13% AUROC. Neither the 14,671 nor the 1,201 is stated to be the final denominator for these tests; the final per-class sample sizes are not provided in the passages inspected. The result is therefore suggestive of transfer to later-dated contracts, not a prospective field false-positive measurement.

An additional data-efficiency experiment holds the selected MCDUs fixed and trains only the decoding classifier with increasingly many balanced labels. With 32 labeled examples, the paper reports 85.03% average AUROC across seven intents, improving with more labels. This is operationally relevant because the expensive selection stage could be reused within a specified backbone and task, but it is not proof that the entire pipeline can identify its critical neurons from 32 labels or run as a cheap real-time API. The study uses 2,048-token windows with 512-token overlap for long code and reports one V100 32 GB GPU for probing and classifier training; it does not measure the full data acquisition, source verification and human-triage pipeline.

The Lucky Shiba example makes the possible abuse concrete. A privileged caller can decide which holders are exempt from a delegated-transfer counter, while nonexempt accounts can be blocked on subsequent delegated transfers. Another privileged function overwrites a balance without adjusting total supply or emitting a Transfer event. ContractLens assigns 95.68% predicted probability to the dataset's Mint label on that code, whereas several compared classifiers score it below the paper's decision threshold. This illustrates how separated pieces of contract logic jointly matter, but the paper does not document a transaction loss, an actual victim, or intervention on a running scam.

The lineage also matters. Earlier PyPI neuron-probing research localized malware-related signals in three instruction-tuned models and applied interventions, but it did not evaluate EVM contracts. The DG-VDT dossier investigates a different defensive boundary: classification and address traceability after EVM execution traces have been generated, rather than source-capability classification before execution. Neither work independently reproduces ContractLens.

For a security reviewer, the defensible use would be to flag code for deterministic inspection, simulation or owner-privilege review, with a reproducible source snapshot and calibrated thresholds. A high classifier score cannot determine whether a capability is intentionally abused; conversely, a low score cannot certify safety across unmodeled proxy patterns, owner upgrades or novel evasion. Release-quality evidence would need reproducible checkpoints, published end-to-end feature extraction, audited annotation criteria, exact post-filter test denominators, and prospective error rates on realistic prevalence. Until then, ContractLens demonstrates a promising probe-based research method, not reliable fraud adjudication.