Case · DiggingBeagle record

Four-billion-parameter decision model prioritizes memory-safety candidates with limited recall

A small decision model assists deterministic vulnerability mining and agent review, but its poor patched-pair discrimination limits standalone use as a security verdict.

Scope

Research scan of 4,245 selected code repositories with internal review and ASan-style reproduction; undisclosed candidate defects remain under coordination.

Assessment profile

Assessment method
Setting
evaluation
Exploitation
not established
Evidence
Grade C
Remediation
not applicable
AI role
WITH AI
Basis and provenance

The campaign reports reproduced findings, but independent known-CVE ranking is limited and the EVM adaptation found no confirmed defects; exploitability and production impact remain unverified. Classified as evaluation with exploit status not_established. Evidence grade C reflects the original paper or maintainer advisory and its stated limitations, not an independent audit of all deployments. Comparable realized harm is not established, so impact stays unrated; remediation status not_applicable is limited to the cited experiment or affected software version. New source-specific boundary: Increasing the model's classification accuracy alone cannot recover a defect discarded during initial repository mining or candidate-budget selection.

Assessed Nov 15, 2026 using diggingbeagle.assessment/1.

At a glance

Mechanism and trust boundary

  1. 01

    Entry or starting condition

    Deterministic miners and localizers shortlist suspicious source locations across selected code repositories.

    Boundary: attacker or research precondition

  2. 02

    Crossing the relevant boundary

    Intern-Decision-4B ranks candidate vulnerabilities and reachability questions before stronger tools and human reviewers investigate.

    Boundary: mechanism and authorization boundary

  3. 03

    Observed or reported result

    The campaign reports reproduced findings, but independent known-CVE ranking is limited and the EVM adaptation found no confirmed defects; exploitability and production impact remain unverified.

    Boundary: measured outcome and evidentiary limit

Claims & evidence

CLM-ENR250-16For a held-out set of 75 known CVEs, the authors report that 72 were represented in the input corpus, only 29 were marked by the preliminary miner and 18 survived subsequent candidate limits; 16 reached the final top-25 list. Most misses occurred before the small decision model could rank the evidence.supported

Basis: reported finding

Link to claim
CLM-R245-SMALL-MODEL-VULN-PRIORITY-01Authors describe 187 internally reviewed memory-safety findings, yet independent-project known-CVE top-25 ranking recovers only 16 of 75 test defects and smart-contract adaptation yields 61 false positives with zero confirmed findings.supported

Basis: reported finding

Link to claim
CLM-R245-SMALL-MODEL-VULN-PRIORITY-02Deterministic candidate miners and suspicious-write locators shortlist code fragments; Intern-Decision-4B then ranks vulnerability and reachability questions before larger coding agents and human reviewers investigate.supported

Basis: reported finding

Link to claim
CLM-R246-SMALL-MODEL-VULN-PRIORITY-BOUNDARYThe decision model scores native C/C++ vulnerable-versus-patched pairs with ROC-AUC 0.564, and at a threshold of 0.5 detects only 15 percent of vulnerable examples with 6 percent benign false positives; that is inadequate as a standalone vulnerability verdict.supported

Basis: reported finding

Link to claim

Unknowns and contradictions

  • Reproduced crashes do not establish exploitability on an installed entry point, and the withheld findings cannot yet be independently counted or verified. Model rankings are not automatic fixes.
  • No evidence here establishes the number of production victims or a broader failure rate outside the cited evaluation.

Sources and citation

Material revision history

  1. Nov 15, 2026 · Canonical change recorded · new in release · revision 251