An LLM guided by known static-analysis alerts made apparently ordinary JavaScript rewrites that preserved unsafe flows while removing some CodeQL warnings. A model receives a vulnerable code fragment and the associated warning, then makes bounded maintenance-style edits while researchers check whether the original runtime flow remains and CodeQL stops…
Inspect the ClaimsCase · DiggingBeagle record
Vulnerability-preserving code rewrites evade CodeQL taint tracking in a controlled study
An LLM guided by known static-analysis alerts made apparently ordinary JavaScript rewrites that preserved unsafe flows while removing some CodeQL warnings.
172 originally detected JavaScript source-to-sink flows and 514 valid controlled rewrite attempts, with no confirmed malicious software delivery.
Assessment profile
Assessment method- Setting
- evaluation
- Exploitation
- demonstrated
- Evidence
- Grade B
- Remediation
- none known
- AI role
- WITH AI
Basis and provenance
The original alert disappears for 26.2 percent of 172 initially detected flows within three tries; manual and controlled tests still reproduce the unsafe flow. Classified as evaluation with exploit status demonstrated. Evidence grade B reflects the original paper or maintainer advisory and its stated limitations, not an independent audit of all deployments. Comparable realized harm is not established, so impact stays unrated; remediation status none_known is limited to the cited experiment or affected software version. New source-specific boundary: Independent exploit replay and source-to-sink checks are needed to show that a scanner-silent rewrite still permits the original unauthorized effect.
Assessed Nov 15, 2026 using diggingbeagle.assessment/1.
Evidence basis: CLM-R245-CODEQL-ALERT-SILENCING-01 · CLM-R245-CODEQL-ALERT-SILENCING-02 · CLM-R246-CODEQL-ALERT-SILENCING-BOUNDARY · CLM-ENR250-04 · Vulnerability-preserving code rewrites evade CodeQL taint tracking in a controlled study (research paper)
At a glance
The denominator is a selected known-alert benchmark, not all real vulnerabilities or all source analyzers. Seven successful targeted rewrites retained another same-rule finding, so a silent targeted path is not proof of a clean function. No evidence here establishes the number of production victims or a broader failure rate outside the cited evaluation.
Limits and uncertaintyFull account
An LLM guided by known static-analysis alerts made apparently ordinary JavaScript rewrites that preserved unsafe flows while removing some CodeQL warnings.
A model receives a vulnerable code fragment and the associated warning, then makes bounded maintenance-style edits while researchers check whether the original runtime flow remains and CodeQL stops reporting that specific path. Within three tries, the researchers silenced a targeted alert for 26.2 percent of the initially detected flows; alert-unaware benign rewrites silenced none of 516 control attempts.
The denominator is a selected known-alert benchmark, not all real vulnerabilities or all source analyzers. Seven successful targeted rewrites retained another same-rule finding, so a silent targeted path is not proof of a clean function.
The attacker advantage is knowledge of an existing alert. A model given the reported source-to-sink path can move the tainted value through a representation CodeQL does not follow, for example a WHATWG URL property or particular joining and array operations. Researchers then manually and dynamically check that the unsafe behavior remains. Ordinary refactoring performed without seeing the alert silenced none of 516 control attempts; the difference supports targeted exploitation of modeling gaps rather than the claim that all LLM refactoring is unsafe.
A vanished alert is not a fixed vulnerability, and neither a clean dashboard nor a successfully merged change is a runtime proof. A regression protocol must preserve the original exploit-relevant input, trace its output effect, and compare before and after semantics, especially when a code change directly references a known finding. The 26.2 percent figure is a pass-at-three rate on previously detectable flows, not a general vulnerability prevalence or measured malware success rate.
Mechanism and trust boundary
- 01
Entry or starting condition
A defender has a JavaScript source-to-sink vulnerability already flagged by CodeQL.
Boundary: attacker or research precondition
- 02
Crossing the relevant boundary
An alert-aware LLM makes a plausible maintenance rewrite that moves tainted data through a gap in the analyzer model while preserving unsafe runtime behavior.
Boundary: mechanism and authorization boundary
- 03
Observed or reported result
The original alert disappears for 26.2 percent of 172 initially detected flows within three tries; manual and controlled tests still reproduce the unsafe flow.
Boundary: measured outcome and evidentiary limit
Claims & evidence
CLM-ENR250-04The researchers attribute 43 of 70 successful vulnerability-preserving scanner-evasion rewrites to a WHATWG URL parser modeling gap and report that 64 of those 70 rewrites pass through APIs that CodeQL models at least in part. This is a specific source-to-sink coverage problem, not evidence that 43 deployed applications were compromised.supported
Basis: reported finding
- supportsVulnerability-preserving code rewrites evade CodeQL taint tracking in a controlled study (research paper)primary disclosure
Section 6.2, evasion mechanism analysis and WHATWG URL parser/source-to-sink modeling counts
CLM-R245-CODEQL-ALERT-SILENCING-01Within three tries, the researchers silenced a targeted alert for 26.2 percent of the initially detected flows; alert-unaware benign rewrites silenced none of 516 control attempts.supported
Basis: reported finding
- supportsVulnerability-preserving code rewrites evade CodeQL taint tracking in a controlled study (research paper)primary disclosure
Experiments, rewrite-success and control comparison results
CLM-R245-CODEQL-ALERT-SILENCING-02A model receives a vulnerable code fragment and the associated warning, then makes bounded maintenance-style edits while researchers check whether the original runtime flow remains and CodeQL stops reporting that specific path.supported
Basis: reported finding
- supportsVulnerability-preserving code rewrites evade CodeQL taint tracking in a controlled study (research paper)primary disclosure
Approach and taint-model failure taxonomy
CLM-R246-CODEQL-ALERT-SILENCING-BOUNDARYOf the 70 verified alert-silencing rewrites, seven still produced another finding under the same CodeQL rule; suppressing the targeted source-to-sink flow was therefore not equivalent to clearing every finding in the function.supported
Basis: reported finding
- supportsVulnerability-preserving code rewrites evade CodeQL taint tracking in a controlled study (research paper)primary disclosure
Section 6, RQ1 and manual verification of silenced targeted flows
Unknowns and contradictions
- The denominator is a selected known-alert benchmark, not all real vulnerabilities or all source analyzers. Seven successful targeted rewrites retained another same-rule finding, so a silent targeted path is not proof of a clean function.
- No evidence here establishes the number of production victims or a broader failure rate outside the cited evaluation.
Sources and citation
Material revision history
- Nov 15, 2026 · Canonical change recorded · new in release · revision 251