Evidence · DiggingBeagle record
PatchHolmes improves fixing-commit ranking without proving patch correctness (research paper)
A local listwise coding agent inspects retrieved commit candidates to link CVEs to likely fixing changes, but selection accuracy does not establish that a patch is safe or complete.
- Published
- Nov 1, 2026
- Source role
- primary disclosure
Evidence record
A local listwise coding agent inspects retrieved commit candidates to link CVEs to likely fixing changes, but selection accuracy does not establish that a patch is safe or complete.
Claim-level citations (4)
- supportsPatchHolmes improves fixing-commit ranking without proving patch correctness: PatchHolmes first ranks a window of roughly 100 plausible repository commits and then inspects a much smaller selection, typically three to ten commits, through its constrained inspection tools. Ranking candidates is not the same event as verifying that a proposed commit repairs the vulnerable path.
Approach and efficiency evaluation: initial candidate window, iterative inspection budget and final fix-link verification
- supportsPatchHolmes improves fixing-commit ranking without proving patch correctness: For a PatchFinder top-ten transfer set, the authors report top-one retrieval improving from 24.28 to 39.86 percent under their candidate and ground-truth labels.
Results, PatchFinder transfer evaluation
- supportsPatchHolmes improves fixing-commit ranking without proving patch correctness: Hybrid retrieval yields a set of 100 candidate fixing commits; a frozen open-weight agent compares candidates within one conversation and inspects a small shortlist using bounded local tools.
Approach, hybrid retrieval and listwise commit inspection
- supportsPatchHolmes improves fixing-commit ranking without proving patch correctness: The authors report that moving PatchHolmes to the independent PatchFinder_top10 candidate set raises top-one fixing-commit retrieval from 24.28 percent to 39.86 percent; this is candidate identification accuracy, not proof that the linked commit securely fixes a deployed product.
AACL 2026 paper 115, Abstract and cross-dataset transfer evaluation