Evidence · DiggingBeagle record
On the Reliability of LLM-Based Vulnerability Patching Benchmarks
Northwestern University study comparing proof-of-crash suppression with maintainer functionality tests across historical bugs.
- Source role
- primary disclosure
Evidence record
Northwestern University study comparing proof-of-crash suppression with maintainer functionality tests across historical bugs.
Claim-level citations (1)
- supportsA patch that suppresses one exploit can still violate maintainer functionality tests: Northwestern researchers evaluated 112 historical defects and found material disagreement between a patch stopping the supplied proof of concept and the same patch passing developer-authored functionality tests.
Abstract; evaluation design and controlled patch comparisons