FALCON verifies executable counterexamples before rewarding AI security repairs

A coding agent can produce a patch that passes its own security judge without changing the vulnerable behavior. FALCON addresses this by searching for counterexamples, checking whether their outcomes genuinely demonstrate a problem and feeding the verified result into iterative repair. The paper compares its evidence-grounded signal with heuristic detectors and model-only judgments on test corpora.

The authors report a substantial benchmark improvement in securely repaired programs over five feedback rounds. The result is not a proof that arbitrary code produced with FALCON is safe, and an intentionally misleading test can still cause trouble unless the counterexample's claimed effect is validated against the real executable. This study complements SolBehave: one checks fidelity against reference contract behavior, the other checks whether a generated security fix eliminates a targeted failure.

Counterexamples are useful only when they reproduce the claimed failure against the real candidate program. The reported improvement depends on executing and validating the probe before turning it into a repair signal. This matters when a model or external tool can propose a convincing but irrelevant attack trace: a merely textual assertion of failure would reintroduce the same unverified judgment that executable checking was intended to replace.