More code does not mean more verified software

A development team that merges more lines of code may still ship the same number of features. In a Harvard study of AI coding adoption, agent-using firms generated substantially more commits and pull requests, but the measured interval from pull-request submission to merge grew, and the researchers did not find a corresponding statistically significant rise in completed software output. That gap matters to security teams asked to approve ever larger amounts of machine-produced code.

Fiona Chen and James Stratton analyze approximately 300 million engineering work events involving 725,938 workers at 718 firms, from January 2021 to March 2026. Their staggered difference-in-differences design compares firms by adoption timing. Its causal interpretation depends on assumptions about how early and late adopters would have evolved without the tools. This is not a randomized experiment, nor a count of security vulnerabilities.

The authors also found no significant employment decline attributable to adoption within their identification design. For agent adoption, they say they can rule out an increase in completed software output larger than 12% of the baseline mean. That upper bound is not a finding that output increased by 12%.

The source data describes commits, pull requests, tasks and work allocation. It does not include inspection of the code content for exploitable defects. A slower review might reflect more complicated changes, improved scrutiny or additional rework. The paper identifies a production bottleneck, not a measured increase in breach frequency.

A separate context privilege escalation experiment found that attacker-controlled repository instructions could narrow a Codex review enough for it to approve vulnerable code that a control review rejected. That result concerns the authority of instructions assembled by the reviewer, not the number of pull requests generated by a firm. Moving the review queue to an AI model does not automatically remove the trust boundary.

Microsoft's FORGE research supplies a different perspective. Its reported vulnerability findings required crash reproduction, triage and maintainer response; the quoted model cost and elapsed time for successful proof-of-crash generation exclude several of those upstream and downstream stages. Automated discovery can increase the supply of candidate findings while leaving validation and remediation effort external to the quoted model benchmark.

Taken together, the three sources suggest an audit that does not confuse productivity, recognition and authority. Count distinct changes reaching review, separately measure what the verifier independently proves, and trace whether tested behavior reaches production. A reviewer's approval should not be inferred from a model's reassuring explanation or a passing test that can be influenced by the submitted change. This is a testable security methodology derived from the cases, not a claim that the Harvard study experimentally proved a security impact.