An agent selected a patched skill but executed an old loaded handle in a benchmark

SkillLifeBench tests lifecycle decisions for agent skills over a collection of scenarios, including cases where an updated package is available after a vulnerable version. One example has an agent correctly select the patched v1 package but continue to use a previously loaded v0 tool handle. The old handle can still expose files such as .git/config despite the agent’s correct-looking version decision. This is a runtime identity and revocation failure, not necessarily a bad textual review of the new package.

The study also reports scenarios that satisfy ordinary task completion while reaching an attacker objective, a reminder that a final answer can hide intermediate side effects. The relevant regression checks compare installed artifact hash, current tool registry, active process handle and actual call target. Merely logging a planned update does not establish that the old authority has been removed.