The report
A defender can be penalized for issuing an action that the environment never carries out. That is what a CAGE 4 audit found in the Restore workflow of a cyber-defense simulator: if the agent was already busy, the simulator ignored an attempted Restore yet still charged it. The busy flag was absent from the documented normal observation but available in some special policy wrappers, giving wrappers an important practical advantage. The audit reproduced reference scores and then isolated the effect of that state handling.
Why the scores moved
Removing a wrapper's busy-state guard degraded a reference defender's score by about 99.58 points per episode in one comparison, despite an equivalent sequence of actions that actually executed. Repeated ineffective Restore attempts accumulated cost. Supervising actions so they waited until the simulator was idle improved outcomes without necessarily making the defender better at detecting attacks. The paper also identified unseeded policy sampling and other reproducibility limits that contribute substantial variance to published comparisons.
This is a warning about benchmark measurement, not proof that the defender would fail to stop a real ransomware campaign. The CAGE 4 Case tracks attempted versus executed actions, scoring assumptions and the uncertainty from incomplete seeds. A score comparison between systems that see different hidden state may reward interface access more than defensive reasoning.
What an independent comparison needs
Evaluation should record proposed commands, accepted commands, executed commands and the separately measured security effect, with identical state and seed access for every competitor. It should also report uncertainty over policy sampling, attacker profiles and scoring weights. Without those controls, a headline result can change because a benchmark silently charged discarded actions, not because a cyber-defense agent improved its protection.