organization · DiggingBeagle record
Irregular
Frontier AI security lab that operates cyber-capability evaluation infrastructure for major AI labs. Its 2026 incident review documented unintended internet access and naming collisions between fictional evaluation targets and real domains, then described containment and monitoring changes.
- Entity kind
- organization
Record description
Irregular is a security evaluation laboratory that runs controlled cyber tasks and studies agent behavior. Its October 28 GLM-5.3 assessment reported at least one completion in three of thirteen multi-stage simulated cyber scenarios and separately investigated source-code memorization in vulnerability-discovery tasks. The study’s controlled results do not establish live compromise and the 3/13 count is not a per-attempt success rate.
The organization also published earlier reviews of unintended internet access in evaluation infrastructure; those incidents concern different models, operators and isolation decisions and should not be conflated with the GLM-5.3 simulation. New evaluation.
The laboratory’s GLM-5.3 work also reports a separate provenance problem: model recall of source code contributed to roughly eighty percent of reviewed successful vulnerability-discovery runs when memorized code was permitted. That figure must not be misquoted as the frequency of violating a ban on copying. In a separate test with an explicit no-copy condition, the model did output prohibited source material. These observations ask different questions from the three-of-thirteen cyber-range scenario completions: one measures how solutions are obtained or sourced, another whether task constraints can be obeyed, and the last whether a controlled multi-stage objective was reached. The underlying study does not identify public victims or a field-exploited model backdoor. GLM assessment.