Evidence · DiggingBeagle record
ServerBench: AI-authored inference server correctness and performance
Coding agents wrote and optimized model inference servers during eight-hour development runs. Some implementations exceeded reference throughput but failed correctness verification or natural stop under load. ServerBench reports 22 of 33 versions passed its verifier, 13 beat the reference and only five passed recorded natural-stop tests in the heavy-load setting. Cost/value ratios were benchmark-specific, not real sales. Research/disclosure claims are specific to the documented settings; do not infer unreported victims or additional compromise.
- Source role
- primary disclosure
Evidence record
Coding agents wrote and optimized model inference servers during eight-hour development runs. Some implementations exceeded reference throughput but failed correctness verification or natural stop under load. ServerBench reports 22 of 33 versions passed its verifier, 13 beat the reference and only five passed recorded natural-stop tests in the heavy-load setting. Cost/value ratios were benchmark-specific, not real sales. Research/disclosure claims are specific to the documented settings; do not infer unreported victims or additional compromise.
Claim-level citations (3)
- supportsAgent-written inference servers traded throughput for verification and stop failures: ServerBench reports 22 of 33 versions passed its verifier, 13 beat the reference and only five passed recorded natural-stop tests in the heavy-load setting. Cost/value ratios were benchmark-specific, not real sales.
ServerBench: 11 models x three workloads; verification and natural stop
- supportsAgent-written inference servers traded throughput for verification and stop failures: Coding agents wrote and optimized model inference servers during eight-hour development runs. Some implementations exceeded reference throughput but failed correctness verification or natural stop under load.
ServerBench: 11 models x three workloads; verification and natural stop
- supportsAgent-written inference servers traded throughput for verification and stop failures: ServerBench reports that 22 of 33 generated inference servers passed its verification suite while only five passed the recorded natural-stop tests in long-context and loaded settings. Throughput gains therefore do not imply full inference correctness.
ServerBench verification, natural-stop and long-context results