SGLang disaggregated-prefill KV-cache transfer safety research

Evidence record

SGLang team technical report on asynchronous RDMA cache-slot reuse following cancellation, with deterministic two-H200 reproduction and deferred-release patch. It separately describes Cloudflare rejection-path remediation.

Read the original source ↗

Claim-level citations (5)

• supportsSGLang asynchronous RDMA transfers could overwrite a newly assigned user cache: The SGLang report describes posted RDMA writes surviving abort and writing into recycled KV-cache storage in a disaggregated-prefill/decode deployment.

  Technical analysis; RDMA transfer cancellation and slot reuse

• supportsSGLang asynchronous RDMA transfers could overwrite a newly assigned user cache: The authors reproduced cross-request cache corruption with a controlled two-H200 setup and describe affected transfer backends including Mooncake, NIXL and MORI.

  Reproduction; Two-GPU experiment and transfer backends

• supportsSGLang asynchronous RDMA transfers could overwrite a newly assigned user cache: SGLang describes deferred cache-slot release until outstanding transfers finish, combined with earlier rejection/admission checks. The RDMA cancellation reproduction is not the same event as Cloudflare’s 155 production responses.

  Related KV cache lifetime issues; Deferred Release; Immediate remediation and upstream fix

• supportsSGLang: SGLang documents disaggregated prefill/decode KV-cache transfer and the risks of asynchronous RDMA writes after request cancellation.

  Technical analysis; disaggregated prefill and decode architecture

• supportsSGLang: Cloudflare reports that its GLM-5.3 cross-customer incident came from an over-capacity prefill request entering decode, while the SGLang team separately reproduces an asynchronous RDMA cancellation/write-after-reassignment hazard. The two mechanisms must not be counted as the same confirmed production incident.

  Related KV cache lifetime issues; deterministic reproduction; admission fix