KVGuard identifies request-level GPU cache ownership failures missed by allocation checks

GPU KV-cache allocations can be valid at the device-memory level while individual inference requests read uninitialized or cross-request cache regions. A request-level sanitizer annotates ownership and initialization state.

KVGuard reports finding 29 historic vLLM/SGLang violations and eight new cases under its harness, with measured token-generation overhead. A detected memory-safety condition does not establish a production cross-tenant prompt leak.

GPU memory allocators can report a valid allocation even when request A reads a KV-cache block that was last owned by request B. KVGuard adds a separate shadow layer tracking request ownership and initialization inside large device allocations, including movement of cache blocks across memory tiers. This is closer to how a multitenant inference server actually reuses cache space than a conventional out-of-bounds allocator check. The paper reports detection of 29 earlier violations plus eight newly found conditions; those numbers count test defects, not confirmed cross-tenant disclosures. Real confidentiality exposure would require demonstrating that request data or outputs crossed a tenant boundary in an affected deployment.

A GPU allocation can remain inside its legal address range while containing tokens owned by the wrong inference request or values not initialized for that request. Ordinary allocation sanitizers therefore cannot establish KV-cache confidentiality or integrity at the request boundary. KVGuard's report of eight new violations calls for canary-based tests that track ownership across batching, reuse and eviction. It does not by itself demonstrate that production customers received one another's prompts, and proposed upstream patches need separate release verification.