sgl-project/sglang · error · NotImplementedError

prefix-valid commit is unsupported under the page-major layo

Error message

prefix-valid commit is unsupported under the page-major layout (_set_kv_buffer_prefix_valid_impl assumes 3-D contiguous + row_dim).

What it means

set_kv_buffer_prefix_valid commits prefix-cached KV rows via _set_kv_buffer_prefix_valid_impl, which assumes a 3-D contiguous buffer with a row_dim. The page-major 4-D strided envelope has no such per-layer contiguous rows, so the call raises NotImplementedError rather than mis-committing bytes.

Source

Thrown at python/sglang/srt/mem_cache/memory_pool.py:3332

            "page-major layout has no per-layer contiguous regions; KV transfer / "
            "disaggregation is unsupported (TODO: expose the single _raw buffer "
            "with a page-aware transfer scheme)."
        )

    def get_cpu_copy(self, indices, mamba_indices=None):
        raise NotImplementedError(
            "CPU offloading is unsupported under the page-major layout "
            "(TODO: split token ids into page/slot for the 4-D index)."
        )

    def load_cpu_copy(self, kv_cache_cpu, indices, mamba_indices=None):
        raise NotImplementedError(
            "CPU offloading is unsupported under the page-major layout "
            "(TODO: split token ids into page/slot for the 4-D index)."
        )

    def set_kv_buffer_prefix_valid(self, *args, **kwargs):
        raise NotImplementedError(
            "prefix-valid commit is unsupported under the page-major layout "
            "(_set_kv_buffer_prefix_valid_impl assumes 3-D contiguous + row_dim)."
        )


class MHATokenToKVPoolMXFP8(MHATokenToKVPool):
    """MHA KV cache pool for MXFP8 block-scaled FP8.

    K/V data is stored as FP8 E4M3. Per-32-element UE8M0 scale factors are
    stored beside it and passed to the FA4 MXFP8 kernel.
    """

    MXFP8_SCALE_BLOCK_SIZE = 32

    def _create_buffers(self):
        with self.memory_saver_adapter.region(GPU_MEMORY_TYPE_KV_CACHE):
            with (
                torch.cuda.use_mem_pool(self.custom_mem_pool)

View on GitHub (pinned to 0132848349)

Solutions

Disable prefix caching / the prefix-valid commit path when using the page-major layout
Fall back to the token-major 3-D pool (page_size=1) to use prefix-valid commits
Port _set_kv_buffer_prefix_valid_impl to handle the 4-D page-major envelope before enabling it

Defensive patterns

Strategy: validation

Validate before calling

def prefix_valid_supported(pool) -> bool:
    return hasattr(pool, '_set_kv_buffer_prefix_valid_impl')

Try / catch

try:
    pool.set_kv_buffer_prefix_valid(layer, loc)
except NotImplementedError:
    logger.warning('prefix-valid commit unsupported; falling back to full write')
    pool.set_kv_buffer(layer, loc, k, v)

Prevention

Check pool capabilities before enabling prefix-valid optimizations
Gate prefix-cache features on pool layout in server args validation

When it happens

Trigger: Using the page-major KV layout together with prefix caching / speculative prefix-valid commit logic that calls pool.set_kv_buffer_prefix_valid(...) after a verified prefix hit.

Common situations: Enabling prefix caching (radix cache) or the prefix-valid optimization on a config that selects the page-major KV pool layout; newer SGlang versions where prefix-valid commit was added but not ported to the 4-D layout.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e2630f9b18a650b3. Report an issue: GitHub.