sgl-project/sglang · error · NotImplementedError

MXFP8 KV cache does not support DCP KV masks.

Error message

MXFP8 KV cache does not support DCP KV masks.

What it means

MHATokenToKVPoolMXFP8.set_kv_buffer does not implement DCP (deep-cache/persistent KV) masked writes: passing a non-None dcp_kv_mask raises NotImplementedError immediately, since the block-scaled MXFP8 buffers plus per-token masks path is unimplemented.

Source

Thrown at python/sglang/srt/mem_cache/memory_pool.py:3483

        return self.v_buffer[layer_id - self.start_layer]

    def get_kv_scale_buffer(self, layer_id: int) -> Tuple[torch.Tensor, torch.Tensor]:
        idx = layer_id - self.start_layer
        return self.k_scale_buffer[idx], self.v_scale_buffer[idx]

    def set_kv_buffer(
        self,
        layer: RadixAttention,
        loc_info,
        cache_k: torch.Tensor,
        cache_v: torch.Tensor,
        k_scale: Optional[torch.Tensor] = None,
        v_scale: Optional[torch.Tensor] = None,
        layer_id_override: Optional[int] = None,
        dcp_kv_mask: Optional[torch.Tensor] = None,
    ):
        if dcp_kv_mask is not None:
            raise NotImplementedError("MXFP8 KV cache does not support DCP KV masks.")
        loc, _, _ = unwrap_write_loc(loc_info)
        maybe_detect_oob(
            loc, 0, self.size + self.page_size, "set_kv_buffer (MHA-MXFP8)"
        )
        layer_id = (
            layer_id_override if layer_id_override is not None else layer.layer_id
        )
        idx = layer_id - self.start_layer

        if k_scale is None or v_scale is None:
            # Fused path (SGLANG_OPT_INKLING_MXFP8_FUSED_QUANT_STORE): the layer
            # hands us bf16 K/V and one kernel quantizes + scatters the fp8
            # payload and the interleaved UE8M0 scales.
            if not self.mxfp8_sf_interleaved or cache_k.dtype == self.store_dtype:
                raise ValueError("MXFP8 KV cache requires K and V scale tensors.")
            from sglang.kernels.ops.quantization.mxfp8_quant import quant_store_kv_mxfp8

            quant_store_kv_mxfp8(

View on GitHub (pinned to 0132848349)

Solutions

1. Disable the DCP / dcp_kv_mask feature when using MXFP8 KV cache
2. Switch kv-cache-dtype to a supported one (fp8_e4m3/bf16) if DCP masks are required
3. Implement dcp_kv_mask handling in the MXFP8 set_kv_buffer before combining the features

Example fix

# before
server_args = ServerArgs(kv_cache_dtype='mxfp8', enable_dcp_kv_mask=True)
# after
server_args = ServerArgs(kv_cache_dtype='mxfp8')  # or keep DCP but use fp8_e4m3 dtype

def dcp_mask_supported(pool) -> bool:
    return 'MXFP8' not in type(pool).__name__
# or: gate server args
assert not (args.kv_cache_dtype == 'mxfp8' and args.enable_dcp_kv_mask)

try:
    pool.set_kv_buffer(layer, loc, k, v, dcp_kv_mask=mask)
except NotImplementedError as e:
    if 'DCP' in str(e):
        pool.set_kv_buffer(layer, loc, k, v)  # unmasked fallback if semantics allow

When it happens

Trigger: Calling set_kv_buffer(..., dcp_kv_mask=tensor) on the MXFP8 pool, i.e. enabling the DCP KV mask feature (deep cache prompt / selective KV writing) together with MXFP8 KV cache dtype.

Common situations: Enabling DCP/deep-cache features (which pass dcp_kv_mask into set_kv_buffer) on a server configured with MXFP8 KV cache; version where DCP was added for other dtypes only.

Related errors

• prefix-valid commit is unsupported for MXFP8 KV cache (it do
• MXFP8 fused prologue requires interleaved K/V scale buffers
• MXFP8 fused prologue requires contiguous interleaved SFK/SFV
• MXFP8 fused decode prologue requires K/V scale buffers.
• native MXFP8 MoE only supports gated swiglu-oai, got {activa

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7e83f273274b2af3. Report an issue: GitHub.