sgl-project/sglang · error · ValueError

--prefill-only-disable-kv-cache does not currently support -

Error message

--prefill-only-disable-kv-cache does not currently support --kv-cache-dtype=mxfp8 because the MXFP8 pool stores separate scale-factor buffers.

What it means

--prefill-only-disable-kv-cache swaps in a no-op MHA KV pool, but the MXFP8 pool keeps separate scale-factor buffers alongside the data, which the no-op pool path does not manage. The validator therefore rejects --kv-cache-dtype=mxfp8 when KV-cache disabling is enabled.

Source

Thrown at python/sglang/srt/server_args.py:8168

        # This flag is intentionally scoped to embedding mode for now. Other
        # prefill-only paths (for example scoring and MIS) can benefit from
        # the same idea later, but some of them still stage K/V through the
        # paged cache today.
        if not cfg.is_embedding:
            raise ValueError(
                "--prefill-only-disable-kv-cache currently requires --is-embedding. "
                "Other prefill-only workloads may be supported in a future change once "
                "their attention paths stop reading or writing the paged KV cache."
            )
        if cfg.kv_cache_dtype in ("nvfp4", "fp4_mx_block16"):
            raise ValueError(
                "--prefill-only-disable-kv-cache does not currently support "
                "--kv-cache-dtype=nvfp4 or --kv-cache-dtype=fp4_mx_block16 because "
                "the FP4 pool uses a separate allocation path."
            )
        if cfg.kv_cache_dtype == "mxfp8":
            raise ValueError(
                "--prefill-only-disable-kv-cache does not currently support "
                "--kv-cache-dtype=mxfp8 because the MXFP8 pool stores separate "
                "scale-factor buffers."
            )

        # Structural preconditions for the FA backend's fa_skip_kv_cache path,
        # which is the only embedding path that doesn't read or write the pool:
        # - chunked_prefill_size == -1 keeps a request in a single forward,
        #   so K/V never has to be reused across prefill chunks.
        # - disable_radix_cache stops the prefix cache from indexing pool
        #   slots that no longer hold real data.
        if cfg.chunked_prefill_size != -1:
            raise ValueError(
                "--prefill-only-disable-kv-cache requires --chunked-prefill-size=-1 so the FA "
                "backend takes the fa_skip_kv_cache path; otherwise the pool would be touched "
                "between prefill chunks."
            )
        if not cfg.disable_radix_cache:

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --kv-cache-dtype=mxfp8 when using --prefill-only-disable-kv-cache
2. If MXFP8 quantization is required, drop --prefill-only-disable-kv-cache
3. Watch upstream for MXFP8 scale-factor buffer support in the no-op pool path

Example fix

# before
--is-embedding --prefill-only-disable-kv-cache --kv-cache-dtype mxfp8
# after
--is-embedding --prefill-only-disable-kv-cache

if want_disable_kv_cache and kv_cache_dtype == "mxfp8":
    raise SystemExit("mxfp8 KV dtype is incompatible with --prefill-only-disable-kv-cache")

When it happens

Trigger: Launching with --prefill-only-disable-kv-cache and --kv-cache-dtype mxfp8 (cfg.kv_cache_dtype == "mxfp8").

Common situations: Enabling MXFP8 KV cache quantization (common on Blackwell for memory savings) on an embedding server that also disables the KV cache.

Related errors

• MXFP8 KV cache requires the FA4 backend.
• MXFP8 KV cache requires K and V scale tensors.
• --prefill-only-disable-kv-cache does not currently support -
• MXFP8 KV cache requires per-token Q scales (q_descale) from
• MXFP8 KV cache requires head_dim divisible by {self.MXFP8_SC

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fa51025f25e3ec7b. Report an issue: GitHub.