sgl-project/sglang · error · RuntimeError

--prefill-only-disable-kv-cache is not supported for {unsupp

Error message

--prefill-only-disable-kv-cache is not supported for {unsupported_pool_family}. Supported configurations today: plain MHA models on CUDA with the FA (fa3/fa4) prefill backend, --is-embedding, --chunked-prefill-size=-1, --disable-radix-cache, no context-parallel attention, no HiSparse, and --kv-cache-dtype not in {nvfp4, fp4_mx_block16}.

What it means

A pre-pool validation in _validate_prefill_only_disable_kv_cache_pool_family: --prefill-only-disable-kv-cache rejects pool families it cannot back with a NoOp pool — currently hybrid linear/Mamba KV pools and FP4 (float4_e2m1fn_x2) MHA pools are named as unsupported.

Source

Thrown at python/sglang/srt/mem_cache/kv_cache_configurator.py:751

        elif (
            get_exec().kernel.attention_backend == "ascend" and not self.mambaish_config
        ):
            unsupported_pool_family = "NPU/Ascend KV pool"
        elif self.use_mla_backend and self.is_hybrid_swa:
            unsupported_pool_family = "hybrid DSA/MLA-SWA KV pool"
        elif self.use_mla_backend and is_dsa_model:
            unsupported_pool_family = "DSA/MLA KV pool"
        elif self.use_mla_backend and not self.mambaish_config:
            unsupported_pool_family = "MLA KV pool"
        elif self.is_hybrid_swa:
            unsupported_pool_family = "SWA KV pool"
        elif self.mambaish_config:
            unsupported_pool_family = "hybrid linear/Mamba KV pool"
        elif is_float4_e2m1fn_x2(self.kv_cache_dtype):
            unsupported_pool_family = "FP4 MHA KV pool"

        if unsupported_pool_family is not None:
            raise RuntimeError(
                "--prefill-only-disable-kv-cache is not supported for "
                f"{unsupported_pool_family}. Supported configurations today: plain MHA "
                "models on CUDA with the FA (fa3/fa4) prefill backend, --is-embedding, "
                "--chunked-prefill-size=-1, --disable-radix-cache, no context-parallel "
                "attention, no HiSparse, and --kv-cache-dtype not in {nvfp4, fp4_mx_block16}."
            )

    def _build_req_to_token_pool(self, *, max_num_reqs: int) -> ReqToTokenPool:
        extra_max_context_len = get_req_to_token_extra_context_len()

        if get_disagg().disaggregation_mode == "decode":
            # Extra slots for pre-allocated requests
            pre_alloc_size = get_disagg().disaggregation_decode_extra_slots
            if self.mambaish_config:
                req_to_token_pool = self._build_hybrid_mamba_decode_req_pool(
                    max_num_reqs=max_num_reqs,
                    extra_max_context_len=extra_max_context_len,
                    pre_alloc_size=pre_alloc_size,

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --prefill-only-disable-kv-cache
2. Switch to a plain MHA model with auto/fp8 KV dtype if prefill-only mode is required

Example fix

# before
python -m sglang.launch_server --model falcon-h1-... --prefill-only-disable-kv-cache
# after
python -m sglang.launch_server --model falcon-h1-...

if server_args.prefill_only_disable_kv_cache and (is_mambaish(model_config) or server_args.kv_cache_dtype in ("nvfp4", "fp4_mx_block16")):
    raise SystemExit("prefill-only-disable-kv-cache unsupported for this pool family")

When it happens

Trigger: Set --prefill-only-disable-kv-cache on a model with a mambaish_config (hybrid linear/Mamba), or with --kv-cache-dtype nvfp4/fp4_mx_block16; the named unsupported_pool_family branch raises.

Common situations: Prefill-only serving (context embedding / ingest-only) attempted on hybrid SSM models or FP4-quantized KV cache.

Related errors

• --prefill-only-disable-kv-cache expected NoOpMHATokenToKVPoo
• NoOpMHATokenToKVPool.set_kv_buffer was called. This pool is
• v_cache must be provided
• k_cache can only be None when only_qv=True
• rope_pool_fused expects pool tensors to be 3-D

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/16d22f35a78c6f93. Report an issue: GitHub.