sgl-project/sglang · error · RuntimeError

--prefill-only-disable-kv-cache expected NoOpMHATokenToKVPoo

Error message

--prefill-only-disable-kv-cache expected NoOpMHATokenToKVPool but the runtime pool is {type(token_to_kv_pool).__name__}. This pool family is not yet supported by --prefill-only-disable-kv-cache. Supported configurations today: plain MHA models on CUDA with the FA (fa3/fa4) prefill backend, --is-embedding, --chunked-prefill-size=-1, --disable-radix-cache, no context-parallel attention, no HiSparse, and --kv-cache-dtype not in {nvfp4, fp4_mx_block16}.

What it means

A post-boot invariant check: when --prefill-only-disable-kv-cache is set (and this is not the draft worker), the runtime KV pool must be a NoOpMHATokenToKVPool. Any other pool class means the configuration quietly produced a real/unsupported pool, so the server fails instead of misbehaving at first request.

Source

Thrown at python/sglang/srt/mem_cache/kv_cache_configurator.py:532

            )

        token_to_kv_pool_allocator = self._build_token_to_kv_pool_allocator(
            sizes=sizes,
            token_to_kv_pool=token_to_kv_pool,
            is_dsv4_model=is_dsv4_model,
            req_to_token_pool=req_to_token_pool,
            token_to_kv_pool_allocator=token_to_kv_pool_allocator,
        )

        # Defensive check: the explicit validation above should reject known
        # unsupported pool families before allocation. Keep this guard here so
        # future pool-selection refactors fail at boot instead of on first use.
        if (
            get_schedule().prefill_only_disable_kv_cache
            and not self.is_draft_worker
            and not isinstance(token_to_kv_pool, NoOpMHATokenToKVPool)
        ):
            raise RuntimeError(
                "--prefill-only-disable-kv-cache expected NoOpMHATokenToKVPool but the "
                f"runtime pool is {type(token_to_kv_pool).__name__}. This pool "
                "family is not yet supported by --prefill-only-disable-kv-cache. "
                "Supported configurations today: plain MHA models on CUDA with the FA "
                "(fa3/fa4) prefill backend, --is-embedding, --chunked-prefill-size=-1, "
                "--disable-radix-cache, no context-parallel attention, no HiSparse, "
                "and --kv-cache-dtype not in {nvfp4, fp4_mx_block16}."
            )
        return _InitializedPools(
            req_to_token_pool=req_to_token_pool,
            token_to_kv_pool=token_to_kv_pool,
            token_to_kv_pool_allocator=token_to_kv_pool_allocator,
        )

    def _init_unified_mamba_pools(
        self, *, max_num_reqs: int, max_total_num_tokens: int
    ) -> UnifiedPoolBundle:
        """Build the shared-KV-pool stack for a hybrid-Mamba model:

View on GitHub (pinned to 0132848349)

Solutions

1. Check the message for the actual pool class and remove the unsupported option (e.g. drop --kv-cache-dtype nvfp4, disable CP/HiSparse)
2. Only use --prefill-only-disable-kv-cache with plain MHA models on CUDA + FA (fa3/fa4) prefill backend, --is-embedding, --chunked-prefill-size=-1, --disable-radix-cache
3. If you believe the config should be supported, report a bug — the pool family list is the source of truth

Example fix

# before
python -m sglang.launch_server --model qwen-... --prefill-only-disable-kv-cache --kv-cache-dtype nvfp4
# after
python -m sglang.launch_server --model qwen-... --prefill-only-disable-kv-cache --kv-cache-dtype auto

sa = server_args
ok = (sa.prefill_only_disable_kv_cache and sa.device == "cuda"
      and sa.attention_backend in ("fa3", "fa4")
      and sa.chunked_prefill_size == -1 and sa.disable_radix_cache
      and sa.kv_cache_dtype not in ("nvfp4", "fp4_mx_block16")
      and not sa.enable_context_parallel and not getattr(sa, "enable_hiparse", False))
assert ok, "--prefill-only-disable-kv-cache config unsupported"

When it happens

Trigger: Enable --prefill-only-disable-kv-cache on a config that doesn't reduce to a plain MHA CUDA FA3/FA4 pool: hybrid/Mamba models, quantized KV dtype (nvfp4/fp4_mx_block16), radix cache on, chunked prefill enabled, CP attention, HiSparse, etc.

Common situations: Using the prefill-only (no KV cache) mode on an unsupported model/backend, or a pool-selection refactor changing which pool class gets built.

Related errors

• --enable-unified-memory with PD disaggregation does not supp
• --enable-unified-memory only supports hybrid Mamba and hybri
• --prefill-only-disable-kv-cache is not supported for {unsupp
• NoOpMHATokenToKVPool.set_kv_buffer was called. This pool is
• v_cache must be provided

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/67bc7c91ced148e6. Report an issue: GitHub.