sgl-project/sglang · error · ValueError

--prefill-only-disable-kv-cache is incompatible with --enabl

Error message

--prefill-only-disable-kv-cache is incompatible with --enable-prefill-cp: the prefill-CP path stages K/V through the paged cache, which the no-op pool does not support.

What it means

The prefill context-parallel path (--enable-prefill-cp) stages K/V through the paged cache, which the no-op KV pool used by --prefill-only-disable-kv-cache does not support. The validator rejects the combination at startup rather than failing mid-request.

Source

Thrown at python/sglang/srt/server_args.py:8202

                "between prefill chunks."
            )
        if not cfg.disable_radix_cache:
            raise ValueError(
                "--prefill-only-disable-kv-cache requires --disable-radix-cache because the "
                "radix cache indexes KV pool slots that no longer hold real data."
            )

        # Context-parallel prefill stages K/V through cp_allgather_and_save_kv_cache,
        # which writes to the pool via set_kv_buffer. NoOpMHATokenToKVPool intentionally
        # raises on writes, so the engine would boot fine but fail on the first request.
        if self._resolved().attn_cp_size > 1:
            raise ValueError(
                "--prefill-only-disable-kv-cache is incompatible with --attn-cp-size > 1: "
                "the context-parallel attention path writes K/V to the pool via set_kv_buffer, "
                "which the no-op pool intentionally rejects."
            )
        if cfg.enable_prefill_cp:
            raise ValueError(
                "--prefill-only-disable-kv-cache is incompatible with "
                "--enable-prefill-cp: the prefill-CP path stages K/V through "
                "the paged cache, which the no-op pool does not support."
            )

        # HiSparse selects a different pool class (HiSparseDSATokenToKVPool /
        # HiSparseTokenToKVPoolAllocator) that is not the no-op pool.
        if cfg.enable_hisparse:
            raise ValueError(
                "--prefill-only-disable-kv-cache is incompatible with --enable-hisparse: "
                "HiSparse uses a dedicated pool family that is not the no-op MHA pool."
            )

    def _handle_prefill_only_disable_kv_cache(self):
        """Validate --prefill-only-disable-kv-cache backend constraint.

        Must run after _handle_attention_backend_compatibility() (which fills
        the default attention_backend if unset) and _handle_multi_item_scoring()

View on GitHub (pinned to 0132848349)

Solutions

Remove --enable-prefill-cp when using --prefill-only-disable-kv-cache
Scale with tensor/data parallelism instead of prefill CP
If prefill CP is required, drop --prefill-only-disable-kv-cache

Example fix

# before
--prefill-only-disable-kv-cache --enable-prefill-cp
# after
--prefill-only-disable-kv-cache --tp-size 2

Defensive patterns

Strategy: validation

Validate before calling

if want_disable_kv_cache and enable_prefill_cp:
    raise SystemExit("--enable-prefill-cp is incompatible with --prefill-only-disable-kv-cache")

Prevention

Audit long-context flags (CP, prefill-CP) before enabling the no-op pool
Keep multi-GPU scaling via TP/DP for embedding workloads

When it happens

Trigger: Launching with --prefill-only-disable-kv-cache together with --enable-prefill-cp (cfg.enable_prefill_cp true).

Common situations: A user enables prefill CP to split long prefills across GPUs for an embedding/reranking workload, then adds KV-cache disabling to save memory.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d2fa9327347451fd. Report an issue: GitHub.