sgl-project/sglang · error · ValueError

--prefill-only-disable-kv-cache is incompatible with --attn-

Error message

--prefill-only-disable-kv-cache is incompatible with --attn-cp-size > 1: the context-parallel attention path writes K/V to the pool via set_kv_buffer, which the no-op pool intentionally rejects.

What it means

Context-parallel prefill stages K/V through cp_allgather_and_save_kv_cache, which writes to the pool via set_kv_buffer. The NoOpMHATokenToKVPool intentionally raises on writes, so the engine would boot but fail on the first request; the validator rejects --attn-cp-size > 1 up front.

Source

Thrown at python/sglang/srt/server_args.py:8196

        # - disable_radix_cache stops the prefix cache from indexing pool
        #   slots that no longer hold real data.
        if cfg.chunked_prefill_size != -1:
            raise ValueError(
                "--prefill-only-disable-kv-cache requires --chunked-prefill-size=-1 so the FA "
                "backend takes the fa_skip_kv_cache path; otherwise the pool would be touched "
                "between prefill chunks."
            )
        if not cfg.disable_radix_cache:
            raise ValueError(
                "--prefill-only-disable-kv-cache requires --disable-radix-cache because the "
                "radix cache indexes KV pool slots that no longer hold real data."
            )

        # Context-parallel prefill stages K/V through cp_allgather_and_save_kv_cache,
        # which writes to the pool via set_kv_buffer. NoOpMHATokenToKVPool intentionally
        # raises on writes, so the engine would boot fine but fail on the first request.
        if self._resolved().attn_cp_size > 1:
            raise ValueError(
                "--prefill-only-disable-kv-cache is incompatible with --attn-cp-size > 1: "
                "the context-parallel attention path writes K/V to the pool via set_kv_buffer, "
                "which the no-op pool intentionally rejects."
            )
        if cfg.enable_prefill_cp:
            raise ValueError(
                "--prefill-only-disable-kv-cache is incompatible with "
                "--enable-prefill-cp: the prefill-CP path stages K/V through "
                "the paged cache, which the no-op pool does not support."
            )

        # HiSparse selects a different pool class (HiSparseDSATokenToKVPool /
        # HiSparseTokenToKVPoolAllocator) that is not the no-op pool.
        if cfg.enable_hisparse:
            raise ValueError(
                "--prefill-only-disable-kv-cache is incompatible with --enable-hisparse: "
                "HiSparse uses a dedicated pool family that is not the no-op MHA pool."
            )

View on GitHub (pinned to 0132848349)

Solutions

Remove or set --attn-cp-size 1 when using --prefill-only-disable-kv-cache
Use tensor parallelism (--tp-size) instead of context parallelism for multi-GPU embedding serving
If CP is essential, drop --prefill-only-disable-kv-cache and keep the real pool

Example fix

# before
--prefill-only-disable-kv-cache --attn-cp-size 2
# after
--prefill-only-disable-kv-cache --attn-cp-size 1 --tp-size 2

Defensive patterns

Strategy: validation

Validate before calling

if want_disable_kv_cache and attn_cp_size > 1:
    raise SystemExit("use --tp-size instead of --attn-cp-size with --prefill-only-disable-kv-cache")

Prevention

Do not combine context parallelism with pool-free serving
Prefer TP for multi-GPU embedding servers

When it happens

Trigger: Launching with --prefill-only-disable-kv-cache and --attn-cp-size set to a value greater than 1.

Common situations: A long-context embedding workload (e.g. large-document embeddings) where the user enables attention context parallelism for memory, then also disables the KV cache.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fa6eedb1a4719698. Report an issue: GitHub.