sgl-project/sglang · error · ValueError

--prefill-only-disable-kv-cache requires --chunked-prefill-s

Error message

--prefill-only-disable-kv-cache requires --chunked-prefill-size=-1 so the FA backend takes the fa_skip_kv_cache path; otherwise the pool would be touched between prefill chunks.

What it means

The FA backend only takes its fa_skip_kv_cache path (which never touches the pool) when a prefill fits in a single forward. chunked_prefill_size == -1 guarantees that; any other value means K/V must be reused across prefill chunks, so --prefill-only-disable-kv-cache requires it.

Source

Thrown at python/sglang/srt/server_args.py:8181

                "--prefill-only-disable-kv-cache does not currently support "
                "--kv-cache-dtype=nvfp4 or --kv-cache-dtype=fp4_mx_block16 because "
                "the FP4 pool uses a separate allocation path."
            )
        if cfg.kv_cache_dtype == "mxfp8":
            raise ValueError(
                "--prefill-only-disable-kv-cache does not currently support "
                "--kv-cache-dtype=mxfp8 because the MXFP8 pool stores separate "
                "scale-factor buffers."
            )

        # Structural preconditions for the FA backend's fa_skip_kv_cache path,
        # which is the only embedding path that doesn't read or write the pool:
        # - chunked_prefill_size == -1 keeps a request in a single forward,
        #   so K/V never has to be reused across prefill chunks.
        # - disable_radix_cache stops the prefix cache from indexing pool
        #   slots that no longer hold real data.
        if cfg.chunked_prefill_size != -1:
            raise ValueError(
                "--prefill-only-disable-kv-cache requires --chunked-prefill-size=-1 so the FA "
                "backend takes the fa_skip_kv_cache path; otherwise the pool would be touched "
                "between prefill chunks."
            )
        if not cfg.disable_radix_cache:
            raise ValueError(
                "--prefill-only-disable-kv-cache requires --disable-radix-cache because the "
                "radix cache indexes KV pool slots that no longer hold real data."
            )

        # Context-parallel prefill stages K/V through cp_allgather_and_save_kv_cache,
        # which writes to the pool via set_kv_buffer. NoOpMHATokenToKVPool intentionally
        # raises on writes, so the engine would boot fine but fail on the first request.
        if self._resolved().attn_cp_size > 1:
            raise ValueError(
                "--prefill-only-disable-kv-cache is incompatible with --attn-cp-size > 1: "
                "the context-parallel attention path writes K/V to the pool via set_kv_buffer, "
                "which the no-op pool intentionally rejects."

View on GitHub (pinned to 0132848349)

Solutions

  1. Set --chunked-prefill-size -1 in the launch command
  2. Check server logs/scripts for a chunked-prefill-size default injected by a profile or launch wrapper and override it
  3. Ensure the model's context length fits in one forward, since -1 disables chunking

Example fix

# before
--prefill-only-disable-kv-cache --chunked-prefill-size 8192
# after
--prefill-only-disable-kv-cache --chunked-prefill-size -1
Defensive patterns

Strategy: validation

Validate before calling

if want_disable_kv_cache and chunked_prefill_size != -1:
    raise SystemExit("set --chunked-prefill-size -1 for --prefill-only-disable-kv-cache")

Prevention

When it happens

Trigger: Launching with --prefill-only-disable-kv-cache while cfg.chunked_prefill_size != -1 (default chunked prefill, or an explicit positive value).

Common situations: A user copies a general serving config (which sets --chunked-prefill-size 8192 or relies on the default) and adds --prefill-only-disable-kv-cache for embedding serving.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/71fe9f867cede1c6. Report an issue: GitHub.