sgl-project/sglang · error · ValueError

--prefill-only-disable-kv-cache requires --disable-radix-cac

Error message

--prefill-only-disable-kv-cache requires --disable-radix-cache because the radix cache indexes KV pool slots that no longer hold real data.

What it means

When the KV pool is disabled, its slots hold no real data, so the radix (prefix) cache would index stale/garbage slots and serve wrong results. --prefill-only-disable-kv-cache therefore requires --disable-radix-cache.

Source

Thrown at python/sglang/srt/server_args.py:8187

                "--prefill-only-disable-kv-cache does not currently support "
                "--kv-cache-dtype=mxfp8 because the MXFP8 pool stores separate "
                "scale-factor buffers."
            )

        # Structural preconditions for the FA backend's fa_skip_kv_cache path,
        # which is the only embedding path that doesn't read or write the pool:
        # - chunked_prefill_size == -1 keeps a request in a single forward,
        #   so K/V never has to be reused across prefill chunks.
        # - disable_radix_cache stops the prefix cache from indexing pool
        #   slots that no longer hold real data.
        if cfg.chunked_prefill_size != -1:
            raise ValueError(
                "--prefill-only-disable-kv-cache requires --chunked-prefill-size=-1 so the FA "
                "backend takes the fa_skip_kv_cache path; otherwise the pool would be touched "
                "between prefill chunks."
            )
        if not cfg.disable_radix_cache:
            raise ValueError(
                "--prefill-only-disable-kv-cache requires --disable-radix-cache because the "
                "radix cache indexes KV pool slots that no longer hold real data."
            )

        # Context-parallel prefill stages K/V through cp_allgather_and_save_kv_cache,
        # which writes to the pool via set_kv_buffer. NoOpMHATokenToKVPool intentionally
        # raises on writes, so the engine would boot fine but fail on the first request.
        if self._resolved().attn_cp_size > 1:
            raise ValueError(
                "--prefill-only-disable-kv-cache is incompatible with --attn-cp-size > 1: "
                "the context-parallel attention path writes K/V to the pool via set_kv_buffer, "
                "which the no-op pool intentionally rejects."
            )
        if cfg.enable_prefill_cp:
            raise ValueError(
                "--prefill-only-disable-kv-cache is incompatible with "
                "--enable-prefill-cp: the prefill-CP path stages K/V through "
                "the paged cache, which the no-op pool does not support."

View on GitHub (pinned to 0132848349)

Solutions

  1. Add --disable-radix-cache to the launch command
  2. Also disable derived prefix-cache features (e.g. HiCache settings) that imply radix caching
  3. Rerun and confirm no other radix-cache-enabling flags remain

Example fix

# before
--prefill-only-disable-kv-cache
# after
--prefill-only-disable-kv-cache --disable-radix-cache --chunked-prefill-size -1 --is-embedding
Defensive patterns

Strategy: validation

Validate before calling

if want_disable_kv_cache and not disable_radix_cache:
    raise SystemExit("add --disable-radix-cache with --prefill-only-disable-kv-cache")

Prevention

When it happens

Trigger: Launching with --prefill-only-disable-kv-cache without --disable-radix-cache (i.e. prefix caching is enabled by default or explicitly).

Common situations: A user enables the flag on an embedding server but leaves prefix caching on (the default), or a shared config template sets radix-cache-related flags.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f9560420891054dc. Report an issue: GitHub.