vllm-project/vllm · error · ValueError

--mamba-block-size can only be set with --enable-prefix-cach

Error message

--mamba-block-size can only be set with --enable-prefix-caching

What it means

A non-default `--mamba-block-size` (set and different from max_model_len) only makes sense with the hybrid KV cache manager + prefix caching, because mamba state blocks are only managed that way. The `validate_mamba_block_size` model validator requires `--enable-prefix-caching` when mamba_block_size is overridden.

Source

Thrown at vllm/config/vllm.py:2553

            and self.model_config.use_mla
        ):
            raise ValueError(
                "nvfp4 KV cache is not supported with MLA (Multi-head Latent "
                "Attention) backends. Please use a different --kv-cache-dtype "
                "(e.g., 'fp8' or 'auto') for MLA models such as DeepSeek."
            )
        return self

    @model_validator(mode="after")
    def validate_mamba_block_size(self) -> "VllmConfig":
        if self.model_config is None:
            return self
        mamba_block_size_is_set = (
            self.cache_config.mamba_block_size is not None
            and self.cache_config.mamba_block_size != self.model_config.max_model_len
        )
        if mamba_block_size_is_set and not self.cache_config.enable_prefix_caching:
            raise ValueError(
                "--mamba-block-size can only be set with --enable-prefix-caching"
            )
        return self

    @model_validator(mode="after")
    def validate_mamba_cached_kernel(self) -> "VllmConfig":
        if not self.cache_config.use_replayssm:
            return self
        # ReplaySSM adds a 3-tensor ring to the mamba state; only models that
        # opt in (supports_replayssm) build a consistent shape on both the layer
        # and config paths. Reject others so the mamba page size cannot desync.
        if self.model_config is not None and not self.model_config.supports_replayssm:
            raise ValueError(
                "--use-replayssm is only supported for Nemotron-H models "
                f"(got architecture {self.model_config.architecture!r})"
            )
        if self.cache_config.mamba_cache_mode == "all":
            raise ValueError(

View on GitHub (pinned to c794754062)

Solutions

1. Add `--enable-prefix-caching` to the launch command.
2. Or remove `--mamba-block-size` to use the default (block size follows max_model_len without prefix caching).

Example fix

# before
vllm serve nvidia/Nemotron-H-8B-v1 --mamba-block-size 128

# after
vllm serve nvidia/Nemotron-H-8B-v1 --mamba-block-size 128 --enable-prefix-caching

if mamba_block_size is not None and mamba_block_size != max_model_len and not enable_prefix_caching:
    enable_prefix_caching = True

try:
    LLM(mamba_block_size=128, ...)
except ValueError as e:
    if "mamba-block-size" in str(e):
        LLM(mamba_block_size=128, enable_prefix_caching=True, ...)
    else:
        raise

When it happens

Trigger: Launching a Mamba/hybrid model with `--mamba-block-size N` (N != max_model_len) but without `--enable-prefix-caching`.

Common situations: Tuning mamba state page sizes for memory footprint on Mamba2/Nemotron-H models while prefix caching was disabled (e.g. by a baseline benchmarking script).

Related errors

• Stochastic rounding for Mamba cache requires the SSM cache t
• --use-replayssm supports prefix caching only in align mode;
• Currently, async scheduling is only supported with EAGLE/MTP
• Async scheduling is not compatible with disable_padded_draft
• The Proton profiler requires CUDA graphs to be disabled. Use

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/a5d68dda36d645c8. Report an issue: GitHub.