vllm-project/vllm · error · ValueError

max_num_seqs ({max_num_reqs}) exceeds available Mamba cache

Error message

max_num_seqs ({max_num_reqs}) exceeds available Mamba cache blocks ({kv_cache_config.num_blocks}). Each decode sequence requires one Mamba cache block, so CUDA graph capture cannot proceed. Please lower max_num_seqs to at most {kv_cache_config.num_blocks} or increase gpu_memory_utilization.

What it means

Error "max_num_seqs ({max_num_reqs}) exceeds available Mamba cache blocks ({kv_cache_config.num_blocks}). Each decode sequence requires one Mamba cache block, so CUDA graph capture cannot proceed. Please lower max_num_seqs to at most {kv_cache_config.num_blocks} or increase gpu_memory_utilization." thrown in vllm-project/vllm.

Source

Thrown at vllm/config/compilation.py:1507

                tensor_parallel_size,
            )

        # For Mamba models with FULL decode cudagraphs, each decode
        # sequence needs one Mamba cache block. The decode cudagraph
        # dispatcher already caps batch sizes at max_num_seqs, so we just
        # need to verify that enough blocks exist. Raising here instead
        # of silently capping cudagraph_capture_sizes avoids unintended
        # restrictions on PIECEWISE (prefill) cudagraphs.
        # See: https://github.com/vllm-project/vllm/issues/34094
        if (
            kv_cache_config is not None
            and max_num_reqs is not None
            and cudagraph_mode.has_full_cudagraphs()
            and not is_profiling
            and kv_cache_config.has_mamba_layers
            and max_num_reqs > kv_cache_config.num_blocks
        ):
            raise ValueError(
                f"max_num_seqs ({max_num_reqs}) exceeds available Mamba cache "
                f"blocks ({kv_cache_config.num_blocks}). Each decode sequence "
                "requires one Mamba cache block, so CUDA graph capture cannot "
                "proceed. Please lower max_num_seqs to at most "
                f"{kv_cache_config.num_blocks} or increase "
                "gpu_memory_utilization."
            )

        self.cudagraph_mode = cudagraph_mode
        return cudagraph_mode

    def adjust_cudagraph_sizes_for_spec_decode(
        self, uniform_decode_query_len: int, tensor_parallel_size: int
    ):
        multiple_of = uniform_decode_query_len
        if tensor_parallel_size > 1 and self.pass_config.enable_sp:
            multiple_of = max(uniform_decode_query_len, tensor_parallel_size)
            if (

View on GitHub (pinned to c794754062)

Solutions

1. Lower max_num_seqs to at most the number of available Mamba cache blocks reported in the error.
2. Or increase gpu_memory_utilization so more Mamba cache blocks are allocated.

When it happens

Trigger: Raised at vllm/config/compilation.py:1507 when validation fails: max_num_seqs exceeds available Mamba cache blocks so CUDA graph capture cannot proceed. Typically triggered by an incompatible or incomplete vLLM configuration, an unsupported platform/backend combination, or a runtime resource/dependency that is missing.

Common situations: Commonly encountered at vllm/config/compilation.py:1507 during vLLM startup/config validation or runtime setup when: (1) conflicting CLI flags or config fields are combined, (2) the current platform (CUDA/ROCm/CPU/XPU) or installed optional packages do not support the requested feature, or (3) a required value is absent or out of range. Resolve by correcting the configuration as described in the message, or by selecting a supported alternative.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/0d507f1f308fddab. Report an issue: GitHub.