vllm-project/vllm · error · ValueError

No valid cudagraph sizes after rounding to multiple of {mult

Error message

No valid cudagraph sizes after rounding to multiple of {multiple_of} (num_speculative_tokens + 1 or tp if sequence parallelism is enabled) please adjust num_speculative_tokens ({uniform_decode_query_len - 1}) or max_cudagraph_capture_size ({self.max_cudagraph_capture_size}) or cudagraph_capture_sizes ({self.cudagraph_capture_sizes})

What it means

Error "No valid cudagraph sizes after rounding to multiple of {multiple_of} (num_speculative_tokens + 1 or tp if sequence parallelism is enabled) please adjust num_speculative_tokens ({uniform_decode_query_len - 1}) or max_cudagraph_capture_size ({self.max_cudagraph_capture_size}) or cudagraph_capture_sizes ({self.cudagraph_capture_sizes})" thrown in vllm-project/vllm.

Source

Thrown at vllm/config/compilation.py:1555

        if not self.cudagraph_capture_sizes or multiple_of <= 1:
            return

        assert self.max_cudagraph_capture_size is not None
        rounded_sizes = sorted(
            set(
                round_up(size, multiple_of)
                for size in self.cudagraph_capture_sizes
                if round_up(size, multiple_of) <= self.max_cudagraph_capture_size
            )
        )

        if len(rounded_sizes) == 0 and multiple_of <= self.max_cudagraph_capture_size:
            # if one valid but would be round_down use that
            rounded_sizes = [multiple_of]

        if len(rounded_sizes) == 0:
            raise ValueError(
                f"No valid cudagraph sizes after rounding to multiple of {multiple_of} "
                f"(num_speculative_tokens + 1 or tp if sequence parallelism is enabled)"
                f" please adjust num_speculative_tokens ({uniform_decode_query_len - 1}"
                f") or max_cudagraph_capture_size ({self.max_cudagraph_capture_size})"
                f" or cudagraph_capture_sizes ({self.cudagraph_capture_sizes})"
            )

        self.max_cudagraph_capture_size = rounded_sizes[-1]
        self.cudagraph_capture_sizes = rounded_sizes

    def get_compile_ranges(self) -> list[Range]:
        """Get the compile ranges for the compilation config."""
        if self.compile_ranges_endpoints is None:
            return []
        endpoints = sorted(set(self.compile_ranges_endpoints))
        return [Range(s + 1, e) for s, e in zip([0] + endpoints[:-1], endpoints)]

View on GitHub (pinned to c794754062)

Solutions

1. Adjust num_speculative_tokens so at least one valid cudagraph size remains after rounding.
2. Or increase max_cudagraph_capture_size / adjust cudagraph_capture_sizes.

When it happens

Trigger: Raised at vllm/config/compilation.py:1555 when validation fails: No valid cudagraph sizes after rounding to the required multiple. Typically triggered by an incompatible or incomplete vLLM configuration, an unsupported platform/backend combination, or a runtime resource/dependency that is missing.

Common situations: Commonly encountered at vllm/config/compilation.py:1555 during vLLM startup/config validation or runtime setup when: (1) conflicting CLI flags or config fields are combined, (2) the current platform (CUDA/ROCm/CPU/XPU) or installed optional packages do not support the requested feature, or (3) a required value is absent or out of range. Resolve by correcting the configuration as described in the message, or by selecting a supported alternative.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/70e988eeb7cf15c1. Report an issue: GitHub.