vllm-project/vllm · error · ValueError

Can't determine cudagraph shapes that are both a multiple of

Error message

Can't determine cudagraph shapes that are both a multiple of {uniform_decode_query_len} (num_speculative_tokens + 1) required by spec-decode and {tensor_parallel_size} (tensor_parallel_size) required by sequence parallelism please adjust num_speculative_tokens or disable sequence parallelism

What it means

Error "Can't determine cudagraph shapes that are both a multiple of {uniform_decode_query_len} (num_speculative_tokens + 1) required by spec-decode and {tensor_parallel_size} (tensor_parallel_size) required by sequence parallelism please adjust num_speculative_tokens or disable sequence parallelism" thrown in vllm-project/vllm.

Source

Thrown at vllm/config/compilation.py:1529

                "proceed. Please lower max_num_seqs to at most "
                f"{kv_cache_config.num_blocks} or increase "
                "gpu_memory_utilization."
            )

        self.cudagraph_mode = cudagraph_mode
        return cudagraph_mode

    def adjust_cudagraph_sizes_for_spec_decode(
        self, uniform_decode_query_len: int, tensor_parallel_size: int
    ):
        multiple_of = uniform_decode_query_len
        if tensor_parallel_size > 1 and self.pass_config.enable_sp:
            multiple_of = max(uniform_decode_query_len, tensor_parallel_size)
            if (
                multiple_of % uniform_decode_query_len != 0
                or multiple_of % tensor_parallel_size != 0
            ):
                raise ValueError(
                    f"Can't determine cudagraph shapes that are both a "
                    f"multiple of {uniform_decode_query_len} "
                    f"(num_speculative_tokens + 1) required by spec-decode "
                    f"and {tensor_parallel_size} (tensor_parallel_size) "
                    f"required by sequence parallelism please adjust "
                    f"num_speculative_tokens or disable sequence parallelism"
                )

        if not self.cudagraph_capture_sizes or multiple_of <= 1:
            return

        assert self.max_cudagraph_capture_size is not None
        rounded_sizes = sorted(
            set(
                round_up(size, multiple_of)
                for size in self.cudagraph_capture_sizes
                if round_up(size, multiple_of) <= self.max_cudagraph_capture_size
            )

View on GitHub (pinned to c794754062)

Solutions

1. Adjust num_speculative_tokens so that (num_speculative_tokens + 1) and tensor_parallel_size share compatible multiples.
2. Or disable sequence parallelism.

When it happens

Trigger: Raised at vllm/config/compilation.py:1529 when validation fails: Can't determine cudagraph shapes compatible with spec-decode and sequence parallelism. Typically triggered by an incompatible or incomplete vLLM configuration, an unsupported platform/backend combination, or a runtime resource/dependency that is missing.

Common situations: Commonly encountered at vllm/config/compilation.py:1529 during vLLM startup/config validation or runtime setup when: (1) conflicting CLI flags or config fields are combined, (2) the current platform (CUDA/ROCm/CPU/XPU) or installed optional packages do not support the requested feature, or (3) a required value is absent or out of range. Resolve by correcting the configuration as described in the message, or by selecting a supported alternative.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/40f64de4d27220c8. Report an issue: GitHub.