vllm-project/vllm · error · ValueError

customized max_cudagraph_capture_size(={self.compilation_con

Error message

customized max_cudagraph_capture_size(={self.compilation_config.max_cudagraph_capture_size}) should be consistent with the max value of cudagraph_capture_sizes(={valid_max_size})

What it means

In `_set_cudagraph_sizes`, vLLM truncates `max_cudagraph_capture_size` to the largest entry of `cudagraph_capture_sizes` when only one is given. If BOTH `--compilation-config.max_cudagraph_capture_size` and an explicit `cudagraph_capture_sizes` list are set and their maxima disagree, it is treated as a user error rather than silently truncating.

Source

Thrown at vllm/config/vllm.py:2015

                and self.compilation_config.pass_config.enable_sp
            ):
                cudagraph_capture_sizes = self.update_sizes_for_sequence_parallelism(
                    cudagraph_capture_sizes
                )

            # user-specific compilation_config.max_cudagraph_capture_size get
            # truncated to valid_max_size when they are inconsistent.
            valid_max_size = (
                cudagraph_capture_sizes[-1] if cudagraph_capture_sizes else 0
            )
            if (
                self.compilation_config.max_cudagraph_capture_size is not None
                and self.compilation_config.max_cudagraph_capture_size != valid_max_size
            ):
                # raise error only when both two flags are user-specified
                # and they are inconsistent with each other
                if self.compilation_config.cudagraph_capture_sizes is not None:
                    raise ValueError(
                        "customized max_cudagraph_capture_size"
                        f"(={self.compilation_config.max_cudagraph_capture_size}) "
                        "should be consistent with the max value of "
                        f"cudagraph_capture_sizes(={valid_max_size})"
                    )

                logger.warning(
                    "Truncating max_cudagraph_capture_size to %d",
                    valid_max_size,
                )
            # always set the final max_cudagraph_capture_size
            self.compilation_config.max_cudagraph_capture_size = valid_max_size

            if self.compilation_config.cudagraph_capture_sizes is not None and len(
                cudagraph_capture_sizes
            ) < len(self.compilation_config.cudagraph_capture_sizes):
                # If users have specified capture sizes, we only need to
                # compare the lens before and after modification since the modified

View on GitHub (pinned to c794754062)

Solutions

Make max_cudagraph_capture_size equal to the last (max) entry of your cudagraph_capture_sizes list.
Or specify only one of the two options — set only cudagraph_capture_sizes, or only max_cudagraph_capture_size and let vLLM truncate.

Example fix

# before
--compilation-config '{"max_cudagraph_capture_size": 128,
                     "cudagraph_capture_sizes": [1,2,4,8,16,32,64]}'

# after
--compilation-config '{"max_cudagraph_capture_size": 64,
                     "cudagraph_capture_sizes": [1,2,4,8,16,32,64]}'

Defensive patterns

Strategy: validation

Validate before calling

cc = {"cudagraph_capture_sizes": sizes} if sizes else {"max_cudagraph_capture_size": mx}
# never set both; or enforce consistency:
assert "max_cudagraph_capture_size" not in cc or cc["max_cudagraph_capture_size"] == max(cc["cudagraph_capture_sizes"])

Try / catch

try:
    LLM(compilation_config=cc, ...)
except ValueError as e:
    if "max_cudagraph_capture_size" in str(e):
        cc.pop("max_cudagraph_capture_size", None)  # let sizes list drive it
    else:
        raise

Prevention

Set either the sizes list or the max, not both
Derive max programmatically as max(sizes) to avoid drift

When it happens

Trigger: Passing both `max_cudagraph_capture_size` and a `cudagraph_capture_sizes` list via --compilation-config whose maximum element differs from max_cudagraph_capture_size.

Common situations: Fine-tuning CUDA graph capture ranges (e.g. capping graph size for memory) and specifying the full sizes list plus the max separately, with one number off by a power of two.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/a9b048cc956c06b9. Report an issue: GitHub.