vllm-project/vllm · error · NotImplementedError

cudagraph_capture_sizes not supported in compile_sizes.This

Error message

cudagraph_capture_sizes not supported in compile_sizes.This should be handled in `post_init_cudagraph_sizes`.

What it means

PiecewiseBackend separates compile-time sizes (compile_sizes as concrete integers) from cudagraph capture sizes, which must be finalized after initialization (post_init_cudagraph_sizes) because they depend on runtime state. Passing the magic string 'cudagraph_capture_sizes' inside compile_sizes is explicitly rejected with NotImplementedError.

Source

Thrown at vllm/compilation/piecewise_backend.py:169

        log_string = f"PiecewiseBackend: compile_ranges: {self.compile_ranges}"
        logger.debug_once(log_string)

        self.compile_sizes = self.compilation_config.compile_sizes
        log_string = f"PiecewiseBackend: compile_sizes: {self.compile_sizes}"
        logger.debug_once(log_string)

        self.sym_shape_indices = sym_shape_indices
        self.returns_tuple = returns_tuple

        # the entries for ranges that we need to either
        self.range_entries: dict[Range, RangeEntry] = {}

        # We only keep compilation management inside this class directly.
        if self.compile_sizes is not None:
            for size in self.compile_sizes:
                if isinstance(size, str):
                    assert size == "cudagraph_capture_sizes"
                    raise NotImplementedError(
                        "cudagraph_capture_sizes not supported in compile_sizes."
                        "This should be handled in `post_init_cudagraph_sizes`."
                    )
                else:
                    assert isinstance(size, int)
                    range = Range(start=size, end=size)
                    if range not in self.compile_ranges:
                        self.range_entries[range] = RangeEntry(
                            compile_range=range,
                        )

        for range in self.compile_ranges:
            self.range_entries[range] = RangeEntry(
                compile_range=range,
            )

        # Track whether we've logged the graph for this subgraph (only log once)
        self._graph_logged = False

View on GitHub (pinned to c794754062)

Solutions

1. Remove 'cudagraph_capture_sizes' from compile_sizes and supply capture sizes via post_init_cudagraph_sizes().
2. Pass only concrete integer sizes in compile_sizes.

Example fix

# before
backend = PiecewiseBackend(..., compile_sizes=['cudagraph_capture_sizes', 1, 2])
# after
backend = PiecewiseBackend(..., compile_sizes=[1, 2, 4])
# then set capture sizes via post_init_cudagraph_sizes

compile_sizes = [s for s in backend.compile_sizes or [] if not isinstance(s, str)]
assert all(isinstance(s, int) for s in compile_sizes)

When it happens

Trigger: Constructing PiecewiseBackend (or a backend factory returning it) with compile_sizes=['cudagraph_capture_sizes', ...] instead of leaving cudagraph sizes to the dedicated post-init mechanism.

Common situations: Writing a custom compilation backend that mimics vllm's VllmBackend sizes handling but forwards the cudagraph string into compile_sizes; misreading the backend API when porting from an older version where the string was tolerated.

Related errors

• vLLM failed to compile the model. The most likely reason for
• Attribute {key} not exists in the runnable of cudagraph wrap
• Source code has changed since the last compilation. Recompil
• CUDA graph capturing detected at an inappropriate time. This
• Input {arg} to maybe_inplace node {node} is used again after

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/ddc0dd9b4500d875. Report an issue: GitHub.