vllm-project/vllm · error · RuntimeError

Nested BreakableCUDAGraphCapture is not supported.

Error message

Nested BreakableCUDAGraphCapture is not supported.

What it means

BreakableCUDAGraphCapture is a context manager that records a CUDA graph in multiple segments (graph capture, eager break, graph capture ...). It registers itself in thread-local state (_tls.active) for nested code to detect re-entry; entering it while another instance is already active raises RuntimeError, because nested capture pools/segments are unsupported.

Source

Thrown at vllm/compilation/breakable_cudagraph.py:162

        return getattr(cls._tls, "active", None)

    @classmethod
    def is_active(cls) -> bool:
        return cls.current() is not None

    def __init__(self, pool: Any | None = None) -> None:
        self.pool = pool
        self.segments: list[Callable[[], Any]] = []
        self._num_graphs: int = 0
        self._num_eager_breaks: int = 0
        self._current_graph: torch.cuda.CUDAGraph | None = None
        self._capturing: bool = False

    # --- context manager protocol ----------------------------------------

    def __enter__(self) -> BreakableCUDAGraphCapture:
        if getattr(BreakableCUDAGraphCapture._tls, "active", None) is not None:
            raise RuntimeError("Nested BreakableCUDAGraphCapture is not supported.")
        BreakableCUDAGraphCapture._tls.active = self
        self._begin_segment()
        return self

    def __exit__(self, exc_type, exc, tb) -> None:
        try:
            self._end_segment()
        finally:
            BreakableCUDAGraphCapture._tls.active = None

    # --- segment management ----------------------------------------------

    def _begin_segment(self) -> None:
        assert not self._capturing
        g = torch.cuda.CUDAGraph()
        if self.pool is not None:
            g.capture_begin(pool=self.pool)
        else:

View on GitHub (pinned to c794754062)

Solutions

1. Check the guard before entering: `if getattr(BreakableCUDAGraphCapture._tls, 'active', None) is None:` and reuse the active capture instead of nesting
2. Refactor so only one layer owns capture; inner code should add segments to the active capture rather than open a new one
3. Disable CUDA graph capture (enforce_eager=True / -O0 cudagraph mode) for the model that cannot avoid nesting

Example fix

# before
with BreakableCUDAGraphCapture(pool):  # inside another capture -> raises
    run(model)
# after
from vllm.compilation.breakable_cudagraph import BreakableCUDAGraphCapture
if getattr(BreakableCUDAGraphCapture._tls, "active", None) is None:
    with BreakableCUDAGraphCapture(pool):
        run(model)
else:
    run(model)  # already capturing; add to active capture

from vllm.compilation.breakable_cudagraph import BreakableCUDAGraphCapture
active = getattr(BreakableCUDAGraphCapture._tls, "active", None)
assert active is None, "already inside a BreakableCUDAGraphCapture"

def can_start_capture() -> bool:
    from vllm.compilation.breakable_cudagraph import BreakableCUDAGraphCapture
    return getattr(BreakableCUDAGraphCapture._tls, "active", None) is None

When it happens

Trigger: Entering `with BreakableCUDAGraphCapture(...)` inside the body of another live `with BreakableCUDAGraphCapture(...)` block on the same thread; e.g. a model's capture helper that itself uses the context manager being run inside an outer capture.

Common situations: Custom model code or a wrapper that unconditionally starts breakable capture, invoked within vLLM's own CUDA-graph capture phase; plugins/hooks that add their own graph capture around quantization or attention modules.

Related errors

• failed to build request runtime
• Expected {ty} but got {type(value)} for {value}
• PostGradPassManager can not be kept in CompilationConfig.
• call_module is not allowed for codegen target {target}.
• Unsupported node from codegen: {node.format_node()}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/d5ab881ab482f778. Report an issue: GitHub.