vllm-project/vllm · error · RuntimeError

CUDA graph capturing detected at an inappropriate time. This

Error message

CUDA graph capturing detected at an inappropriate time. This operation is currently disabled.

What it means

vLLM globally gates cudagraph capturing with a module-level flag (cudagraph_capturing_enabled in vllm/compilation/monitor.py). Capturing is only legal inside vLLM's designated capture windows; outside them validate_cudagraph_capturing_enabled() raises RuntimeError to prevent ad-hoc torch.cuda.graphs capture that would corrupt memory pools or state at the wrong lifecycle point.

Source

Thrown at vllm/compilation/monitor.py:96

        "backend compilation occurred during the initial profiling run; "
        "all compilation should be complete before the profiling run starts."
    )
    logger.info_once(
        "Initial profiling/warmup run took %.2f s",
        elapsed,
    )


cudagraph_capturing_enabled: bool = True


def validate_cudagraph_capturing_enabled() -> None:
    # used to monitor whether a cudagraph capturing is legal at runtime.
    # should be called before any cudagraph capturing.
    # if an illegal cudagraph capturing happens, raise an error.
    global cudagraph_capturing_enabled
    if not cudagraph_capturing_enabled:
        raise RuntimeError(
            "CUDA graph capturing detected at an inappropriate "
            "time. This operation is currently disabled."
        )


def set_cudagraph_capturing_enabled(enabled: bool) -> None:
    global cudagraph_capturing_enabled
    cudagraph_capturing_enabled = enabled

View on GitHub (pinned to c794754062)

Solutions

Move the capture into vLLM's official capture phase (let CUDAGraphWrapper capture it via compile_sizes/cudagraph_capture_sizes) instead of capturing manually.
If you must capture manually, bracket it with set_cudagraph_capturing_enabled(True) and restore to False afterwards — only if you own the lifecycle.
Disable cudagraph for the offending component (e.g. cudagraph_mode='') as a workaround.

Example fix

# before
class MyOp:
    def forward(self, x):
        if self._graph is None:
            g = torch.cuda.CUDAGraph()  # RuntimeError: CUDA graph capturing detected...
# after
class MyOp:
    def forward(self, x):
        return self._eager(x)  # let vLLM capture via its cudagraph wrapper

Defensive patterns

Strategy: validation

Validate before calling

from vllm.compilation.monitor import cudagraph_capturing_enabled

def safe_to_capture() -> bool:
    return cudagraph_capturing_enabled
# assert safe_to_capture() before any manual torch.cuda.CUDAGraph use

Try / catch

from vllm.compilation.monitor import validate_cudagraph_capturing_enabled
try:
    validate_cudagraph_capturing_enabled()
    with torch.cuda.graph(g):
        ...
except RuntimeError as e:
    if 'inappropriate time' in str(e):
        fall_back_to_eager()
    else:
        raise

Prevention

Never capture cudagraphs inside forward paths
Let CUDAGraphWrapper own all capture timing
Test custom kernels with cudagraph_mode='NONE' first, then FULL

When it happens

Trigger: Calling torch.cuda.CUDAGraph / graph capture (directly or via a library) at a point where vLLM has called set_cudagraph_capturing_enabled(False) — e.g. during memory profiling, worker warmup, or normal forward execution after the capture phase — and the capture path invokes validate_cudagraph_capturing_enabled().

Common situations: Custom plugins or quantization kernels that lazily capture their own cudagraph on first call; monkey-patched forwards triggering capture during profiling; user code calling CUDAGraphWrapper capture outside capture_one_batch size ranges.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/82c172d787b6e198. Report an issue: GitHub.