{"record":{"id":"82c172d787b6e198","repo":"vllm-project/vllm","slug":"cuda-graph-capturing-detected-at-an-inappropriate","errorCode":null,"errorMessage":"CUDA graph capturing detected at an inappropriate time. This operation is currently disabled.","messagePattern":"CUDA graph capturing detected at an inappropriate time\\. This operation is currently disabled\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"vllm/compilation/monitor.py","lineNumber":96,"sourceCode":"        \"backend compilation occurred during the initial profiling run; \"\n        \"all compilation should be complete before the profiling run starts.\"\n    )\n    logger.info_once(\n        \"Initial profiling/warmup run took %.2f s\",\n        elapsed,\n    )\n\n\ncudagraph_capturing_enabled: bool = True\n\n\ndef validate_cudagraph_capturing_enabled() -> None:\n    # used to monitor whether a cudagraph capturing is legal at runtime.\n    # should be called before any cudagraph capturing.\n    # if an illegal cudagraph capturing happens, raise an error.\n    global cudagraph_capturing_enabled\n    if not cudagraph_capturing_enabled:\n        raise RuntimeError(\n            \"CUDA graph capturing detected at an inappropriate \"\n            \"time. This operation is currently disabled.\"\n        )\n\n\ndef set_cudagraph_capturing_enabled(enabled: bool) -> None:\n    global cudagraph_capturing_enabled\n    cudagraph_capturing_enabled = enabled\n","sourceCodeStart":78,"sourceCodeEnd":105,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/compilation/monitor.py#L78-L105","documentation":"vLLM globally gates cudagraph capturing with a module-level flag (cudagraph_capturing_enabled in vllm/compilation/monitor.py). Capturing is only legal inside vLLM's designated capture windows; outside them validate_cudagraph_capturing_enabled() raises RuntimeError to prevent ad-hoc torch.cuda.graphs capture that would corrupt memory pools or state at the wrong lifecycle point.","triggerScenarios":"Calling torch.cuda.CUDAGraph / graph capture (directly or via a library) at a point where vLLM has called set_cudagraph_capturing_enabled(False) — e.g. during memory profiling, worker warmup, or normal forward execution after the capture phase — and the capture path invokes validate_cudagraph_capturing_enabled().","commonSituations":"Custom plugins or quantization kernels that lazily capture their own cudagraph on first call; monkey-patched forwards triggering capture during profiling; user code calling CUDAGraphWrapper capture outside capture_one_batch size ranges.","solutions":["Move the capture into vLLM's official capture phase (let CUDAGraphWrapper capture it via compile_sizes/cudagraph_capture_sizes) instead of capturing manually.","If you must capture manually, bracket it with set_cudagraph_capturing_enabled(True) and restore to False afterwards — only if you own the lifecycle.","Disable cudagraph for the offending component (e.g. cudagraph_mode='') as a workaround."],"exampleFix":"# before\nclass MyOp:\n    def forward(self, x):\n        if self._graph is None:\n            g = torch.cuda.CUDAGraph()  # RuntimeError: CUDA graph capturing detected...\n# after\nclass MyOp:\n    def forward(self, x):\n        return self._eager(x)  # let vLLM capture via its cudagraph wrapper","handlingStrategy":"validation","validationCode":"from vllm.compilation.monitor import cudagraph_capturing_enabled\n\ndef safe_to_capture() -> bool:\n    return cudagraph_capturing_enabled\n# assert safe_to_capture() before any manual torch.cuda.CUDAGraph use","typeGuard":null,"tryCatchPattern":"from vllm.compilation.monitor import validate_cudagraph_capturing_enabled\ntry:\n    validate_cudagraph_capturing_enabled()\n    with torch.cuda.graph(g):\n        ...\nexcept RuntimeError as e:\n    if 'inappropriate time' in str(e):\n        fall_back_to_eager()\n    else:\n        raise","preventionTips":["Never capture cudagraphs inside forward paths","Let CUDAGraphWrapper own all capture timing","Test custom kernels with cudagraph_mode='NONE' first, then FULL"],"tags":["cudagraph","runtime-monitor","cuda","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}