{"record":{"id":"d5ab881ab482f778","repo":"vllm-project/vllm","slug":"nested-breakablecudagraphcapture-is-not-supported","errorCode":null,"errorMessage":"Nested BreakableCUDAGraphCapture is not supported.","messagePattern":"Nested BreakableCUDAGraphCapture is not supported\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"vllm/compilation/breakable_cudagraph.py","lineNumber":162,"sourceCode":"        return getattr(cls._tls, \"active\", None)\n\n    @classmethod\n    def is_active(cls) -> bool:\n        return cls.current() is not None\n\n    def __init__(self, pool: Any | None = None) -> None:\n        self.pool = pool\n        self.segments: list[Callable[[], Any]] = []\n        self._num_graphs: int = 0\n        self._num_eager_breaks: int = 0\n        self._current_graph: torch.cuda.CUDAGraph | None = None\n        self._capturing: bool = False\n\n    # --- context manager protocol ----------------------------------------\n\n    def __enter__(self) -> BreakableCUDAGraphCapture:\n        if getattr(BreakableCUDAGraphCapture._tls, \"active\", None) is not None:\n            raise RuntimeError(\"Nested BreakableCUDAGraphCapture is not supported.\")\n        BreakableCUDAGraphCapture._tls.active = self\n        self._begin_segment()\n        return self\n\n    def __exit__(self, exc_type, exc, tb) -> None:\n        try:\n            self._end_segment()\n        finally:\n            BreakableCUDAGraphCapture._tls.active = None\n\n    # --- segment management ----------------------------------------------\n\n    def _begin_segment(self) -> None:\n        assert not self._capturing\n        g = torch.cuda.CUDAGraph()\n        if self.pool is not None:\n            g.capture_begin(pool=self.pool)\n        else:","sourceCodeStart":144,"sourceCodeEnd":180,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/compilation/breakable_cudagraph.py#L144-L180","documentation":"BreakableCUDAGraphCapture is a context manager that records a CUDA graph in multiple segments (graph capture, eager break, graph capture ...). It registers itself in thread-local state (_tls.active) for nested code to detect re-entry; entering it while another instance is already active raises RuntimeError, because nested capture pools/segments are unsupported.","triggerScenarios":"Entering `with BreakableCUDAGraphCapture(...)` inside the body of another live `with BreakableCUDAGraphCapture(...)` block on the same thread; e.g. a model's capture helper that itself uses the context manager being run inside an outer capture.","commonSituations":"Custom model code or a wrapper that unconditionally starts breakable capture, invoked within vLLM's own CUDA-graph capture phase; plugins/hooks that add their own graph capture around quantization or attention modules.","solutions":["Check the guard before entering: `if getattr(BreakableCUDAGraphCapture._tls, 'active', None) is None:` and reuse the active capture instead of nesting","Refactor so only one layer owns capture; inner code should add segments to the active capture rather than open a new one","Disable CUDA graph capture (enforce_eager=True / -O0 cudagraph mode) for the model that cannot avoid nesting"],"exampleFix":"# before\nwith BreakableCUDAGraphCapture(pool):  # inside another capture -> raises\n    run(model)\n# after\nfrom vllm.compilation.breakable_cudagraph import BreakableCUDAGraphCapture\nif getattr(BreakableCUDAGraphCapture._tls, \"active\", None) is None:\n    with BreakableCUDAGraphCapture(pool):\n        run(model)\nelse:\n    run(model)  # already capturing; add to active capture","handlingStrategy":"validation","validationCode":"from vllm.compilation.breakable_cudagraph import BreakableCUDAGraphCapture\nactive = getattr(BreakableCUDAGraphCapture._tls, \"active\", None)\nassert active is None, \"already inside a BreakableCUDAGraphCapture\"","typeGuard":"def can_start_capture() -> bool:\n    from vllm.compilation.breakable_cudagraph import BreakableCUDAGraphCapture\n    return getattr(BreakableCUDAGraphCapture._tls, \"active\", None) is None","tryCatchPattern":null,"preventionTips":["Check the TLS active flag before entering capture in reusable helpers","Keep exactly one layer responsible for CUDA graph capture per thread"],"tags":["cuda-graph","compilation","runtime","nested-context"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}