{"record":{"id":"b30aa8e9e7887349","repo":"sgl-project/sglang","slug":"cuda-vmm-multimodal-pool-is-closing","errorCode":null,"errorMessage":"CUDA VMM multimodal pool is closing","messagePattern":"CUDA VMM multimodal pool is closing","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/cuda_vmm_transport_utils.py","lineNumber":348,"sourceCode":"            self.device_index,\n        )\n        self.memory_pool = memory_pool\n\n    @property\n    def control_size(self) -> int:\n        return align_up(self.consumer_count * _CONTROL_WORD_BYTES, _CONTROL_ALIGNMENT)\n\n    def _raise_if_failed(self) -> None:\n        if self._pool_error is not None:\n            raise RuntimeError(\"CUDA VMM multimodal pool failed\") from self._pool_error\n        if self._fd_broker is not None:\n            self._fd_broker.raise_if_failed()\n\n    def _reserve_for_publish(self, required_size: int) -> _CudaVmmMemoryChunk | None:\n        with self._publisher_condition:\n            self._raise_if_failed()\n            if self._closing or self._closed:\n                raise RuntimeError(\"CUDA VMM multimodal pool is closing\")\n            chunk = self._reserve_chunk(required_size)\n            if chunk is not None:\n                self._active_publishers += 1\n            return chunk\n\n    def _finish_publish(self) -> None:\n        with self._publisher_condition:\n            self._active_publishers -= 1\n            if self._active_publishers == 0:\n                self._publisher_condition.notify_all()\n\n    def wrap_tensor(self, tensor: torch.Tensor):\n        self._raise_if_failed()\n        if not tensor.is_contiguous():\n            tensor = tensor.contiguous()\n        data_nbytes = tensor.numel() * tensor.element_size()\n        required_size = align_up(self.control_size + data_nbytes, _CONTROL_ALIGNMENT)\n        source_bytes = tensor.reshape(-1).view(torch.uint8)","sourceCodeStart":330,"sourceCodeEnd":366,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/cuda_vmm_transport_utils.py#L330-L366","documentation":"A publisher called _reserve_for_publish after close() began (self._closing/_closed set). The pool refuses new publish reservations during shutdown to avoid allocating into memory being torn down.","triggerScenarios":"Race between one thread shutting down the pool (server shutdown, scheduler exit) and another thread wrapping tensors for publish — e.g. an in-flight multimodal batch finishing during shutdown.","commonSituations":"Unclean shutdowns with in-flight requests; signal handling that closes the transport while workers still publish; missing drain of publishers before close.","solutions":["Drain/await in-flight publishers before calling close() on the pool","Catch RuntimeError in publish paths during shutdown and treat as a clean cancel","Ensure worker threads observe the shutdown flag before publishing"],"exampleFix":"# before\npool.close()  # while worker still publishing\n# after\nawait drain_publishers()\npool.close()","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    chunk = pool_reserve(publish_fn)\nexcept RuntimeError as e:\n    if \"closing\" in str(e):\n        return  # clean cancel during shutdown","preventionTips":["Drain in-flight publishers before close()","Set shutdown flags workers check before publishing","Catch and swallow 'is closing' during graceful shutdown paths"],"tags":["cuda","vmm","shutdown","race-condition","lifecycle"],"backgroundTag":"use-after-close-race","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}