{"record":{"id":"51eb1be402967412","repo":"sgl-project/sglang","slug":"cuda-vmm-tensor-has-already-released-its-pool-slic","errorCode":null,"errorMessage":"CUDA VMM tensor has already released its pool slice","messagePattern":"CUDA VMM tensor has already released its pool slice","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/cuda_vmm_transport_utils.py","lineNumber":815,"sourceCode":"\n    def acknowledge_consumption(self, consumer_count: int | None = None) -> None:\n        consumer_count = self._resolve_consumer_count(consumer_count)\n        device_index = torch.cuda.current_device()\n        with torch.cuda.device(device_index):\n            self._acknowledge_consumption(device_index, consumer_count)\n\n    def reconstruct_on_target_device(\n        self, rebuild_device_idx, consumer_count: int | None = None\n    ):\n        consumer_count = self._resolve_consumer_count(consumer_count)\n        rebuild_device = torch.device(f\"cuda:{rebuild_device_idx}\")\n        if (\n            isinstance(self.reconstruct_tensor, torch.Tensor)\n            and self.reconstruct_tensor.device == rebuild_device\n        ):\n            return self.reconstruct_tensor\n        if self._consumer_acknowledged:\n            raise RuntimeError(\"CUDA VMM tensor has already released its pool slice\")\n\n        pool = self._pool(rebuild_device_idx)\n        try:\n            with torch.cuda.device(rebuild_device):\n                source = pool.memory[\n                    self.data_offset : self.data_offset + self.data_nbytes\n                ]\n                reconstructed = torch.empty(\n                    self.shape, dtype=self.dtype, device=rebuild_device\n                ).contiguous()\n                reconstructed.reshape(-1).view(torch.uint8).copy_(\n                    source, non_blocking=True\n                )\n                self._acknowledge_consumption(rebuild_device_idx, consumer_count)\n        except BaseException as error:\n            try:\n                with torch.cuda.device(rebuild_device):\n                    self._acknowledge_consumption(rebuild_device_idx, consumer_count)","sourceCodeStart":797,"sourceCodeEnd":833,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/cuda_vmm_transport_utils.py#L797-L833","documentation":"Raised when reconstruct_on_target_device is called after the tensor already acknowledged consumption, which releases its slice back to the shared VMM pool. Once released, the underlying memory may be reused by other producers, so reconstruction is refused to prevent reading aliased data.","triggerScenarios":"Calling reconstruct_on_target_device() after acknowledge_consumption() on the same proxy; a consumer acknowledging early then lazily trying to materialize the tensor.","commonSituations":"Eager-release optimizations that acknowledge before the tensor is actually used; reorderings where acknowledgement happens in a prefetch path but reconstruction happens later.","solutions":["Reconstruct the tensor first, then acknowledge (call reconstruct_on_target_device before acknowledge_consumption)","Cache the reconstructed tensor — the method returns the cached tensor if device matches, so keep and reuse it","Remove duplicate reconstruction attempts after release"],"exampleFix":"// before\nproxy.acknowledge_consumption()\nt = proxy.reconstruct_on_target_device(idx)\n\n// after\nt = proxy.reconstruct_on_target_device(idx)\nproxy.acknowledge_consumption()","handlingStrategy":"validation","validationCode":"if proxy._consumer_acknowledged:\n    return cached  # do not reconstruct after release","typeGuard":"def can_reconstruct(proxy) -> bool:\n    return not proxy._consumer_acknowledged","tryCatchPattern":null,"preventionTips":["Always reconstruct before acknowledge; cache the returned tensor","Treat acknowledge as the terminal operation on a proxy"],"tags":["cuda","vmm","use-after-free","lifecycle"],"backgroundTag":"use-after-release","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}