{"record":{"id":"fe7c3e8eccbf88b1","repo":"sgl-project/sglang","slug":"packed-cuda-vmm-features-must-be-reconstructed-bef","errorCode":null,"errorMessage":"Packed CUDA VMM features must be reconstructed before release","messagePattern":"Packed CUDA VMM features must be reconstructed before release","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/cuda_vmm_transport_utils.py","lineNumber":902,"sourceCode":"        owner: _CudaVmmPackedTransportOwner,\n        layout: _CudaVmmPackedTensorLayout,\n    ) -> None:\n        super().__init__(\n            fabric_handle=owner.fabric_handle,\n            posix_socket_path=owner.posix_socket_path,\n            allocation_size=owner.allocation_size,\n            data_offset=owner.data_offset + layout.relative_offset,\n            data_nbytes=layout.data_nbytes,\n            control_offset=owner.control_offset,\n            consumer_count=owner.consumer_count,\n            shape=layout.shape,\n            dtype=layout.dtype,\n        )\n        self._packed_owner = owner\n        self._packed_relative_offset = layout.relative_offset\n\n    def acknowledge_consumption(self, consumer_count: int | None = None) -> None:\n        raise RuntimeError(\n            \"Packed CUDA VMM features must be reconstructed before release\"\n        )\n\n    def reconstruct_on_target_device(\n        self, rebuild_device_idx, consumer_count: int | None = None\n    ):\n        rebuild_device = torch.device(f\"cuda:{rebuild_device_idx}\")\n        if (\n            isinstance(self.reconstruct_tensor, torch.Tensor)\n            and self.reconstruct_tensor.device == rebuild_device\n        ):\n            return self.reconstruct_tensor\n        if self._consumer_acknowledged:\n            raise RuntimeError(\"CUDA VMM tensor has already released its pool slice\")\n\n        packed_buffer = self._packed_owner.reconstruct_on_target_device(\n            rebuild_device_idx, consumer_count=consumer_count\n        )","sourceCodeStart":884,"sourceCodeEnd":920,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/cuda_vmm_transport_utils.py#L884-L920","documentation":"This is a deliberately unconditional raise: packed multi-feature CUDA VMM containers do not support per-item acknowledgement. Because several features share one packed pool chunk, releasing on behalf of one item is ambiguous, so acknowledge_consumption is fail-closed — the packed buffer must be reconstructed (which copies data out) before any release.","triggerScenarios":"Calling acknowledge_consumption() on a packed features object produced by wrap_items; treating a packed container like a single-feature proxy.","commonSituations":"Generic consumer code that acknowledges every item uniformly; migrating from per-feature transport to packed transport without updating the release path.","solutions":["Call reconstruct_on_target_device on the packed object first; acknowledgement then flows through the packed owner","Branch consumer logic on whether the item is packed before acknowledging","Refactor to acknowledge the owning packed buffer once, not each member"],"exampleFix":"// before\nproxy.acknowledge_consumption()\n\n// after\nbuf = proxy.reconstruct_on_target_device(device_idx)\n# slice out this feature's tensor via its relative offset, then release via owner","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"def is_packed(item) -> bool:\n    return getattr(item, \"_packed_owner\", None) is not None","tryCatchPattern":null,"preventionTips":["Branch on packed vs single before acknowledging","Reconstruct packed buffers first; release via the owner"],"tags":["cuda","vmm","packed-tensors","api-misuse"],"backgroundTag":"unsupported-operation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}