sgl-project/sglang · error · RuntimeError

Packed CUDA VMM features must be reconstructed before releas

Error message

Packed CUDA VMM features must be reconstructed before release

What it means

This is a deliberately unconditional raise: packed multi-feature CUDA VMM containers do not support per-item acknowledgement. Because several features share one packed pool chunk, releasing on behalf of one item is ambiguous, so acknowledge_consumption is fail-closed — the packed buffer must be reconstructed (which copies data out) before any release.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_transport_utils.py:902

        owner: _CudaVmmPackedTransportOwner,
        layout: _CudaVmmPackedTensorLayout,
    ) -> None:
        super().__init__(
            fabric_handle=owner.fabric_handle,
            posix_socket_path=owner.posix_socket_path,
            allocation_size=owner.allocation_size,
            data_offset=owner.data_offset + layout.relative_offset,
            data_nbytes=layout.data_nbytes,
            control_offset=owner.control_offset,
            consumer_count=owner.consumer_count,
            shape=layout.shape,
            dtype=layout.dtype,
        )
        self._packed_owner = owner
        self._packed_relative_offset = layout.relative_offset

    def acknowledge_consumption(self, consumer_count: int | None = None) -> None:
        raise RuntimeError(
            "Packed CUDA VMM features must be reconstructed before release"
        )

    def reconstruct_on_target_device(
        self, rebuild_device_idx, consumer_count: int | None = None
    ):
        rebuild_device = torch.device(f"cuda:{rebuild_device_idx}")
        if (
            isinstance(self.reconstruct_tensor, torch.Tensor)
            and self.reconstruct_tensor.device == rebuild_device
        ):
            return self.reconstruct_tensor
        if self._consumer_acknowledged:
            raise RuntimeError("CUDA VMM tensor has already released its pool slice")

        packed_buffer = self._packed_owner.reconstruct_on_target_device(
            rebuild_device_idx, consumer_count=consumer_count
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Call reconstruct_on_target_device on the packed object first; acknowledgement then flows through the packed owner
2. Branch consumer logic on whether the item is packed before acknowledging
3. Refactor to acknowledge the owning packed buffer once, not each member

Example fix

// before
proxy.acknowledge_consumption()

// after
buf = proxy.reconstruct_on_target_device(device_idx)
# slice out this feature's tensor via its relative offset, then release via owner

def is_packed(item) -> bool:
    return getattr(item, "_packed_owner", None) is not None

When it happens

Trigger: Calling acknowledge_consumption() on a packed features object produced by wrap_items; treating a packed container like a single-feature proxy.

Common situations: Generic consumer code that acknowledges every item uniformly; migrating from per-feature transport to packed transport without updating the release path.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• CUDA VMM POSIX FD broker failed
• CUDA VMM POSIX FD broker returned no file descriptor
• memory_size must be positive
• consumer_count must be positive
• recycle_interval must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fe7c3e8eccbf88b1. Report an issue: GitHub.