sgl-project/sglang · error · TypeError

Input 'data' must be a torch.Tensor, but got

Error message

Input 'data' must be a torch.Tensor, but got {type}

What it means

The CUDA IPC consumer-side handle constructor demands both data and info_data be torch.Tensor objects; anything else (numpy arrays, cupy, storage objects) fails this TypeError immediately. IPC handles wrap raw CUDA tensors shared across processes, so no implicit conversion is attempted.

Solutions

1. Wrap buffers in torch.Tensor before constructing the handle: torch.frombuffer / tensor.view(torch.uint8)
2. Ensure CUDA tensors, not CPU numpy — the transport requires device tensors
3. Keep info_data as a uint8 CUDA tensor per the transport contract

Example fix

# before
handle = ConsumerHandle(data=np_array, info_data=info_tensor, ...)
# after
data = torch.frombuffer(np_array.get(), dtype=np.uint8).cuda()
handle = ConsumerHandle(data=data, info_data=info_tensor, ...)

assert isinstance(data, torch.Tensor) and data.is_cuda, 'data must be a CUDA tensor'

import torch
def is_cuda_tensor(x) -> bool:
    return isinstance(x, torch.Tensor) and x.is_cuda

When it happens

Trigger: Constructing the IPC consumer with data as np.ndarray, a UntypedStorage, or a cupy array — even if info_data is a tensor, either failing isinstance triggers the raise.

Common situations: Interoperability layers passing numpy-backed multimodal embeddings; refactors that switched internal buffers from tensors to other array types; a storage handle opened from bytes not yet wrapped via torch.Tensor._from_storage.

Related errors

• consumer_count must be positive
• max_inflight_slices must be positive
• memory_size must be positive
• acknowledgements support one consumer or the complete…
• tokenizer_worker_num must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7930c86556268a7a. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/multimodal/transport/cuda_ipc.py:194

    or device-wide synchronization.
    """

    def __init__(
        self,
        data: torch.Tensor,
        info_data: torch.Tensor,
        pool_ipc_handle,
        pool_byte_offset: int,
        ready_byte_offset: int,
        ack_byte_offset: int,
        generation: int,
        total_consumer_count: int,
        use_pool_handle_cache: bool,
    ):
        if (not isinstance(data, torch.Tensor)) or (
            not isinstance(info_data, torch.Tensor)
        ):
            raise TypeError(
                f"Input 'data' must be a torch.Tensor, but got {type(data)}"
            )

        self._init_stream_ordered_consumer(
            ready_byte_offset=ready_byte_offset,
            ack_byte_offset=ack_byte_offset,
            generation=generation,
            total_consumer_count=total_consumer_count,
            transport_name="CUDA IPC",
        )

        self.proxy_state = {
            "ipc_extra": {
                "pool_handle": pool_ipc_handle,
                "pool_byte_offset": pool_byte_offset,
                "shape": data.shape,
                "dtype": data.dtype,
                "stride": data.stride(),

View on GitHub (pinned to 0132848349)