sgl-project/sglang · error · ValueError

The MLX tensor bridge supports CPU and MPS tensors, got {ten

Error message

The MLX tensor bridge supports CPU and MPS tensors, got {tensor.device}

What it means

_torch_to_mlx only accepts torch tensors on CPU or MPS; anything else (cuda, xpu, meta, etc.) falls through to this ValueError naming the actual device. It is the general device guard for the bridge.

Source

Thrown at python/sglang/srt/utils/tensor_bridge.py:96

            torch.mps.synchronize()
        return mx.asarray(tensor, copy=copy)
    if tensor.device.type == "cpu":
        # CPU tensors always get MLX-owned storage.  In particular, do not
        # expose a NumPy/memoryview alias whose lifetime is controlled by the
        # caller.
        if tensor.dtype == torch.complex128:
            raise ValueError(
                "MLX 0.32 does not support complex128; convert the Torch tensor "
                "to complex64 explicitly"
            )
        # MLX 0.32 does not support float64 on its default Metal stream.  Keep
        # the dtype by constructing this uncommon CPU value on the CPU stream
        # instead of silently downcasting it to float32.
        if tensor.dtype == torch.float64:
            with mx.stream(mx.cpu):
                return mx.array(tensor, dtype=mx.float64)
        return mx.array(tensor)
    raise ValueError(
        f"The MLX tensor bridge supports CPU and MPS tensors, got {tensor.device}"
    )


class MlxTensorView:
    """A lifetime-bound, zero-copy MLX view of a Torch MPS tensor.

    The view deliberately retains a detached Torch tensor *and* the imported
    MLX array.  Holding only the array is insufficient: a later parameter
    replacement or garbage collection could invalidate the borrowed storage
    while MLX still has a lazy graph referring to it. This class is intended
    for immutable inference weights; construct a new view after replacing the
    source storage.
    """

    __slots__ = ("torch_tensor", "array")

    def __init__(self, tensor: torch.Tensor, *, synchronize: bool = True):

View on GitHub (pinned to 0132848349)

Solutions

1. Move the tensor to CPU or MPS first: t = t.to('mps') / t.cpu()
2. Keep MLX bridge usage confined to macOS Metal pipelines

Example fix

# before
mx_t = torch_to_mlx(cuda_tensor)
# after
mx_t = torch_to_mlx(cuda_tensor.cpu())  # or .to('mps') on macOS

assert tensor.device.type in {"cpu", "mps"}, f"move {tensor.device} tensor to cpu/mps first"

def bridgeable(t: torch.Tensor) -> bool:
    return t.device.type in {"cpu", "mps"}

When it happens

Trigger: Passing a CUDA or other-device tensor to torch_to_mlx, mlx_call, mlx_call_multi, or MlxTensorView.

Common situations: Code written for MPS that gets run on a CUDA box, or tensors moved to GPU earlier in the pipeline before an MLX step.

Related errors

• MlxTensorView requires a Torch MPS tensor, got {owner.device
• SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.
• MLX 0.32 does not support complex128; convert the Torch tens
• borrow_torch_tensors requires MPS tensors, got {devices}
• The MLX tensor bridge supports CPU and MPS targets, got {tar

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5b6e8e17530801ff. Report an issue: GitHub.