sgl-project/sglang · error · ValueError

The MLX tensor bridge supports CPU and MPS targets, got {tar

Error message

The MLX tensor bridge supports CPU and MPS targets, got {target_device}

What it means

mlx_call resolves the export target device (defaulting to the current torch device) and only supports cpu or mps targets; cuda/meta/etc. raise with the resolved device before any work or borrowing happens.

Source

Thrown at python/sglang/srt/utils/tensor_bridge.py:213

def mlx_call(
    operation: Callable[..., mx.array],
    *tensors: torch.Tensor | MlxTensorView,
    device: torch.device | Literal["mps", "cpu"] | None = None,
) -> torch.Tensor:
    """Run one MLX operation with zero-copy Torch MPS input imports.

    The imported MLX arrays remain strongly referenced until
    :func:`mlx_to_torch` evaluates and exports ``operation``'s result.  Keep
    the operation inside this call; returning a lazy MLX result for later use
    or stashing a borrowed input through a callback side effect would escape
    the borrow scope.  The caller must also serialize any overlapping MPS work
    outside this function, including use or mutation of source and returned
    tensors.  The operation may allocate its own output normally.
    """
    mx = _mlx_core()
    target_device = _get_torch_device() if device is None else torch.device(device)
    if target_device.type not in {"cpu", "mps"}:
        raise ValueError(
            f"The MLX tensor bridge supports CPU and MPS targets, got {target_device}"
        )
    detached = tuple(
        tensor.detach() for tensor in tensors if isinstance(tensor, torch.Tensor)
    )
    if any(tensor.device.type == "mps" for tensor in detached) or any(
        isinstance(tensor, MlxTensorView) for tensor in tensors
    ):
        torch.mps.synchronize()
    borrowed: tuple[Any, ...] = tuple(
        (
            tensor.array
            if isinstance(tensor, MlxTensorView)
            else _torch_to_mlx(tensor.detach(), copy=False, synchronize=False)
        )
        for tensor in tensors
    )
    # MLX does not support float64 on the Metal stream.  Keep an explicitly

View on GitHub (pinned to 0132848349)

Solutions

1. Use device='cpu' or device='mps' (or omit for auto-detect)
2. Restructure so MLX results are converted to CPU then moved with torch semantics if a GPU is ultimately needed

Example fix

# before
out = mlx_call(fn, tensors, device='cuda')
# after
out = mlx_call(fn, tensors, device='cpu').to('cuda', non_blocking=True)

dev = device or "cpu"
assert dev in {"cpu", "mps"}, f"unsupported target: {dev}"

When it happens

Trigger: Calling mlx_call(fn, tensors, device='cuda') or running with a default torch device that resolves to something other than cpu/mps.

Common situations: Passing a device string copied from CUDA code, or a default-device helper returning 'mps:0' variants on non-Mac hardware.

Related errors

• The MLX tensor bridge supports CPU and MPS tensors, got {ten
• MlxTensorView requires a Torch MPS tensor, got {owner.device
• borrow_torch_tensors requires MPS tensors, got {devices}
• SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.
• SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b566954a3b160507. Report an issue: GitHub.