sgl-project/sglang · error · ValueError

borrow_torch_tensors requires MPS tensors, got {devices}

Error message

borrow_torch_tensors requires MPS tensors, got {devices}

What it means

borrow_torch_tensors borrows multiple tensors as zero-copy MLX views and validates up-front that every detached tensor is on MPS, listing all offending devices in the message. Unlike torch_to_mlx it never copies, hence the strict requirement.

Source

Thrown at python/sglang/srt/utils/tensor_bridge.py:166

@_serialized_bridge
def borrow_torch_tensors(
    *tensors: torch.Tensor, synchronize: bool = True
) -> tuple[MlxTensorView, ...]:
    """Borrow one or more Torch MPS tensors, optionally synchronizing once.

    The returned views own the Torch tensor references for their entire
    lifetime.  No data copy is made.  Set ``synchronize=False`` only when a
    surrounding operation (such as :func:`mlx_call`) performs the producer
    barrier immediately before consuming the views.  This helper is
    intentionally separate from :func:`torch_to_mlx`, whose contract is an
    independent MLX copy.
    """
    detached = tuple(tensor.detach() for tensor in tensors)
    if any(tensor.device.type != "mps" for tensor in detached):
        devices = ", ".join(str(tensor.device) for tensor in detached)
        raise ValueError(f"borrow_torch_tensors requires MPS tensors, got {devices}")
    if synchronize and detached:
        torch.mps.synchronize()
    return tuple(MlxTensorView._from_synchronized(tensor) for tensor in detached)


@_serialized_bridge
def torch_to_mlx(tensor: torch.Tensor) -> mx.array:
    """Convert a PyTorch tensor to an independent MLX array.

    MPS inputs are copied inside the unified Metal device.  Use ``mlx_call``
    when a complete operation needs zero-copy MPS input imports; it owns the
    borrowed MLX arrays for the complete lazy operation.

    Args:
        tensor: PyTorch CPU or MPS tensor.

    Returns:
        MLX array with the same data

View on GitHub (pinned to 0132848349)

Solutions

1. Move all tensors to MPS before borrowing: [t.to('mps') for t in ts]
2. Use torch_to_mlx per-tensor if CPU provenance is acceptable for a copy

Example fix

# before
views = borrow_torch_tensors([a_cpu, b_mps])
# after
views = borrow_torch_tensors([a_cpu.to('mps'), b_mps])

if any(t.device.type != "mps" for t in tensors):
    tensors = [t.to("mps") for t in tensors]
views = borrow_torch_tensors(tensors)

def all_mps(ts) -> bool:
    return all(t.device.type == "mps" for t in ts)

When it happens

Trigger: Calling borrow_torch_tensors([...]) where at least one tensor is on cpu/cuda — the check runs before any synchronization so no side effects occur.

Common situations: Mixed-device batches (some tensors never moved to MPS), or a default-device code path creating tensors on CPU in an otherwise MPS pipeline.

Related errors

• MlxTensorView requires a Torch MPS tensor, got {owner.device
• SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.
• SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.
• The MLX tensor bridge supports CPU and MPS tensors, got {ten
• The MLX tensor bridge supports CPU and MPS targets, got {tar

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f978622eed176d93. Report an issue: GitHub.