{"record":{"id":"5b6e8e17530801ff","repo":"sgl-project/sglang","slug":"the-mlx-tensor-bridge-supports-cpu-and-mps-tensors","errorCode":null,"errorMessage":"The MLX tensor bridge supports CPU and MPS tensors, got {tensor.device}","messagePattern":"The MLX tensor bridge supports CPU and MPS tensors, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/tensor_bridge.py","lineNumber":96,"sourceCode":"            torch.mps.synchronize()\n        return mx.asarray(tensor, copy=copy)\n    if tensor.device.type == \"cpu\":\n        # CPU tensors always get MLX-owned storage.  In particular, do not\n        # expose a NumPy/memoryview alias whose lifetime is controlled by the\n        # caller.\n        if tensor.dtype == torch.complex128:\n            raise ValueError(\n                \"MLX 0.32 does not support complex128; convert the Torch tensor \"\n                \"to complex64 explicitly\"\n            )\n        # MLX 0.32 does not support float64 on its default Metal stream.  Keep\n        # the dtype by constructing this uncommon CPU value on the CPU stream\n        # instead of silently downcasting it to float32.\n        if tensor.dtype == torch.float64:\n            with mx.stream(mx.cpu):\n                return mx.array(tensor, dtype=mx.float64)\n        return mx.array(tensor)\n    raise ValueError(\n        f\"The MLX tensor bridge supports CPU and MPS tensors, got {tensor.device}\"\n    )\n\n\nclass MlxTensorView:\n    \"\"\"A lifetime-bound, zero-copy MLX view of a Torch MPS tensor.\n\n    The view deliberately retains a detached Torch tensor *and* the imported\n    MLX array.  Holding only the array is insufficient: a later parameter\n    replacement or garbage collection could invalidate the borrowed storage\n    while MLX still has a lazy graph referring to it. This class is intended\n    for immutable inference weights; construct a new view after replacing the\n    source storage.\n    \"\"\"\n\n    __slots__ = (\"torch_tensor\", \"array\")\n\n    def __init__(self, tensor: torch.Tensor, *, synchronize: bool = True):","sourceCodeStart":78,"sourceCodeEnd":114,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/tensor_bridge.py#L78-L114","documentation":"_torch_to_mlx only accepts torch tensors on CPU or MPS; anything else (cuda, xpu, meta, etc.) falls through to this ValueError naming the actual device. It is the general device guard for the bridge.","triggerScenarios":"Passing a CUDA or other-device tensor to torch_to_mlx, mlx_call, mlx_call_multi, or MlxTensorView.","commonSituations":"Code written for MPS that gets run on a CUDA box, or tensors moved to GPU earlier in the pipeline before an MLX step.","solutions":["Move the tensor to CPU or MPS first: t = t.to('mps') / t.cpu()","Keep MLX bridge usage confined to macOS Metal pipelines"],"exampleFix":"# before\nmx_t = torch_to_mlx(cuda_tensor)\n# after\nmx_t = torch_to_mlx(cuda_tensor.cpu())  # or .to('mps') on macOS","handlingStrategy":"type-guard","validationCode":"assert tensor.device.type in {\"cpu\", \"mps\"}, f\"move {tensor.device} tensor to cpu/mps first\"","typeGuard":"def bridgeable(t: torch.Tensor) -> bool:\n    return t.device.type in {\"cpu\", \"mps\"}","tryCatchPattern":null,"preventionTips":["Move tensors to the target device at pipeline boundaries","Parameterize device and assert it on Apple-Silicon-only paths"],"tags":["sglang","mlx","device","torch"],"backgroundTag":"unsupported-device","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}