{"record":{"id":"b566954a3b160507","repo":"sgl-project/sglang","slug":"the-mlx-tensor-bridge-supports-cpu-and-mps-targets","errorCode":null,"errorMessage":"The MLX tensor bridge supports CPU and MPS targets, got {target_device}","messagePattern":"The MLX tensor bridge supports CPU and MPS targets, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/tensor_bridge.py","lineNumber":213,"sourceCode":"def mlx_call(\n    operation: Callable[..., mx.array],\n    *tensors: torch.Tensor | MlxTensorView,\n    device: torch.device | Literal[\"mps\", \"cpu\"] | None = None,\n) -> torch.Tensor:\n    \"\"\"Run one MLX operation with zero-copy Torch MPS input imports.\n\n    The imported MLX arrays remain strongly referenced until\n    :func:`mlx_to_torch` evaluates and exports ``operation``'s result.  Keep\n    the operation inside this call; returning a lazy MLX result for later use\n    or stashing a borrowed input through a callback side effect would escape\n    the borrow scope.  The caller must also serialize any overlapping MPS work\n    outside this function, including use or mutation of source and returned\n    tensors.  The operation may allocate its own output normally.\n    \"\"\"\n    mx = _mlx_core()\n    target_device = _get_torch_device() if device is None else torch.device(device)\n    if target_device.type not in {\"cpu\", \"mps\"}:\n        raise ValueError(\n            f\"The MLX tensor bridge supports CPU and MPS targets, got {target_device}\"\n        )\n    detached = tuple(\n        tensor.detach() for tensor in tensors if isinstance(tensor, torch.Tensor)\n    )\n    if any(tensor.device.type == \"mps\" for tensor in detached) or any(\n        isinstance(tensor, MlxTensorView) for tensor in tensors\n    ):\n        torch.mps.synchronize()\n    borrowed: tuple[Any, ...] = tuple(\n        (\n            tensor.array\n            if isinstance(tensor, MlxTensorView)\n            else _torch_to_mlx(tensor.detach(), copy=False, synchronize=False)\n        )\n        for tensor in tensors\n    )\n    # MLX does not support float64 on the Metal stream.  Keep an explicitly","sourceCodeStart":195,"sourceCodeEnd":231,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/tensor_bridge.py#L195-L231","documentation":"mlx_call resolves the export target device (defaulting to the current torch device) and only supports cpu or mps targets; cuda/meta/etc. raise with the resolved device before any work or borrowing happens.","triggerScenarios":"Calling mlx_call(fn, tensors, device='cuda') or running with a default torch device that resolves to something other than cpu/mps.","commonSituations":"Passing a device string copied from CUDA code, or a default-device helper returning 'mps:0' variants on non-Mac hardware.","solutions":["Use device='cpu' or device='mps' (or omit for auto-detect)","Restructure so MLX results are converted to CPU then moved with torch semantics if a GPU is ultimately needed"],"exampleFix":"# before\nout = mlx_call(fn, tensors, device='cuda')\n# after\nout = mlx_call(fn, tensors, device='cpu').to('cuda', non_blocking=True)","handlingStrategy":"validation","validationCode":"dev = device or \"cpu\"\nassert dev in {\"cpu\", \"mps\"}, f\"unsupported target: {dev}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Do not reuse CUDA device strings in MLX bridge calls","Centralize device selection in one helper for the Apple-Silicon path"],"tags":["sglang","mlx","device","target"],"backgroundTag":"unsupported-device","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}