sgl-project/sglang · error · RuntimeError

SGLANG_USE_MLX requires an available PyTorch MPS device

Error message

SGLANG_USE_MLX requires an available PyTorch MPS device

What it means

The MLX validator requires that PyTorch's MPS (Metal Performance Shaders) backend is present and torch.backends.mps.is_available() returns True after version checks pass. This fires when Torch has no MPS backend attribute, or MPS reports unavailable (no Apple GPU, macOS too old, or a CUDA/Linux build of Torch).

Source

Thrown at python/sglang/srt/hardware_backend/mlx/runtime.py:55

            "but MLX is not installed; reinstall with "
            "the srt_mps extra"
        ) from None
    mlx_version = getattr(mx, "__version__", None)
    torch_version = getattr(torch, "__version__", None)
    if not _is_stable_series(
        torch_version, _SUPPORTED_TORCH_SERIES
    ) or not _is_stable_at_least(mlx_version, _MIN_MLX_VERSION):
        raise RuntimeError(
            "SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.0; "
            "found "
            f"Torch {torch_version or 'unknown'} + MLX {mlx_version or 'unknown'}; "
            "reinstall with the srt_mps extra"
        )

    mps_backend = getattr(torch.backends, "mps", None)
    is_mps_available = getattr(mps_backend, "is_available", None)
    if not callable(is_mps_available) or not is_mps_available():
        raise RuntimeError("SGLANG_USE_MLX requires an available PyTorch MPS device")

    metal = getattr(mx, "metal", None)
    is_available = getattr(metal, "is_available", None)
    if not callable(is_available) or not is_available():
        raise RuntimeError("SGLANG_USE_MLX requires an available MLX Metal device")


@lru_cache(maxsize=1)
def use_mlx() -> bool:
    """Return whether the validated MLX backend was explicitly enabled."""
    enabled = bool(envs.SGLANG_USE_MLX.get())
    if enabled:
        _validate_runtime()
    return enabled

View on GitHub (pinned to 0132848349)

Solutions

1. Install the macOS/MPS build of torch 2.13.x (e.g. pip install torch==2.13.* --force-reinstall, ensuring the arm64 wheel)
2. Verify torch.backends.mps.is_available() returns True in the target environment
3. Run on Apple Silicon hardware with a supported macOS version
4. Unset SGLANG_USE_MLX on non-Apple machines

Example fix

# before (CUDA/Linux wheel)
pip install torch==2.13.0+cu126
# after (macOS arm64 wheel)
pip install torch==2.13.0
python -c "import torch; assert torch.backends.mps.is_available()"

def mps_available() -> bool:
    import torch
    fn = getattr(getattr(torch.backends, "mps", None), "is_available", None)
    return callable(fn) and fn()

try:
    use_mlx()
except RuntimeError as e:
    if "MPS device" in str(e):
        os.environ.pop("SGLANG_USE_MLX", None)  # graceful fallback

When it happens

Trigger: SGLANG_USE_MLX=1 with a CPU/CUDA build of torch (no torch.backends.mps), on a Mac without a supported GPU, on macOS version below MPS requirements, or MPS build available but is_available() returning False.

Common situations: Using the default pip torch wheel (CUDA build) on a Mac; running in a Linux container with SGLANG_USE_MLX set; macOS < 12.3 where MPS is unsupported.

Related errors

• SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.
• SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.
• SGLANG_USE_MLX requires an available MLX Metal device
• MlxTensorView requires a Torch MPS tensor, got {owner.device
• MiniMax-H3 on MPS requires synchronous layerwise offload for

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d707e25309e5dbae. Report an issue: GitHub.