sgl-project/sglang · error · RuntimeError

SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.

Error message

SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.0, but MLX is not installed; reinstall with the srt_mps extra

What it means

Raised by SGLang's MLX runtime validator when SGLANG_USE_MLX is enabled but the `mlx` Python package cannot be imported. The MLX hardware backend (Apple Silicon MPS path) requires both a stable Torch 2.13.x and MLX >= 0.32.0, so a missing MLX install fails fast with an actionable message pointing to the `srt_mps` extra.

Source

Thrown at python/sglang/srt/hardware_backend/mlx/runtime.py:35

    except InvalidVersion:
        return False
    return not version.is_prerelease and (version.major, version.minor) == series


def _is_stable_at_least(raw_version: object, minimum: Version) -> bool:
    try:
        version = Version(str(raw_version))
    except InvalidVersion:
        return False
    return not version.is_prerelease and version >= minimum


@lru_cache(maxsize=1)
def _validate_runtime() -> None:
    try:
        import mlx.core as mx
    except ImportError:
        raise RuntimeError(
            "SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.0, "
            "but MLX is not installed; reinstall with "
            "the srt_mps extra"
        ) from None
    mlx_version = getattr(mx, "__version__", None)
    torch_version = getattr(torch, "__version__", None)
    if not _is_stable_series(
        torch_version, _SUPPORTED_TORCH_SERIES
    ) or not _is_stable_at_least(mlx_version, _MIN_MLX_VERSION):
        raise RuntimeError(
            "SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.0; "
            "found "
            f"Torch {torch_version or 'unknown'} + MLX {mlx_version or 'unknown'}; "
            "reinstall with the srt_mps extra"
        )

    mps_backend = getattr(torch.backends, "mps", None)
    is_mps_available = getattr(mps_backend, "is_available", None)

View on GitHub (pinned to 0132848349)

Solutions

1. Install the MLX extra: pip install -e ".[srt_mps]" (or pip install mlx>=0.32.0) in the same environment SGLang runs in
2. Verify with `python -c "import mlx.core as mx; print(mx.__version__)"` that MLX is importable and >= 0.32.0
3. Confirm you are on macOS Apple Silicon; MLX is unavailable elsewhere
4. Unset SGLANG_USE_MLX if you did not intend to use the MPS backend

Example fix

# before
export SGLANG_USE_MLX=1
python -m sglang.launch_server ...
# after
pip install -e ".[srt_mps]"
python -c "import mlx.core as mx; print(mx.__version__)"  # >= 0.32.0
export SGLANG_USE_MLX=1
python -m sglang.launch_server ...

import importlib.util, os
if os.environ.get("SGLANG_USE_MLX"):
    if importlib.util.find_spec("mlx") is None:
        raise SystemExit("Install the srt_mps extra before enabling SGLANG_USE_MLX")

try:
    from sglang.srt.hardware_backend.mlx.runtime import use_mlx
    use_mlx()
except RuntimeError as e:
    if "srt_mps extra" in str(e):
        logger.warning("MLX backend unavailable: %s", e)
        # fall back to default backend
    else:
        raise

When it happens

Trigger: Setting SGLANG_USE_MLX=1 (which calls use_mlx() -> _validate_runtime()) on a machine where `import mlx.core` raises ImportError — e.g. MLX was never installed, installed in a different venv, or installed via an incompatible wheel/platform (Linux x86 instead of macOS Apple Silicon).

Common situations: Enabling the MLX backend without installing the srt_mps extra; wrong conda/venv activated; trying MLX on non-Apple-Silicon hardware; CI runners without the MLX dependency.

Related errors

• SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.
• SGLANG_USE_MLX requires an available PyTorch MPS device
• SGLANG_USE_MLX requires an available MLX Metal device
• MlxTensorView requires a Torch MPS tensor, got {owner.device
• borrow_torch_tensors requires MPS tensors, got {devices}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f3330bcd52e2414b. Report an issue: GitHub.