sgl-project/sglang · error · RuntimeError

SGLANG_USE_MLX requires an available MLX Metal device

Error message

SGLANG_USE_MLX requires an available MLX Metal device

What it means

Final device check in the MLX validator: after Torch MPS passes, mlx.core.metal.is_available() must also report an available Metal device. This fires when the MLX package imports but cannot see a usable Metal GPU — typical on non-Apple platforms, VMs without GPU passthrough, or broken/older MLX builds.

Source

Thrown at python/sglang/srt/hardware_backend/mlx/runtime.py:60

    if not _is_stable_series(
        torch_version, _SUPPORTED_TORCH_SERIES
    ) or not _is_stable_at_least(mlx_version, _MIN_MLX_VERSION):
        raise RuntimeError(
            "SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.0; "
            "found "
            f"Torch {torch_version or 'unknown'} + MLX {mlx_version or 'unknown'}; "
            "reinstall with the srt_mps extra"
        )

    mps_backend = getattr(torch.backends, "mps", None)
    is_mps_available = getattr(mps_backend, "is_available", None)
    if not callable(is_mps_available) or not is_mps_available():
        raise RuntimeError("SGLANG_USE_MLX requires an available PyTorch MPS device")

    metal = getattr(mx, "metal", None)
    is_available = getattr(metal, "is_available", None)
    if not callable(is_available) or not is_available():
        raise RuntimeError("SGLANG_USE_MLX requires an available MLX Metal device")


@lru_cache(maxsize=1)
def use_mlx() -> bool:
    """Return whether the validated MLX backend was explicitly enabled."""
    enabled = bool(envs.SGLANG_USE_MLX.get())
    if enabled:
        _validate_runtime()
    return enabled

View on GitHub (pinned to 0132848349)

Solutions

1. Run on real Apple Silicon hardware with a Metal-capable GPU
2. Upgrade MLX to >= 0.32.0 via pip install -U 'mlx>=0.32.0' so the metal module exists
3. Verify with python -c "import mlx.core as mx; print(mx.metal.is_available())"
4. Unset SGLANG_USE_MLX in CI/headless environments

Example fix

# before
export SGLANG_USE_MLX=1  # in headless CI
# after
python -c "import mlx.core as mx; assert mx.metal.is_available()" || unset SGLANG_USE_MLX

def mlx_metal_available() -> bool:
    import mlx.core as mx
    fn = getattr(getattr(mx, "metal", None), "is_available", None)
    return callable(fn) and fn()

try:
    use_mlx()
except RuntimeError as e:
    if "MLX Metal device" in str(e):
        logger.error("No Metal GPU for MLX; disabling MLX backend")
        os.environ.pop("SGLANG_USE_MLX", None)

When it happens

Trigger: SGLANG_USE_MLX=1 where mlx.core.metal is missing or metal.is_available() is not callable/returns False — e.g. MLX CPU-only build, macOS VM without Metal, headless CI macOS runner without GPU, or a stale MLX older than the Metal API.

Common situations: Running in GitHub Actions macOS runners (no GPU), VMs, or after installing an unofficial mlx fork; also when mlx is present but torch passed via MPS while MLX was built without Metal support.

Related errors

• sgl_kernel.metal is importable, but the native Metal extensi
• SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.
• SGLANG_USE_MLX requires an available PyTorch MPS device
• q/k/v dtypes must match
• setup_metal.py only supports macOS (Apple Silicon).

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a56a3f4c5e0e0339. Report an issue: GitHub.