sgl-project/sglang · critical · ImportError

sgl_kernel.metal is importable, but the native Metal extensi

Error message

sgl_kernel.metal is importable, but the native Metal extension or metallib is not available.${reason} Install the Metal kernels with `uv run python/sglang/kernels/aot/setup_metal.py install` from the SGLang repo root in the active environment.

What it means

SGLang's MLX backend optionally uses a custom fused RoPE kernel loaded from a prebuilt Metal extension (`sgl_kernel.metal`). The Python module imports, but the underlying native extension or .metallib artifact is missing (a `_IMPORT_ERROR` is recorded on the module), so the loader raises ImportError instead of returning `metal.rope_pool_fused`. This is an install/environment-completeness error, not a code bug.

Source

Thrown at python/sglang/srt/hardware_backend/mlx/aot.py:28

from sglang.srt.environ import envs

logger = logging.getLogger(__name__)


def _load_metal_rope_pool_fused():
    try:
        from sgl_kernel import metal
    except ImportError as exc:
        raise ImportError(
            "sgl_kernel.metal is not importable. Install sgl-kernel in the "
            "active environment before enabling SGLANG_MLX_USE_CUSTOM_ROPE."
        ) from exc

    import_error = getattr(metal, "_IMPORT_ERROR", None)
    if getattr(metal, "_metal", None) is None or import_error is not None:
        reason = f" Reason: {import_error}." if import_error is not None else ""
        raise ImportError(
            "sgl_kernel.metal is importable, but the native Metal extension "
            f"or metallib is not available.{reason} Install the Metal kernels "
            "with `uv run python/sglang/kernels/aot/setup_metal.py install` "
            "from the SGLang repo root in the active environment."
        ) from import_error
    return metal.rope_pool_fused


@dataclass
class MlxAOTRoPEKernel:
    base: float = 0.0
    config: dict[str, Any] = field(default_factory=dict)
    rope_pool_fused: Optional[Any] = None

    @property
    def enabled(self) -> bool:
        return (
            self.base > 0.0 and bool(self.config) and self.rope_pool_fused is not None

View on GitHub (pinned to 0132848349)

Solutions

1. Run `uv run python/sglang/kernels/aot/setup_metal.py install` from the SGLang repo root in the active environment.
2. Verify the install by checking `import sgl_kernel.metal as m; m._metal is not None and m._IMPORT_ERROR is None`.
3. If you don't need the fused kernel, unset SGLANG_MLX_USE_CUSTOM_ROPE to fall back to the non-AOT path.
4. Rebuild in the correct virtualenv — the metallib is per-environment; rebuilding in one venv does not fix another.

Example fix

# before
export SGLANG_MLX_USE_CUSTOM_ROPE=1
python -c "from sglang.srt.hardware_backend.mlx.aot import _load_metal_rope_pool_fused; _load_metal_rope_pool_fused()"  # ImportError

# after
uv run python/sglang/kernels/aot/setup_metal.py install
python -c "import sgl_kernel.metal as m; assert m._metal is not None"

import sgl_kernel.metal as metal
if getattr(metal, "_metal", None) is None or getattr(metal, "_IMPORT_ERROR", None):
    raise SystemExit("Metal kernels missing; run setup_metal.py install")

When it happens

Trigger: Enabling SGLANG_MLX_USE_CUSTOM_ROPE (or otherwise triggering `_load_metal_rope_pool_fused` via `_build_rope_kernel`) in an environment where `sgl_kernel.metal` imports but `metal._metal` is None or `metal._IMPORT_ERROR` is set — e.g. the package was installed without building/downloading the metallib, or the AOT build step was skipped.

Common situations: Running on Apple Silicon with the MLX backend after a partial install (pip wheel without Metal artifacts), switching uv environments so the metallib built in one venv is missing in another, or after upgrading sgl-kernel without re-running the Metal AOT setup.

Related errors

• SGLANG_USE_MLX requires an available MLX Metal device
• q/k/v dtypes must match
• setup_metal.py only supports macOS (Apple Silicon).
• BatchedDecodeContext requires full_kv_pool_index_by_layer wh
• SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2340bdeeaf02505b. Report an issue: GitHub.