sgl-project/sglang · critical · ValueError

Serialized W4A4 checkpoints are not supported on MPS

Error message

Serialized W4A4 checkpoints are not supported on MPS

What it means

KitchenW4A4Config.__init__ hard-fails on MPS because the W4A4 dequant/matmul kernels have no Metal implementation. This check runs before any layers are built, so load fails immediately on Apple Silicon.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py:37

    KitchenInt8Config,
)
from sglang.multimodal_gen.runtime.layers.quantization.kitchen_w4a4 import (
    KitchenW4A4LinearMethod,
)
from sglang.multimodal_gen.runtime.platforms import current_platform

_QUANT_GROUP_SIZE = 64
_SUPPORTED_CONVROT_GROUP_SIZES = (16, 64, 256)
_SUPPORTED_LINEAR_DTYPES = ("int4", "int8")


class KitchenW4A4Config(QuantizationConfig):
    """Dispatch serialized W4A4 linears and their optional INT8 companions."""

    def __init__(self, layer_markers: dict[str, dict[str, Any]]) -> None:
        super().__init__()
        if current_platform.is_mps():
            raise ValueError("Serialized W4A4 checkpoints are not supported on MPS")
        if current_platform.is_cuda():
            capability = current_platform.get_device_capability()
            if (
                capability is not None
                and capability.to_int() < self.get_min_capability()
            ):
                raise ValueError(
                    "Serialized W4A4 checkpoints require CUDA compute capability "
                    f">= {self.get_min_capability() / 10:.1f}; got "
                    f"{capability.to_int() / 10:.1f}"
                )
        self.layer_markers = layer_markers
        self.checkpoint_uses_native_qkv_layout = True
        self.selected: list[str] = []
        int8_markers = {
            prefix: marker
            for prefix, marker in layer_markers.items()
            if marker.get("format") == "int8_tensorwise"

View on GitHub (pinned to 0132848349)

Solutions

1. Run on a CUDA GPU (Linux/NVIDIA) instead
2. Load an unquantized or int8 (MPS-supported) variant of the checkpoint
3. Force CPU execution path if supported by the surrounding runtime

Example fix

// before: run on macOS GPU
/device mps
// after
/device cuda  # or use an fp16/int8 checkpoint

import torch
if torch.backends.mps.is_available() and device == "mps":
    raise SystemExit("W4A4 checkpoints need CUDA; use an int8/fp16 export on MPS")

def w4a4_supported_here() -> bool:
    import torch
    return torch.cuda.is_available()

When it happens

Trigger: Instantiating KitchenW4A4Config on a machine where current_platform.is_mps() is true (torch device type 'mps', Apple Silicon GPU).

Common situations: Developing on an M-series Mac and loading a W4A4-serialized Comfy checkpoint; CI running on macOS runners.

Related errors

• Serialized W4A8 checkpoints are not supported on MPS
• MiniMax-H3 on MPS requires synchronous layerwise offload for
• Nunchaku SVDQuant is only supported on NVIDIA CUDA GPUs (Amp
• SGLANG_USE_MLX requires stable Torch 2.13.x and MLX >= 0.32.
• SGLANG_USE_MLX requires an available PyTorch MPS device

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/04157c10c930db3c. Report an issue: GitHub.