{"record":{"id":"d83eeb9e9d7f9fe4","repo":"sgl-project/sglang","slug":"hip-does-not-support-fused-marlin-moe-currently","errorCode":null,"errorMessage":"HIP does not support fused_marlin_moe currently.","messagePattern":"HIP does not support fused_marlin_moe currently\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"python/sglang/srt/layers/quantization/awq/awq.py","lineNumber":265,"sourceCode":"\n    # num_bits -> type\n    TYPE_MAP = {\n        4: scalar_types.uint4,\n        8: scalar_types.uint8,\n    }\n\n    def __init__(\n        self,\n        weight_bits: int,\n        group_size: int,\n        zero_point: bool,\n        lm_head_quantized: bool,\n        modules_to_not_convert: Optional[list[str]],\n        full_config: dict[str, Any],\n    ) -> None:\n        super().__init__()\n        if _is_hip:\n            warnings.warn(f\"HIP does not support fused_marlin_moe currently.\")\n        self.pack_factor = 32 // weight_bits  # packed into int32\n        self.group_size = group_size\n        self.zero_point = zero_point\n        self.lm_head_quantized = lm_head_quantized\n        self.weight_bits = weight_bits\n        self.modules_to_not_convert = modules_to_not_convert or []\n        self.full_config = full_config\n\n        if self.weight_bits not in self.TYPE_MAP:\n            raise ValueError(\n                f\"Unsupported num_bits = {self.weight_bits}. \"\n                f\"Supported num_bits = {self.TYPE_MAP.keys()}\"\n            )\n\n        self.quant_type = self.TYPE_MAP[self.weight_bits]\n\n        verify_marlin_supported(\n            self.quant_type, group_size=self.group_size, has_zp=self.zero_point","sourceCodeStart":247,"sourceCodeEnd":283,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/awq/awq.py#L247-L283","documentation":"Warning from AWQMarlinConfig.__init__: on HIP (ROCm) GPUs the fused_marlin_moe kernel is not supported. The config still initializes, but MoE layers will not use the fused Marlin path and may fall back to slower execution or fail later.","triggerScenarios":"Loading an AWQ-Marlin quantized MoE model (e.g. Mixtral/Meta-Llama MoE AWQ checkpoints) on an AMD ROCm (HIP) system; _is_hip is true when awq.py is imported on ROCm.","commonSituations":"Running AWQ Marlin MoE models on MI200/MI300 GPUs; porting a CUDA deployment to ROCm without checking kernel coverage.","solutions":["Use a non-Marlin AWQ variant or different quantization format on ROCm","Check sglang release notes for fused_marlin_moe ROCm support and upgrade","Alternatively force a different MoE quant method via quantization config override"],"exampleFix":"# before\n--quantization awq_marlin  # on ROCm with MoE model\n# after\n--quantization awq  # or a ROCm-supported format","handlingStrategy":"validation","validationCode":"from sglang.srt.utils import is_hip\nif is_hip():\n    # avoid fused_marlin_moe AWQ checkpoints\n    choose_non_marlin_checkpoint()","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Maintain a ROCm-compatible model list","Test quantized MoE paths on the target accelerator before deployment"],"tags":["sglang","awq","marlin","rocm","moe","platform-support"],"backgroundTag":"unsupported-platform","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}