{"record":{"id":"30ee6b3782bbbea0","repo":"sgl-project/sglang","slug":"self-quantization-quantization-is-currently-not","errorCode":null,"errorMessage":"{self.quantization} quantization is currently not supported in ROCm.","messagePattern":"(.+?) quantization is currently not supported in ROCm\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/configs/model_config.py","lineNumber":1652,"sourceCode":"                    logger.info_once(\n                        f\"Requantizing from quant_method='{quant_method}' to the requested online quantization='{self.quantization}'. Beware that requantization may incur a loss in accuracy, the requantized model should be re-validated/re-evaluated. More details at https://docs.sglang.io/advanced_features/quantization.html#online-quantization.\"\n                    )\n                else:\n                    raise ValueError(\n                        \"Quantization method specified in the model config \"\n                        f\"({quant_method}) does not match the quantization \"\n                        f\"method specified in the `quantization` argument \"\n                        f\"({self.quantization}).\"\n                    )\n\n        if self.quantization is not None:\n            if self.quantization not in supported_quantization:\n                raise ValueError(\n                    f\"Unknown quantization method: {self.quantization}. Must \"\n                    f\"be one of {supported_quantization}.\"\n                )\n            if is_hip() and self.quantization not in rocm_supported_quantization:\n                raise ValueError(\n                    f\"{self.quantization} quantization is currently not \"\n                    f\"supported in ROCm.\"\n                )\n            if self.quantization not in optimized_quantization_methods:\n                # Don't warn for MXFP4/MXFP8 on SM100 since they have optimized kernels\n                if not (\n                    self.quantization in [\"mxfp4\", \"mxfp8\"] and is_sm100_supported()\n                ):\n                    logger.warning(\n                        \"%s quantization is not fully \"\n                        \"optimized yet. The speed can be slower than \"\n                        \"non-quantized models.\",\n                        self.quantization,\n                    )\n\n    def _verify_dual_chunk_attention_config(self) -> None:\n        if hasattr(self.hf_config, \"dual_chunk_attention_config\"):\n            # Try loading the sparse attention config","sourceCodeStart":1634,"sourceCodeEnd":1670,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/configs/model_config.py#L1634-L1670","documentation":"The requested quantization method is supported on CUDA but not in the rocm_supported_quantization list, and is_hip() is true (AMD GPU). _verify_quantization blocks the launch rather than failing later in a kernel.","triggerScenarios":"Running on ROCm/AMD hardware with --quantization set to a CUDA-only method (e.g. many marlin/fp4 kernels).","commonSituations":"Running containerized sglang on MI2xx/MI3xx GPUs with configs borrowed from NVIDIA recipes, or CI on AMD runners using default quantization args.","solutions":["Switch to a ROCm-supported quantization method listed in rocm_supported_quantization (or drop --quantization to use the native one)","Run on NVIDIA hardware if that specific quantization is required","Check sglang ROCm docs for the current per-method support matrix"],"exampleFix":"# before (on ROCm)\npython -m sglang.launch_server --model MODEL --quantization awq_marlin\n# after\npython -m sglang.launch_server --model MODEL --quantization awq","handlingStrategy":"validation","validationCode":"import torch\nif torch.version.hip:\n    assert requested_quant in ROCM_SUPPORTED, f\"{requested_quant} not available on ROCm\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Gate quantization flags on accelerator vendor in launch scripts","Consult the ROCm support matrix when deploying on AMD"],"tags":["quantization","rocm","amd","hardware-support","sglang"],"backgroundTag":"unsupported-platform-feature","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}