{"record":{"id":"0ff9d1bc61de98ac","repo":"sgl-project/sglang","slug":"the-package-amd-quark-is-required-to-use-mx-fp4","errorCode":null,"errorMessage":"The package `amd-quark` is required to use MX-FP4 models. Please install it with `pip install amd-quark`.","messagePattern":"The package `amd-quark` is required to use MX-FP4 models\\. Please install it with `pip install amd-quark`\\.","errorType":"error_code","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/mxfp4.py","lineNumber":277,"sourceCode":"    return quant_tensor, InFlexData(), scale\n\n\ndef _dequant_mxfp4_fake(\n    x: torch.Tensor, scale: torch.Tensor, float_dtype: torch.dtype\n) -> torch.Tensor:\n    return torch.empty(\n        (*x.shape[:-1], x.shape[-1] * 2), dtype=float_dtype, device=x.device\n    )\n\n\n@register_custom_op(fake_impl=_dequant_mxfp4_fake)\ndef dequant_mxfp4(\n    x: torch.Tensor, scale: torch.Tensor, float_dtype: torch.dtype\n) -> torch.Tensor:\n    try:\n        from quark.torch.kernel import mx\n    except ImportError as err:\n        raise ImportError(\n            \"The package `amd-quark` is required to use \"\n            \"MX-FP4 models. Please install it with `pip install \"\n            \"amd-quark`.\"\n        ) from err\n\n    return mx.dq_mxfp4(x, scale, float_dtype)\n\n\n@register_custom_op(out_shape=\"x\")\ndef quant_dequant_mxfp4(\n    x: torch.Tensor, scale_calculation_mode: str = \"even\"\n) -> torch.Tensor:\n    try:\n        from quark.torch.kernel import mx\n    except ImportError as err:\n        raise ImportError(\n            \"The package `amd-quark` is required to use \"\n            \"MX-FP4 models. Please install it with `pip install \"","sourceCodeStart":259,"sourceCodeEnd":295,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/mxfp4.py#L259-L295","documentation":"dequant_mxfp4 needs AMD's quark library kernel (mx.dq_mxfp4) to dequantize MX-FP4 tensors back to floating point. If `amd-quark` is not installed, the import of quark.torch.kernel.mx fails and the error is raised with the install command chained from the original ImportError.","triggerScenarios":"Calling mxfp4.dequant_mxfp4(x, scale, float_dtype) (directly or via a model whose checkpoint stores MX-FP4 weights that must be dequantized on load) in a Python environment without the amd-quark package — e.g. on an NVIDIA machine or a venv missing the wheel.","commonSituations":"Loading an MX-FP4 quantized model on a box where amd-quark was never installed; environments where the ROCm/AMD extras were skipped; CI images built for CUDA that still reference MX-FP4 checkpoints.","solutions":["pip install amd-quark in the serving environment","If on NVIDIA hardware, prefer an SGLang path that keeps MX-FP4 in kernel-native format (no dequant fallback) or use a checkpoint format your stack supports","Pin a working amd-quark version in requirements if a release regressed the quark.torch.kernel.mx import"],"exampleFix":"# before\ny = dequant_mxfp4(x, scale, torch.bfloat16)  # ImportError\n# after\n# pip install amd-quark\ny = dequant_mxfp4(x, scale, torch.bfloat16)","handlingStrategy":"retry","validationCode":"try:\n    from quark.torch.kernel import mx  # noqa: F401\n    HAS_QUARK = True\nexcept ImportError:\n    HAS_QUARK = False\n\nif not HAS_QUARK:\n    raise SystemExit(\"pip install amd-quark before dequantizing MX-FP4 weights\")\ny = dequant_mxfp4(x, scale, torch.bfloat16)","typeGuard":"import importlib.util\n\ndef has_amd_quark() -> bool:\n    return importlib.util.find_spec(\"quark\") is not None","tryCatchPattern":"try:\n    y = dequant_mxfp4(x, scale, dt)\nexcept ImportError as e:\n    if \"amd-quark\" in str(e):\n        subprocess.check_call([sys.executable, \"-m\", \"pip\", \"install\", \"amd-quark\"])\n        y = dequant_mxfp4(x, scale, dt)  # retry once after install\n    else:\n        raise","preventionTips":["Include amd-quark in the environment spec for any MX-FP4 workload","Gate MX-FP4 loading paths behind an importlib.util.find_spec('quark') check at startup"],"tags":["mxfp4","amd","quark","missing-dependency","quantization"],"backgroundTag":"missing-python-package","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}