{"record":{"id":"023bfea786a6e2a4","repo":"sgl-project/sglang","slug":"mixed-precision-layer-group-tail-r-uses-unsuppor","errorCode":null,"errorMessage":"MIXED_PRECISION layer group {tail!r} uses unsupported quant algo {algo!r}; online requantization supports NVFP4 (-> MXFP4) and FP8 (kept as-is) only.","messagePattern":"MIXED_PRECISION layer group (.+?) uses unsupported quant algo (.+?); online requantization supports NVFP4 \\(-> MXFP4\\) and FP8 \\(kept as-is\\) only\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/quark/quark.py","lineNumber":238,"sourceCode":"\n    layer_quant_config: Dict[str, Any] = {}\n    has_nvfp4 = False\n    for tail, algos in tail_algos.items():\n        if len(algos) != 1:\n            raise NotImplementedError(\n                f\"MIXED_PRECISION layer group {tail!r} has inconsistent \"\n                f\"quant algos across layers: {sorted(algos)}. SGLang requires \"\n                \"all layers in a group to share one algo.\"\n            )\n        algo = next(iter(algos))\n        pattern = \"*\" + tail\n        if algo in (\"NVFP4\", \"W4A16_NVFP4\"):\n            layer_quant_config[pattern] = _MXFP4_TARGET_SPEC\n            has_nvfp4 = True\n        elif algo == \"FP8\":\n            layer_quant_config[pattern] = fp8_spec\n        else:\n            raise NotImplementedError(\n                f\"MIXED_PRECISION layer group {tail!r} uses unsupported \"\n                f\"quant algo {algo!r}; online requantization supports NVFP4 \"\n                \"(-> MXFP4) and FP8 (kept as-is) only.\"\n            )\n    return layer_quant_config, has_nvfp4\n\n\ndef _build_excluded_fp8_config(config: Dict[str, Any]) -> Optional[\"Fp8Config\"]:\n    \"\"\"Build a load-as-is `Fp8Config` for the excluded layers of a\n    mixed-precision NVFP4 source, or None if excluded layers are bf16.\n\n    Two producer conventions are handled:\n\n    - FP8-serialized base (``quant_method == \"fp8\"``, e.g.\n      DeepSeek-V4-Pro-NVFP4): the routed experts are NVFP4 (requantized to\n      MXFP4) while attn / shared_experts stay FP8 and are listed in the\n      excludes. Those FP8 layers load through `Fp8LinearMethod`;\n      ``weight_block_size`` selects block (e.g. ``[128, 128]``) vs per-tensor","sourceCodeStart":220,"sourceCodeEnd":256,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/quark/quark.py#L220-L256","documentation":"In the same MIXED_PRECISION builder, each per-tail group whose algo is consistent is then mapped to a target spec: NVFP4/W4A16_NVFP4 -> MXFP4 target, FP8 -> kept as FP8. Any other algo (e.g. INT8, INT4, W8A8-INT8) has no online requantization path in sglang, so NotImplementedError is raised naming the offending group and algo.","triggerScenarios":"A MIXED_PRECISION quark checkpoint where some tail group uses an algo other than NVFP4, W4A16_NVFP4, or FP8, loaded with --quantization quark_mxfp4 (from_config -> _build_mixed_precision_layer_quant_config).","commonSituations":"Mixed checkpoints that combine INT8 or INT4 sublayers with FP4 sublayers; IBM Granite-style quark models with newer algo names; attempting to serve a partially-quantized checkpoint through the online MXFP4 requantization scheme.","solutions":["Re-quantize or re-export the checkpoint so every layer group uses NVFP4 or FP8 only.","Serve the model with its native quark quantization (omit --quantization quark_mxfp4) so the unsupported algos are handled by their own quark paths (if supported).","Upgrade sglang — additional algos may have been added to the online requantization allowlist."],"exampleFix":"# before\n# MIXED_PRECISION config with '*.gate_up_proj' algo INT8 -> NotImplementedError\n\n# after\n# re-export with '*.gate_up_proj' as FP8 -> kept as-is, or NVFP4 -> requantized to MXFP4","handlingStrategy":"validation","validationCode":"SUPPORTED = {\"NVFP4\", \"W4A16_NVFP4\", \"FP8\"}\nalgos = {a for algos in tail_algos.values() for a in algos}\nunsupported = algos - SUPPORTED\nif unsupported:\n    raise RuntimeError(f\"Algos not supported for online requantization: {unsupported}\")","typeGuard":"def algo_supported_for_online_requant(algo: str) -> bool:\n    return algo in {\"NVFP4\", \"W4A16_NVFP4\", \"FP8\"}","tryCatchPattern":"try:\n    QuarkConfig.from_config(quant_config=config, hf_config=hf_config)\nexcept NotImplementedError as e:\n    if \"unsupported quant algo\" in str(e):\n        load_with_native_quark_instead(model_path)\n    raise","preventionTips":["Restrict mixed-precision exports to NVFP4 + FP8 sublayers.","Lint checkpoints for algo allowlists before serving.","Watch sglang release notes for newly supported algos."],"tags":["quantization","mixed-precision","quark","unsupported-algo","mxfp4","python"],"backgroundTag":"unsupported-quantization-algo","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}