{"record":{"id":"3639bbb763315cdf","repo":"sgl-project/sglang","slug":"mixed-precision-layer-group-tail-r-has-inconsist","errorCode":null,"errorMessage":"MIXED_PRECISION layer group {tail!r} has inconsistent quant algos across layers: {sorted(algos)}. SGLang requires all layers in a group to share one algo.","messagePattern":"MIXED_PRECISION layer group (.+?) has inconsistent quant algos across layers: (.+?)\\. SGLang requires all layers in a group to share one algo\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/quark/quark.py","lineNumber":225,"sourceCode":"    \"\"\"Collapse a per-layer {name: quant_algo} map into a compact\n    `layer_quant_config` keyed by fnmatch glob patterns.\n    \"\"\"\n    # suffix tail -> set of algos seen (to detect inconsistency)\n    tail_algos: Dict[str, set] = {}\n    for name, algo in layer_map.items():\n        # Suffix after the last `.layers.<idx>.` (or the whole name if\n        # unindexed); this is the part shared across all layer indices.\n        tail = re.split(r\"\\.layers\\.\\d+\\.\", name, maxsplit=1)[-1]\n        tail_algos.setdefault(tail, set()).add(algo)\n\n    fp8_is_dynamic = _fp8_is_dynamic_from_config_groups(config_groups or {})\n    fp8_spec = _fp8_per_tensor_spec(is_dynamic_input=fp8_is_dynamic)\n\n    layer_quant_config: Dict[str, Any] = {}\n    has_nvfp4 = False\n    for tail, algos in tail_algos.items():\n        if len(algos) != 1:\n            raise NotImplementedError(\n                f\"MIXED_PRECISION layer group {tail!r} has inconsistent \"\n                f\"quant algos across layers: {sorted(algos)}. SGLang requires \"\n                \"all layers in a group to share one algo.\"\n            )\n        algo = next(iter(algos))\n        pattern = \"*\" + tail\n        if algo in (\"NVFP4\", \"W4A16_NVFP4\"):\n            layer_quant_config[pattern] = _MXFP4_TARGET_SPEC\n            has_nvfp4 = True\n        elif algo == \"FP8\":\n            layer_quant_config[pattern] = fp8_spec\n        else:\n            raise NotImplementedError(\n                f\"MIXED_PRECISION layer group {tail!r} uses unsupported \"\n                f\"quant algo {algo!r}; online requantization supports NVFP4 \"\n                \"(-> MXFP4) and FP8 (kept as-is) only.\"\n            )\n    return layer_quant_config, has_nvfp4","sourceCodeStart":207,"sourceCodeEnd":243,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/quark/quark.py#L207-L243","documentation":"For MIXED_PRECISION quark checkpoints, _build_mixed_precision_layer_quant_config groups layers by name tail (e.g. 'proj', 'mlp') and requires each group to use exactly one quant algo. If layers sharing a suffix were quantized with different algos, the mapping from group -> target spec is ambiguous, so sglang raises NotImplementedError.","triggerScenarios":"Loading a MIXED_PRECISION quark checkpoint with --quantization quark_mxfp4 where, inside config_groups, two layers matching the same tail pattern (e.g. self_attn.q_proj and mlp.gate_proj if grouped by 'proj') report different quant algos like NVFP4 vs FP8 vs INT8.","commonSituations":"Hand-edited or custom-produced mixed-precision quark configs that quantize different sublayers differently but with colliding name tails; checkpoints exported by newer tooling with finer-grained mixed schemes than sglang's online requantization supports.","solutions":["Inspect the checkpoint's quant config config_groups and list each layer's quant algo; find layers sharing a tail with differing algos.","Re-export the checkpoint so all layers within each tail group share one algo (NVFP4 or FP8), or rely on the native quark method instead of --quantization quark_mxfp4.","File/support a feature request in sglang for finer-grained mixed-precision patterns if the layout is intentional."],"exampleFix":"# before\n# config_groups: self_attn.q_proj -> NVFP4, mlp.gate_proj -> FP8 (same tail 'proj' -> inconsistent)\n\n# after\n# all '*proj' layers share one algo, e.g. all NVFP4 -> pattern \"*proj\" maps to MXFP4 target spec","handlingStrategy":"validation","validationCode":"algos_by_tail = collect_tail_algos(config[\"config_groups\"])  # tail -> set of algos\nbad = {t: sorted(a) for t, a in algos_by_tail.items() if len(a) != 1}\nif bad:\n    raise RuntimeError(f\"Inconsistent algos per tail group: {bad}\")","typeGuard":"def is_consistent_mixed_precision(config: dict) -> bool:\n    groups = config.get(\"config_groups\", {})\n    tails = {}\n    for g in groups.values():\n        tail = g[\"weights\"].get(\"symmetric\") and g  # placeholder: derive tail from layer names\n    return all(len(v) == 1 for v in tails.values()) if tails else True","tryCatchPattern":"try:\n    QuarkConfig.from_config(quant_config=config, hf_config=hf_config)\nexcept NotImplementedError as e:\n    if \"inconsistent quant algos\" in str(e):\n        fail_ci_with_checkpoint_reexport_hint(e)\n    raise","preventionTips":["Validate quark config_groups with a lint script after every export.","Keep one algo per layer-name-tail in mixed-precision exports.","Version-pin the exporter that produced known-good checkpoints."],"tags":["quantization","mixed-precision","quark","mxfp4","config-validation","python"],"backgroundTag":"quantization-config-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}