{"record":{"id":"e10b724524b765ae","repo":"sgl-project/sglang","slug":"mixed-precision-checkpoint-has-no-nvfp4-layers-to","errorCode":null,"errorMessage":"MIXED_PRECISION checkpoint has no NVFP4 layers to requantize; load it with its native quantization method instead of --quantization quark_mxfp4.","messagePattern":"MIXED_PRECISION checkpoint has no NVFP4 layers to requantize; load it with its native quantization method instead of --quantization quark_mxfp4\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/quark/quark.py","lineNumber":436,"sourceCode":"\n    @classmethod\n    def from_config(cls, config: dict[str, Any]) -> \"QuarkConfig\":\n        # Requantization dispatch is gated on requantization_method, NOT on\n        # quant_method. Quark-exported NVFP4 carries quant_method=\"quark\" too\n        if config.get(\"requantization_method\") == \"quark_mxfp4\":\n            hf_config = config[\"hf_config\"]\n\n            # Mixed-precision source: only the NVFP4 layers are requantized to\n            # MXFP4; layers in other precisions (e.g. FP8) load through their\n            # own scheme\n            layer_map = _mixed_precision_layer_map(config)\n            if layer_map is not None:\n                config_groups = config.get(\"config_groups\")\n                layer_quant_config, has_nvfp4 = (\n                    _build_mixed_precision_layer_quant_config(layer_map, config_groups)\n                )\n                if not has_nvfp4:\n                    raise NotImplementedError(\n                        \"MIXED_PRECISION checkpoint has no NVFP4 layers to \"\n                        \"requantize; load it with its native quantization \"\n                        \"method instead of --quantization quark_mxfp4.\"\n                    )\n                source_excludes = _parse_nvfp4_excludes(config)\n                quant_config = QuarkConfig._create_online_mxfp4_config(\n                    model_type=hf_config.model_type,\n                    source_excludes=source_excludes,\n                    layer_quant_config=layer_quant_config,\n                    packed_modules_mapping=config.get(\"packed_modules_mapping\"),\n                )\n                # Excluded layers are kept as-is. When the base checkpoint is\n                # FP8-serialized (e.g. DeepSeek-V4-Pro-NVFP4: FP8 attn/\n                # shared_experts, NVFP4 routed experts) they load through FP8;\n                # `weight_block_size` selects block vs per-tensor. Pure\n                # NVFP4/ModelOpt-mixed sources keep excluded layers in bf16, and\n                # their FP8 layers (if any) are enumerated in the layer map.\n                excluded_fp8_config = _build_excluded_fp8_config(config)","sourceCodeStart":418,"sourceCodeEnd":454,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/quark/quark.py#L418-L454","documentation":"When a MIXED_PRECISION quark checkpoint is loaded with the quark_mxfp4 online requantization scheme, from_config requires at least one NVFP4 layer group to requantize into MXFP4. If the layer map contains only FP8 (or otherwise non-NVFP4) groups, has_nvfp4 is False and NotImplementedError tells you to use the checkpoint's native quantization instead.","triggerScenarios":"--quantization quark_mxfp4 on a MIXED_PRECISION checkpoint where _build_mixed_precision_layer_quant_config finds no group with algo NVFP4/W4A16_NVFP4 — e.g. an FP8-only mixed checkpoint.","commonSituations":"Users applying the quark_mxfp4 flag generically to every quark/Granite model, including FP8 mixed ones; scripts that hardcode --quantization quark_mxfp4 regardless of checkpoint contents.","solutions":["Drop --quantization quark_mxfp4 and let sglang load the model with its native (mixed FP8) quark quantization.","If you actually want MXFP4, start from a checkpoint that contains NVFP4 layers (or re-export one).","Add a pre-flight check on the checkpoint config for NVFP4 groups before enabling the flag (see validation code)."],"exampleFix":"# before\npython -m sglang.launch_server --model granite_fp8_mixed --quantization quark_mxfp4  # raises\n\n# after\npython -m sglang.launch_server --model granite_fp8_mixed  # native mixed FP8 quark path","handlingStrategy":"validation","validationCode":"layer_map = collect_layer_algos(config[\"config_groups\"])\nhas_nvfp4 = any(a in (\"NVFP4\", \"W4A16_NVFP4\") for algos in layer_map.values() for a in algos)\nif not has_nvfp4:\n    raise RuntimeError(\"No NVFP4 layers; serve natively without --quantization quark_mxfp4\")","typeGuard":"def has_nvfp4_layers(config: dict) -> bool:\n    groups = config.get(\"config_groups\", {})\n    return any(\"fp4\" in str(g).lower() for g in groups.values())","tryCatchPattern":"try:\n    QuarkConfig.from_config(quant_config=config, hf_config=hf_config)\nexcept NotImplementedError as e:\n    if \"no NVFP4 layers\" in str(e):\n        relaunch_without_quark_mxfp4_flag(model_path)\n    raise","preventionTips":["Don't hardcode --quantization quark_mxfp4; detect NVFP4 presence from the quant config first.","Keep per-model launch configs annotated with the intended quantization path.","Add a preflight lint that maps checkpoint -> supported launch flags."],"tags":["quantization","quark","mxfp4","mixed-precision","nvfp4","python"],"backgroundTag":"quantization-config-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}