{"record":{"id":"fe191ad703cd94ee","repo":"sgl-project/sglang","slug":"use-mxfp8-true-is-not-supported-in-quark-mxfp4-req-fe191a","errorCode":null,"errorMessage":"use_mxfp8=True is not supported in Quark MXFP4 requantization.","messagePattern":"use_mxfp8=True is not supported in Quark MXFP4 requantization\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/quark/schemes/quark_w4a4_mxfp4_moe.py","lineNumber":132,"sourceCode":"\n        # Handle source-checkpoint -> MXFP4 requantization at load time. The\n        # source may be NVFP4 (ModelOpt/Quark) or FP8 (block-quantized).\n        if self.dequantization_config is not None:\n            if isinstance(self.dequantization_config, Nvfp4SourceConfig):\n                self._create_weights_from_nvfp4_moe(\n                    layer=layer,\n                    num_experts=num_experts,\n                    hidden_size=hidden_size,\n                    intermediate_size_per_partition=intermediate_size_per_partition,\n                    original_weight_loader=original_weight_loader,\n                    extra_weight_attrs=extra_weight_attrs,\n                )\n            elif isinstance(self.dequantization_config, Fp8Config):\n                with_bias = extra_weight_attrs.pop(\"with_bias\", False)\n                self.with_bias = with_bias\n\n                if self.dequantization_config.use_mxfp8:\n                    raise NotImplementedError(\n                        \"use_mxfp8=True is not supported in Quark MXFP4 requantization.\"\n                    )\n\n                block_quant = self.dequantization_config.weight_block_size is not None\n\n                if not block_quant:\n                    raise NotImplementedError(\n                        \"Only block_quant=True is supported in Quark MXFP4 requantization, got block_quant=False.\"\n                    )\n\n                # `_fp8_loaded_numel` is used to trigger FP8 -> MXFP4 requantization once all weights are loaded.\n                # `_fp8_materialized` is used to ensure only one thread materializes weights from meta device.\n                layer._fp8_loaded_numel = 0\n                layer._fp8_materialized = False\n                layer._load_device = torch.get_default_device()\n                layer._fp8_loading_lock = threading.Lock()\n\n                # Custom weight loader handling FP8->MXFP4 conversion.","sourceCodeStart":114,"sourceCodeEnd":150,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/quark/schemes/quark_w4a4_mxfp4_moe.py#L114-L150","documentation":"When requantizing an FP8-serialized checkpoint into MXFP4 inside QuarkW4A4MXFp4MoE.create_weights, the dequantization config is Fp8Config but with use_mxfp8=True. Only plain FP8 checkpoints can be requantized to MXFP4, not already-MXFP8 ones.","triggerScenarios":"Quark quant config whose dequantization_config is Fp8Config with use_mxfp8=True loaded by QuarkW4A4MXFp4MoE.create_weights.","commonSituations":"Mixing an MXFP8 AMD checkpoint with an MXFP4 requant path; hand-edited quant config; newer quark export setting use_mxfp8 by default.","solutions":["Export the model with plain FP8 (use_mxfp8=False) weights so requantization to MXFP4 is possible","Use an MXFP4-serialized checkpoint directly instead of requantizing","Use the appropriate scheme for MXFP8 checkpoints instead of QuarkW4A4MXFp4MoE"],"exampleFix":"// before\n\"quant_config\": {\"dequantization_config\": {\"quant_type\": \"fp8\", \"use_mxfp8\": true}}\n// after\n\"quant_config\": {\"dequantization_config\": {\"quant_type\": \"fp8\", \"use_mxfp8\": false}}","handlingStrategy":"validation","validationCode":"dq = quant_config.dequantization_config\nassert not (type(dq).__name__ == 'Fp8Config' and getattr(dq, 'use_mxfp8', False)), 'use_mxfp8 checkpoints cannot be requantized to MXFP4'","typeGuard":"def is_requantizable_fp8(dq):\n    return type(dq).__name__ == 'Fp8Config' and not getattr(dq, 'use_mxfp8', False) and dq.weight_block_size is not None","tryCatchPattern":null,"preventionTips":["Validate dequantization_config before launching a long job","Keep quark export settings consistent (mxfp8 checkpoints use mxfp8 schemes)"],"tags":["quantization","mxfp8","mxfp4","quark","config-mismatch"],"backgroundTag":"quantization-config-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}