{"record":{"id":"7f6a0ac0ce2ebeb8","repo":"sgl-project/sglang","slug":"only-block-quant-true-is-supported-in-quark-mxfp4-7f6a0a","errorCode":null,"errorMessage":"Only block_quant=True is supported in Quark MXFP4 requantization, got block_quant=False.","messagePattern":"Only block_quant=True is supported in Quark MXFP4 requantization, got block_quant=False\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/quark/schemes/quark_w4a4_mxfp4_moe.py","lineNumber":139,"sourceCode":"                    num_experts=num_experts,\n                    hidden_size=hidden_size,\n                    intermediate_size_per_partition=intermediate_size_per_partition,\n                    original_weight_loader=original_weight_loader,\n                    extra_weight_attrs=extra_weight_attrs,\n                )\n            elif isinstance(self.dequantization_config, Fp8Config):\n                with_bias = extra_weight_attrs.pop(\"with_bias\", False)\n                self.with_bias = with_bias\n\n                if self.dequantization_config.use_mxfp8:\n                    raise NotImplementedError(\n                        \"use_mxfp8=True is not supported in Quark MXFP4 requantization.\"\n                    )\n\n                block_quant = self.dequantization_config.weight_block_size is not None\n\n                if not block_quant:\n                    raise NotImplementedError(\n                        \"Only block_quant=True is supported in Quark MXFP4 requantization, got block_quant=False.\"\n                    )\n\n                # `_fp8_loaded_numel` is used to trigger FP8 -> MXFP4 requantization once all weights are loaded.\n                # `_fp8_materialized` is used to ensure only one thread materializes weights from meta device.\n                layer._fp8_loaded_numel = 0\n                layer._fp8_materialized = False\n                layer._load_device = torch.get_default_device()\n                layer._fp8_loading_lock = threading.Lock()\n\n                # Custom weight loader handling FP8->MXFP4 conversion.\n                fp8_to_mxfp4_weight_loader = self.get_online_fp8_to_mxfp4_weight_loader(\n                    layer, original_weight_loader\n                )\n\n                extra_weight_attrs[\"weight_loader\"] = fp8_to_mxfp4_weight_loader\n                # Create FP8 MoE weight parameters on meta device to avoid device memory overhead during weight loading, as the resulting model uses MXFP4 using less device memory.\n                # The weight loader handles progressive FP8 weight materialization on device.","sourceCodeStart":121,"sourceCodeEnd":157,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/quark/schemes/quark_w4a4_mxfp4_moe.py#L121-L157","documentation":"FP8→MXFP4 requantization in QuarkW4A4MXFp4MoE requires the FP8 checkpoint to use block quantization (weight_block_size set). A per-tensor FP8 checkpoint has no per-block scales, so per-group MXFP4 requantization is impossible.","triggerScenarios":"dequantization_config is Fp8Config with weight_block_size=None in create_weights of QuarkW4A4MXFp4MoE.","commonSituations":"Loading a per-tensor FP8 quark export into the MXFP4 MoE scheme; configs exported for a different kernel expecting per-tensor scales.","solutions":["Use a block-quantized FP8 checkpoint (weight_block_size defined, typically [128,128])","Directly use an MXFP4-serialized checkpoint instead of requantizing","Re-export the model with quark using block-wise FP8 quantization"],"exampleFix":"// before\n\"dequantization_config\": {\"quant_type\": \"fp8\", \"weight_block_size\": null}\n// after\n\"dequantization_config\": {\"quant_type\": \"fp8\", \"weight_block_size\": [128, 128]}","handlingStrategy":"validation","validationCode":"assert quant_config.dequantization_config.weight_block_size is not None, 'FP8->MXFP4 requant requires block-quantized FP8'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Export with block size [128,128] for requantizable FP8","Check weight_block_size in quant config before deploy"],"tags":["quantization","fp8","block-quantization","quark","moe"],"backgroundTag":"quantization-config-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}