{"record":{"id":"a29fb9ecea99e08f","repo":"sgl-project/sglang","slug":"for-fused-moe-layers-only-compressionformat-pack-a29fb9","errorCode":null,"errorMessage":"For Fused MoE layers, only {CompressionFormat.pack_quantized.value} is supported for the following bits: {WNA16_SUPPORTED_BITS}","messagePattern":"For Fused MoE layers, only (.+?) is supported for the following bits: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_wNa16_moe.py","lineNumber":88,"sourceCode":"        weight_quant: QuantizationArgs,\n        num_gpu_experts: int = -1,\n    ):\n        self.quant_config = quant_config\n        # Per-layer scheme already resolved by get_moe_scheme(); reuse it directly\n        # (mixed-precision MoE has no \"Linear\" config group to fall back on).\n        config = weight_quant\n        self.num_bits = config.num_bits\n        self.packed_factor = 32 // config.num_bits\n        self.strategy = config.strategy\n        self.group_size = config.group_size\n        self.actorder = config.actorder\n        self.sym = config.symmetric\n\n        if not (\n            self.quant_config.quant_format == CompressionFormat.pack_quantized.value\n            and self.num_bits in WNA16_SUPPORTED_BITS\n        ):\n            raise ValueError(\n                \"For Fused MoE layers, only \",\n                f\"{CompressionFormat.pack_quantized.value} \",\n                \"is supported for the following bits: \",\n                f\"{WNA16_SUPPORTED_BITS}\",\n            )\n        self.num_gpu_experts = num_gpu_experts\n\n    @classmethod\n    def get_min_capability(cls) -> int:\n        # ampere and up\n        return 80\n\n    def create_weights(\n        self,\n        layer: torch.nn.Module,\n        num_experts: int,\n        hidden_size: int,\n        intermediate_size_per_partition: int,","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_wNa16_moe.py#L70-L106","documentation":"For wNa16 (e.g. W4A16) Fused MoE layers, the checkpoint must use CompressionFormat.pack-quantized and a supported bit width (WNA16_SUPPORTED_BITS). Any other format or bit combination is rejected when building the MoE scheme.","triggerScenarios":"A GPTQ/AWQ-quantized MoE model exported in a non-packed format (or unsupported num_bits) dispatched to CompressedTensorsWNA16MoE.__init__.","commonSituations":"Quantizing MoE models with llmcompressor but exporting 'int-quantized' format instead of packed; mixed-format checkpoints.","solutions":["Re-export with format pack-quantized and supported bits (4)","Verify quantization_config.format in config.json","Use a non-MoE path or unquantized model if the format can't change"],"exampleFix":"// before\n\"quantization_config\": {\"format\": \"int-quantized\", \"weights\": {\"num_bits\": 4}}\n// after\n\"quantization_config\": {\"format\": \"pack-quantized\", \"weights\": {\"num_bits\": 4}}","handlingStrategy":"validation","validationCode":"assert cfg[\"quantization_config\"][\"format\"] == \"pack-quantized\" and cfg[\"weights\"][\"num_bits\"] in (4,)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Export MoE wNa16 models with pack-quantized format"],"tags":["quantization","moe","marlin","format","compressed-tensors"],"backgroundTag":"unsupported-quantization-format","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}