{"record":{"id":"985e004351f38ac1","repo":"sgl-project/sglang","slug":"for-int8-fused-moe-layers-we-require-channelwise-985e00","errorCode":null,"errorMessage":"For INT8 Fused MoE layers, we require channelwise, dynamic per token quantization. Found static input scales.","messagePattern":"For INT8 Fused MoE layers, we require channelwise, dynamic per token quantization\\. Found static input scales\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_int8_moe.py","lineNumber":52,"sourceCode":"        self.input_quant = input_quant\n        self.w13_kernel = NPUW8A8Int8MoEMethod()\n        self.w2_kernel = NPUW8A8Int8MoEMethod()\n\n        self.static_input_scales = not self.input_quant.dynamic\n        per_channel = (\n            self.weight_quant.strategy == QuantizationStrategy.CHANNEL\n            and self.input_quant.strategy == QuantizationStrategy.TOKEN\n        )\n        if not per_channel:\n            raise ValueError(\n                \"For INT8 Fused MoE layers, we require channelwise, \"\n                \"dynamic per token quantization. Found \"\n                f\"{self.weight_quant}, {self.input_quant}\"\n            )\n\n        self.static_input_scales = not self.input_quant.dynamic\n        if self.static_input_scales:\n            raise ValueError(\n                \"For INT8 Fused MoE layers, we require channelwise, \"\n                \"dynamic per token quantization. Found static input scales.\"\n            )\n\n    def create_weights(\n        self,\n        layer: torch.nn.Module,\n        num_experts: int,\n        hidden_size: int,\n        intermediate_size_per_partition: int,\n        params_dtype: torch.dtype,\n        **extra_weight_attrs,\n    ):\n\n        from sglang.srt.layers.moe.fused_moe_triton import FusedMoeWeightScaleSupported\n\n        params_dtype = torch.int8\n","sourceCodeStart":34,"sourceCodeEnd":70,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_int8_moe.py#L34-L70","documentation":"Second gate for INT8 MoE: even with correct channel/token strategies, static input scales are rejected — input_quant.dynamic must be true. Static calibrated activation scales are unsupported for INT8 fused MoE kernels.","triggerScenarios":"input_quant.dynamic == False on a compressed-tensors INT8 MoE checkpoint, i.e. calibration scales embedded in the model.","commonSituations":"Loading a statically-calibrated INT8 MoE export; conversion tools defaulting to static activation quantization.","solutions":["Re-quantize with dynamic activation quantization (drop input_scale tensors, set dynamic=true)","Switch to the FP8 MoE scheme if static scales are mandatory"],"exampleFix":"// before\n\"input_quant\": {\"strategy\": \"token\", \"dynamic\": false}\n// after\n\"input_quant\": {\"strategy\": \"token\", \"dynamic\": true}","handlingStrategy":"validation","validationCode":"assert cfg[\"input_quant\"].get(\"dynamic\", True), \"INT8 MoE rejects static input scales\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Skip calibration scale export for INT8 MoE models"],"tags":["quantization","int8","moe","static-scales"],"backgroundTag":"invalid-quantization-config-combo","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}