{"record":{"id":"054dff2955510128","repo":"sgl-project/sglang","slug":"unsupported-fusedmoe-scheme-weight-quant-inpu","errorCode":null,"errorMessage":"Unsupported FusedMoe scheme: {weight_quant}, {input_quant}","messagePattern":"Unsupported FusedMoe scheme: (.+?), (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py","lineNumber":901,"sourceCode":"                    \"The W8A8Int8 Fused MoE scheme is implemented only for NPU for now.\"\n                )\n        elif self._is_wint4afp8(weight_quant, input_quant):\n            # On NPU prefer the dedicated NPU W4A8Int8 path when activations are INT8.\n            if _is_npu and self._is_dynamic_token_w4a8(weight_quant, input_quant):\n                logger.info_once(\"Using NPUCompressedTensorsW4A8Int8DynamicMoE\")\n                return NPUCompressedTensorsW4A8Int8DynamicMoE(self)\n            logger.info_once(\"Using CompressedTensorsW4AFP8MoE\")\n            return CompressedTensorsW4AFP8MoE(self, weight_quant, input_quant)\n        elif self._is_dynamic_token_w4a8(weight_quant, input_quant):\n            if _is_npu:\n                logger.info_once(\"Using NPUCompressedTensorsW4A8Int8DynamicMoE\")\n                return NPUCompressedTensorsW4A8Int8DynamicMoE(self)\n            else:\n                raise NotImplementedError(\n                    \"The W4A8Int8 Fused MoE scheme is implemented only for NPU for now.\"\n                )\n        else:\n            raise RuntimeError(\n                f\"Unsupported FusedMoe scheme: {weight_quant}, {input_quant}\"\n            )\n\n    def get_linear_scheme(\n        self,\n        layer: torch.nn.Module,\n        layer_name: Optional[str] = None,\n        matched_target: Optional[str] = None,\n    ) -> Optional[CompressedTensorsLinearScheme]:\n        \"\"\"\n        compressed-tensors supports non uniform in the following way:\n\n        targets of config_groups: There can be N config_groups which each\n            have a quantization scheme. Each config_group has a list of targets\n            which can be a full layer_name, a regex for a layer_name, or\n            an nn.Module name.\n\n        Detect whether a layer_name is found in any target and","sourceCodeStart":883,"sourceCodeEnd":919,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py#L883-L919","documentation":"The MoE layer's (weight_quant, input_quant) combination matched none of the recognized compressed-tensors MoE schemes (FP8, dynamic W8A8, W4AFP8, dynamic W4A8, WNA16, etc.), so get_moe_scheme raises RuntimeError listing the offending quant configs.","triggerScenarios":"Loading a compressed-tensors checkpoint whose MoE expert layers use an exotic or unrecognized quantization combo, e.g. new llmcompressor scheme versions, unsupported bit-widths, or unusual sparsity/scale structures; also when MoE layers exist but quant config has no weights scheme at all.","commonSituations":"Quantizing with a newer llmcompressor version producing scheme strings sglang doesn't parse; partially-quantized or custom-recipe checkpoints; version mismatch between checkpoint creation and sglang release.","solutions":["Print/inspect target_scheme_map for the MoE layer to see the actual (weight_quant, input_quant) and compare against supported schemes in get_moe_scheme","Re-quantize the model with a scheme sglang supports (FP8, W8A8, W4A16/WNA16, INT4 group)","Upgrade sglang to a version supporting the new scheme","Exclude MoE layers from quantization (ignore= model.experts) so they run unquantized"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"supported = {\"FP8\",\"INT8 (static)\",\"dynamic INT8\",\"W4A8FP8\",\"dynamic W4A8INT8\",\"WNA16\"}\nprint(model_cfg[\"quantization_config\"][\"config\"])\n# manually compare weights/input_activations combos against sglang get_moe_scheme branches","typeGuard":null,"tryCatchPattern":"try:\n    model = sglang.Engine(model_path=...)\nexcept RuntimeError as e:\n    if \"Unsupported FusedMoe scheme\" in str(e):\n        # log (weight_quant, input_quant) and fall back to unquantized/gptq checkpoint\n        ...","preventionTips":["Pin llmcompressor and sglang versions used to create checkpoints","Prefer widely-supported schemes (FP8, W4A16 group) when quantizing MoE models","Test-launch checkpoints in CI before deploying"],"tags":["quantization","moe","unsupported-scheme","compressed-tensors"],"backgroundTag":"unsupported-quantization-scheme","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}