{"record":{"id":"c0d125a1af101dbf","repo":"sgl-project/sglang","slug":"humming-quantization-for-moe-only-supports-moe-run","errorCode":null,"errorMessage":"Humming quantization for MoE only supports moe_runner_backend='auto' or 'humming', got {moe_runner_backend.value!r}.","messagePattern":"Humming quantization for MoE only supports moe_runner_backend='auto' or 'humming', got (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/humming.py","lineNumber":1237,"sourceCode":"                input_schema=input_schema,\n                weight_schema=weight_schema,\n                has_bias=layer.with_bias,\n                num_experts=layer.num_experts,\n                torch_dtype=layer.param_dtype,\n                sublayer_name=sublayer_name,\n            )\n\n            # preprocess weight for inference\n            HummingMethod.transform_humming_layer(layer, sublayer_name=sublayer_name)\n\n    def create_moe_runner(\n        self,\n        layer: torch.nn.Module,\n        moe_runner_config: MoeRunnerConfig,\n    ):\n        moe_runner_backend = get_moe_runner_backend()\n        if not (moe_runner_backend.is_auto() or moe_runner_backend.is_humming()):\n            raise ValueError(\n                \"Humming quantization for MoE only supports \"\n                f\"moe_runner_backend='auto' or 'humming', got \"\n                f\"{moe_runner_backend.value!r}.\"\n            )\n        self.runner = MoeRunner(MoeRunnerBackend.HUMMING, moe_runner_config)\n\n    def apply(\n        self,\n        layer: torch.nn.Module,\n        dispatch_output: \"DispatchOutput\",\n    ) -> \"CombineInput\":\n        from sglang.srt.layers.moe.moe_runner.humming import HummingMoeQuantInfo\n\n        quant_info = HummingMoeQuantInfo(layer=layer)\n        return self.runner.run(dispatch_output, quant_info)\n","sourceCodeStart":1219,"sourceCodeEnd":1253,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/humming.py#L1219-L1253","documentation":"Humming-quantized MoE layers route through the dedicated HUMMING MoeRunner backend. create_moe_runner reads the global moe_runner_backend setting and only accepts 'auto' or 'humming'; any explicitly forced incompatible backend (triton, flashinfer_trtllm, flashinfer_cutlass, etc.) raises this error when the MoE layer is constructed.","triggerScenarios":"Launching a Humming-quantized model with --moe-runner-backend triton (or another non-auto/humming value) on the command line or via SGLANG_* env overrides; a server script that hardcodes a backend for all models then swaps in a Humming checkpoint.","commonSituations":"Reusing launch scripts written for GPTQ/AWQ models that pin a MoE backend; setting a backend globally in a multi-model serving setup.","solutions":["Remove the --moe-runner-backend flag entirely so it defaults to auto","Or explicitly pass --moe-runner-backend humming","Keep model-specific launch scripts per quantization family instead of one global backend setting"],"exampleFix":"# before\npython -m sglang.launch_server --model humming-model --moe-runner-backend triton\n# after\npython -m sglang.launch_server --model humming-model --moe-runner-backend humming","handlingStrategy":"validation","validationCode":"backend = server_args.moe_runner_backend\nif is_humming_checkpoint(cfg):\n    assert backend in (None, \"auto\", \"humming\"), f\"Humming MoE requires auto/humming, got {backend}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Don't set --moe-runner-backend globally in shared scripts","Default to 'auto' and only pin backends per model family"],"tags":["moe","humming","runner-backend","config-validation"],"backgroundTag":"unsupported-moe-runner-backend","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}