sgl-project/sglang · error · ValueError

Humming quantization for MoE only supports moe_runner_backen

Error message

Humming quantization for MoE only supports moe_runner_backend='auto' or 'humming', got {moe_runner_backend.value!r}.

What it means

Humming-quantized MoE layers route through the dedicated HUMMING MoeRunner backend. create_moe_runner reads the global moe_runner_backend setting and only accepts 'auto' or 'humming'; any explicitly forced incompatible backend (triton, flashinfer_trtllm, flashinfer_cutlass, etc.) raises this error when the MoE layer is constructed.

Source

Thrown at python/sglang/srt/layers/quantization/humming.py:1237

                input_schema=input_schema,
                weight_schema=weight_schema,
                has_bias=layer.with_bias,
                num_experts=layer.num_experts,
                torch_dtype=layer.param_dtype,
                sublayer_name=sublayer_name,
            )

            # preprocess weight for inference
            HummingMethod.transform_humming_layer(layer, sublayer_name=sublayer_name)

    def create_moe_runner(
        self,
        layer: torch.nn.Module,
        moe_runner_config: MoeRunnerConfig,
    ):
        moe_runner_backend = get_moe_runner_backend()
        if not (moe_runner_backend.is_auto() or moe_runner_backend.is_humming()):
            raise ValueError(
                "Humming quantization for MoE only supports "
                f"moe_runner_backend='auto' or 'humming', got "
                f"{moe_runner_backend.value!r}."
            )
        self.runner = MoeRunner(MoeRunnerBackend.HUMMING, moe_runner_config)

    def apply(
        self,
        layer: torch.nn.Module,
        dispatch_output: "DispatchOutput",
    ) -> "CombineInput":
        from sglang.srt.layers.moe.moe_runner.humming import HummingMoeQuantInfo

        quant_info = HummingMoeQuantInfo(layer=layer)
        return self.runner.run(dispatch_output, quant_info)

View on GitHub (pinned to 0132848349)

Solutions

1. Remove the --moe-runner-backend flag entirely so it defaults to auto
2. Or explicitly pass --moe-runner-backend humming
3. Keep model-specific launch scripts per quantization family instead of one global backend setting

Example fix

# before
python -m sglang.launch_server --model humming-model --moe-runner-backend triton
# after
python -m sglang.launch_server --model humming-model --moe-runner-backend humming

backend = server_args.moe_runner_backend
if is_humming_checkpoint(cfg):
    assert backend in (None, "auto", "humming"), f"Humming MoE requires auto/humming, got {backend}"

When it happens

Trigger: Launching a Humming-quantized model with --moe-runner-backend triton (or another non-auto/humming value) on the command line or via SGLANG_* env overrides; a server script that hardcodes a backend for all models then swaps in a Humming checkpoint.

Common situations: Reusing launch scripts written for GPTQ/AWQ models that pin a MoE backend; setting a backend globally in a multi-model serving setup.

Related errors

• Unsupported activation: {self.activation}
• Humming does not support DeepEP {output_dtype} dispatch; use
• The hpc_ops MoE runner backend does not support fused shared
• The hpc_ops MoE runner backend does not support apply_router
• The hpc_ops MoE runner backend does not support no_combine (

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c0d125a1af101dbf. Report an issue: GitHub.