sgl-project/sglang · error · ValueError

The hpc_ops MoE runner backend does not support MoE GEMM bia

Error message

The hpc_ops MoE runner backend does not support MoE GEMM biases (w13_weight_bias / w2_weight_bias); use another --moe-runner-backend for this model.

What it means

The fused hpc_ops MoE kernels accept no per-expert GEMM bias, so if the model defines w13_weight_bias or w2_weight_bias, _get_hpc_ops_quant_info refuses to run rather than silently dropping the bias and producing wrong outputs.

Source

Thrown at python/sglang/srt/layers/quantization/fp8.py:2226

                    "'static' in the checkpoint quantization config)."
                )
            layer.hpc_ops_gate_up_alphas = (
                layer.w13_weight_scale.data.float() * layer.w13_input_scale.data.float()
            )
            layer.hpc_ops_down_alphas = (
                layer.w2_weight_scale.data.float() * layer.w2_input_scale.data.float()
            )

    def _get_hpc_ops_quant_info(self, layer: torch.nn.Module):
        from sglang.srt.layers.moe.moe_runner.hpc_ops import HpcOpsMoeQuantInfo

        # The HPC-Ops fused kernels take no per-expert GEMM bias; refuse
        # instead of silently dropping it.
        if (
            getattr(layer, "w13_weight_bias", None) is not None
            or getattr(layer, "w2_weight_bias", None) is not None
        ):
            raise ValueError(
                "The hpc_ops MoE runner backend does not support MoE GEMM "
                "biases (w13_weight_bias / w2_weight_bias); use another "
                "--moe-runner-backend for this model."
            )

        if self.block_quant:
            return HpcOpsMoeQuantInfo(
                w13_weight=layer.w13_weight,
                w2_weight=layer.w2_weight,
                block_quant=True,
                global_num_experts=int(layer.num_experts),
                moe_ep_rank=int(layer.moe_ep_rank),
                w13_weight_scale_inv=layer.hpc_ops_w13_weight_scale,
                w2_weight_scale_inv=layer.hpc_ops_w2_weight_scale,
                block_shape=self.quant_config.weight_block_size,
            )
        else:
            return HpcOpsMoeQuantInfo(

View on GitHub (pinned to 0132848349)

Solutions

1. Switch --moe-runner-backend to auto, triton, or another backend that supports biases
2. Remove/strip MoE biases via re-quantization if the model tolerates it (usually it doesn't)
3. Keep hpc_ops only for bias-free models like standard DeepSeek FP8

Example fix

# before
--moe-runner-backend hpc_ops  # model has w13_weight_bias
# after
--moe-runner-backend auto

has_bias = getattr(layer, "w13_weight_bias", None) is not None or getattr(layer, "w2_weight_bias", None) is not None
if has_bias and args.moe_runner_backend == "hpc_ops":
    args.moe_runner_backend = "auto"  # bias-bearing model needs another backend

def moe_bias_free(layer) -> bool:
    return getattr(layer, "w13_weight_bias", None) is None and getattr(layer, "w2_weight_bias", None) is None

When it happens

Trigger: Applying the Fp8MoEMethod forward path with runner_backend.is_hpc_ops() on a model whose MoE layers carry weight biases (some GPT-NeoX/Qwen-style or fine-tuned checkpoints include MoE biases).

Common situations: Selecting --moe-runner-backend=hpc_ops for a model architecture with MoE biases; mixing fused-kernel backends with bias-bearing checkpoints.

Related errors

• The hpc_ops MoE runner backend requires static activation sc
• {} does not support QVG KV-cache quantization
• --quantization nvfp4_online supports only --moe-runner-backe
• Kimi expert-pack {role} quant type is unsupported
• The hpc_ops MoE runner backend only supports FP8-quantized M

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/017593dbb6f9ce07. Report an issue: GitHub.