sgl-project/sglang · error · ValueError

The hpc_ops MoE runner backend requires static activation sc

Error message

The hpc_ops MoE runner backend requires static activation scales for per-tensor FP8 models (activation_scheme='static' in the checkpoint quantization config).

What it means

The hpc_ops MoE runner backend needs precomputed alpha factors (weight_scale * input_scale) for per-tensor FP8, so it requires static activation scales. _prepare_hpc_ops_weights raises when the block-quant branch was not taken and w13/w2 input scales are None (dynamic scheme checkpoint).

Source

Thrown at python/sglang/srt/layers/quantization/fp8.py:2205

        - Blockwise FP8: the kernel wants [E, N/128, K/128] float32 dequant
          scales with the K dim padded to a multiple of 4.
        - Per-tensor FP8: the kernel wants per-expert dequant alphas
          (weight_scale * input_scale) and a static w2 input scale; this
          requires the static activation scheme.
        """
        from sglang.srt.layers.moe.moe_runner.hpc_ops import pad_hpc_ops_block_scale

        if self.block_quant:
            layer.hpc_ops_w13_weight_scale = pad_hpc_ops_block_scale(
                layer.w13_weight_scale_inv.data.float()
            )
            layer.hpc_ops_w2_weight_scale = pad_hpc_ops_block_scale(
                layer.w2_weight_scale_inv.data.float()
            )
        else:
            if layer.w13_input_scale is None or layer.w2_input_scale is None:
                raise ValueError(
                    "The hpc_ops MoE runner backend requires static activation "
                    "scales for per-tensor FP8 models (activation_scheme="
                    "'static' in the checkpoint quantization config)."
                )
            layer.hpc_ops_gate_up_alphas = (
                layer.w13_weight_scale.data.float() * layer.w13_input_scale.data.float()
            )
            layer.hpc_ops_down_alphas = (
                layer.w2_weight_scale.data.float() * layer.w2_input_scale.data.float()
            )

    def _get_hpc_ops_quant_info(self, layer: torch.nn.Module):
        from sglang.srt.layers.moe.moe_runner.hpc_ops import HpcOpsMoeQuantInfo

        # The HPC-Ops fused kernels take no per-expert GEMM bias; refuse
        # instead of silently dropping it.
        if (
            getattr(layer, "w13_weight_bias", None) is not None

View on GitHub (pinned to 0132848349)

Solutions

1. Use a static-activation-scheme FP8 checkpoint (activation_scheme="static" with saved input scales) with hpc_ops
2. Switch --moe-runner-backend to auto/triton/deep_gemm which supports dynamic activations
3. Re-quantize the model with static activation scales if hpc_ops performance is required

Example fix

# before
--moe-runner-backend hpc_ops  # dynamic-scheme checkpoint
# after
--moe-runner-backend auto

if args.moe_runner_backend == "hpc_ops" and qcfg.get("activation_scheme") != "static":
    raise SystemExit("hpc_ops MoE requires activation_scheme='static' checkpoint; use --moe-runner-backend auto")

def hpc_ops_compatible(qcfg: dict) -> bool:
    return qcfg.get("activation_scheme") == "static"

When it happens

Trigger: Launching with --moe-runner-backend=hpc_ops on a per-tensor FP8 model whose checkpoint uses activation_scheme="dynamic" (no input scales present) — the else branch finds w13_input_scale/w2_input_scale None and raises.

Common situations: Users forcing hpc_ops for performance on DeepSeek-style per-tensor FP8 checkpoints that were quantized with dynamic activations; older checkpoints without static scales.

Related errors

• Found static activation scheme for checkpoint that was not s
• QuantConfig has static quantization, but found activation sc
• The hpc_ops MoE runner backend only supports FP8-quantized M
• For FP8 Fused MoE layer, we require either per tensor or cha
• The output_size of gate's and up's weight = {intermediate_si

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/75804229b5d67acd. Report an issue: GitHub.