{"record":{"id":"75804229b5d67acd","repo":"sgl-project/sglang","slug":"the-hpc-ops-moe-runner-backend-requires-static-act","errorCode":null,"errorMessage":"The hpc_ops MoE runner backend requires static activation scales for per-tensor FP8 models (activation_scheme='static' in the checkpoint quantization config).","messagePattern":"The hpc_ops MoE runner backend requires static activation scales for per-tensor FP8 models \\(activation_scheme='static' in the checkpoint quantization config\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/fp8.py","lineNumber":2205,"sourceCode":"\n        - Blockwise FP8: the kernel wants [E, N/128, K/128] float32 dequant\n          scales with the K dim padded to a multiple of 4.\n        - Per-tensor FP8: the kernel wants per-expert dequant alphas\n          (weight_scale * input_scale) and a static w2 input scale; this\n          requires the static activation scheme.\n        \"\"\"\n        from sglang.srt.layers.moe.moe_runner.hpc_ops import pad_hpc_ops_block_scale\n\n        if self.block_quant:\n            layer.hpc_ops_w13_weight_scale = pad_hpc_ops_block_scale(\n                layer.w13_weight_scale_inv.data.float()\n            )\n            layer.hpc_ops_w2_weight_scale = pad_hpc_ops_block_scale(\n                layer.w2_weight_scale_inv.data.float()\n            )\n        else:\n            if layer.w13_input_scale is None or layer.w2_input_scale is None:\n                raise ValueError(\n                    \"The hpc_ops MoE runner backend requires static activation \"\n                    \"scales for per-tensor FP8 models (activation_scheme=\"\n                    \"'static' in the checkpoint quantization config).\"\n                )\n            layer.hpc_ops_gate_up_alphas = (\n                layer.w13_weight_scale.data.float() * layer.w13_input_scale.data.float()\n            )\n            layer.hpc_ops_down_alphas = (\n                layer.w2_weight_scale.data.float() * layer.w2_input_scale.data.float()\n            )\n\n    def _get_hpc_ops_quant_info(self, layer: torch.nn.Module):\n        from sglang.srt.layers.moe.moe_runner.hpc_ops import HpcOpsMoeQuantInfo\n\n        # The HPC-Ops fused kernels take no per-expert GEMM bias; refuse\n        # instead of silently dropping it.\n        if (\n            getattr(layer, \"w13_weight_bias\", None) is not None","sourceCodeStart":2187,"sourceCodeEnd":2223,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/fp8.py#L2187-L2223","documentation":"The hpc_ops MoE runner backend needs precomputed alpha factors (weight_scale * input_scale) for per-tensor FP8, so it requires static activation scales. _prepare_hpc_ops_weights raises when the block-quant branch was not taken and w13/w2 input scales are None (dynamic scheme checkpoint).","triggerScenarios":"Launching with --moe-runner-backend=hpc_ops on a per-tensor FP8 model whose checkpoint uses activation_scheme=\"dynamic\" (no input scales present) — the else branch finds w13_input_scale/w2_input_scale None and raises.","commonSituations":"Users forcing hpc_ops for performance on DeepSeek-style per-tensor FP8 checkpoints that were quantized with dynamic activations; older checkpoints without static scales.","solutions":["Use a static-activation-scheme FP8 checkpoint (activation_scheme=\"static\" with saved input scales) with hpc_ops","Switch --moe-runner-backend to auto/triton/deep_gemm which supports dynamic activations","Re-quantize the model with static activation scales if hpc_ops performance is required"],"exampleFix":"# before\n--moe-runner-backend hpc_ops  # dynamic-scheme checkpoint\n# after\n--moe-runner-backend auto","handlingStrategy":"validation","validationCode":"if args.moe_runner_backend == \"hpc_ops\" and qcfg.get(\"activation_scheme\") != \"static\":\n    raise SystemExit(\"hpc_ops MoE requires activation_scheme='static' checkpoint; use --moe-runner-backend auto\")","typeGuard":"def hpc_ops_compatible(qcfg: dict) -> bool:\n    return qcfg.get(\"activation_scheme\") == \"static\"","tryCatchPattern":null,"preventionTips":["Check checkpoint activation_scheme before selecting hpc_ops","Keep an alternate moe-runner-backend flag documented for fallback"],"tags":["quantization","fp8","moe","moe-runner-backend","activation-scheme"],"backgroundTag":"backend-requires-static-activation-scales","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}