{"record":{"id":"717c9f910850c412","repo":"sgl-project/sglang","slug":"the-hpc-ops-moe-runner-backend-only-supports-fp8-q","errorCode":null,"errorMessage":"The hpc_ops MoE runner backend only supports FP8-quantized MoE models (Fp8MoEMethod); got quant info {type(quant_info).__name__}. Note that with expert parallelism this backend also expects global top-k ids, so other quant methods must not run with --moe-runner-backend hpc_ops.","messagePattern":"The hpc_ops MoE runner backend only supports FP8-quantized MoE models \\(Fp8MoEMethod\\); got quant info (.+?)\\. Note that with expert parallelism this backend also expects global top-k ids, so other quant methods must not run with --moe-runner-backend hpc_ops\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/moe/moe_runner/hpc_ops.py","lineNumber":142,"sourceCode":"        )\n\n\n@register_fused_func(\"none\", \"hpc_ops\")\ndef fused_experts_none_to_hpc_ops(\n    dispatch_output: StandardDispatchOutput,\n    quant_info: HpcOpsMoeQuantInfo,\n    runner_config: MoeRunnerConfig,\n) -> StandardCombineInput:\n    import hpc\n\n    from sglang.kernels.ops.quantization.fp8_kernel import (\n        scaled_fp8_quant,\n        sglang_per_token_group_quant_fp8,\n    )\n    from sglang.srt.layers.moe.token_dispatcher.standard import StandardCombineInput\n\n    if not isinstance(quant_info, HpcOpsMoeQuantInfo):\n        raise ValueError(\n            \"The hpc_ops MoE runner backend only supports FP8-quantized MoE \"\n            \"models (Fp8MoEMethod); got quant info \"\n            f\"{type(quant_info).__name__}. Note that with expert parallelism \"\n            \"this backend also expects global top-k ids, so other quant \"\n            \"methods must not run with --moe-runner-backend hpc_ops.\"\n        )\n    assert (\n        quant_info.w13_weight.dtype == torch.float8_e4m3fn\n    ), f\"expected fp8 w13_weight, got {quant_info.w13_weight.dtype}\"\n    assert (\n        quant_info.w2_weight.dtype == torch.float8_e4m3fn\n    ), f\"expected fp8 w2_weight, got {quant_info.w2_weight.dtype}\"\n    _check_runner_config_supported(runner_config)\n\n    x = dispatch_output.hidden_states\n    topk_weights, topk_ids, _ = dispatch_output.topk_output\n\n    assert x.dtype == torch.bfloat16, (","sourceCodeStart":124,"sourceCodeEnd":160,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/moe/moe_runner/hpc_ops.py#L124-L160","documentation":"The hpc_ops fused-experts path only accepts HpcOpsMoeQuantInfo (produced by Fp8MoEMethod), i.e. FP8-quantized MoE models. Any other quant_info type is rejected; under expert parallelism the backend also assumes global top-k ids, so other quant methods must not use it.","triggerScenarios":"Running --moe-runner-backend hpc_ops on a model with non-FP8 MoE quantization (BF16, AWQ, GPTQ, INT8, NVFP4 etc.), so fused_experts_none_to_hpc_ops receives e.g. TritonMoeQuantInfo or no quant info.","commonSituations":"Trying hpc_ops on an unquantized or differently-quantized checkpoint; combining --moe-runner-backend hpc_ops with --enable-ep-* flags on non-FP8 models.","solutions":["Use a FP8-quantized checkpoint (Fp8MoEMethod) with hpc_ops","Switch to the default triton runner for non-FP8 models","Remove --moe-runner-backend hpc_ops entirely"],"exampleFix":"# before\n--model Qwen_Qwen3-30B-A3B-Instruct-2507-AWQ --moe-runner-backend hpc_ops\n# after\n--model Qwen_Qwen3-30B-A3B-Instruct-2507-AWQ  (no hpc_ops flag)","handlingStrategy":"type-guard","validationCode":"from sglang.srt.layers.moe.moe_runner.hpc_ops import HpcOpsMoeQuantInfo\nif server_args.moe_runner_backend == 'hpc_ops' and quant_method.__class__.__name__ != 'Fp8MoEMethod':\n    raise SystemExit('hpc_ops requires an FP8 (Fp8MoEMethod) quantized model')","typeGuard":"def ok_for_hpc_ops(quant_info) -> bool:\n    from sglang.srt.layers.moe.moe_runner.hpc_ops import HpcOpsMoeQuantInfo\n    return isinstance(quant_info, HpcOpsMoeQuantInfo)","tryCatchPattern":"try:\n    fused_experts_none_to_hpc_ops(...)\nexcept ValueError as e:\n    if 'Fp8MoEMethod' in str(e):\n        fallback_to_triton_runner(...)\n    else:\n        raise","preventionTips":["Only pair hpc_ops with FP8 checkpoints","Unit-test quant_info type against backend before serving"],"tags":["sglang","moe","hpc-ops","fp8","quantization","config-validation"],"backgroundTag":"unsupported-quantization-method","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}