sgl-project/sglang · error · ValueError

The hpc_ops MoE runner backend only supports FP8-quantized M

Error message

The hpc_ops MoE runner backend only supports FP8-quantized MoE models (Fp8MoEMethod); got quant info {type(quant_info).__name__}. Note that with expert parallelism this backend also expects global top-k ids, so other quant methods must not run with --moe-runner-backend hpc_ops.

What it means

The hpc_ops fused-experts path only accepts HpcOpsMoeQuantInfo (produced by Fp8MoEMethod), i.e. FP8-quantized MoE models. Any other quant_info type is rejected; under expert parallelism the backend also assumes global top-k ids, so other quant methods must not use it.

Source

Thrown at python/sglang/srt/layers/moe/moe_runner/hpc_ops.py:142

        )


@register_fused_func("none", "hpc_ops")
def fused_experts_none_to_hpc_ops(
    dispatch_output: StandardDispatchOutput,
    quant_info: HpcOpsMoeQuantInfo,
    runner_config: MoeRunnerConfig,
) -> StandardCombineInput:
    import hpc

    from sglang.kernels.ops.quantization.fp8_kernel import (
        scaled_fp8_quant,
        sglang_per_token_group_quant_fp8,
    )
    from sglang.srt.layers.moe.token_dispatcher.standard import StandardCombineInput

    if not isinstance(quant_info, HpcOpsMoeQuantInfo):
        raise ValueError(
            "The hpc_ops MoE runner backend only supports FP8-quantized MoE "
            "models (Fp8MoEMethod); got quant info "
            f"{type(quant_info).__name__}. Note that with expert parallelism "
            "this backend also expects global top-k ids, so other quant "
            "methods must not run with --moe-runner-backend hpc_ops."
        )
    assert (
        quant_info.w13_weight.dtype == torch.float8_e4m3fn
    ), f"expected fp8 w13_weight, got {quant_info.w13_weight.dtype}"
    assert (
        quant_info.w2_weight.dtype == torch.float8_e4m3fn
    ), f"expected fp8 w2_weight, got {quant_info.w2_weight.dtype}"
    _check_runner_config_supported(runner_config)

    x = dispatch_output.hidden_states
    topk_weights, topk_ids, _ = dispatch_output.topk_output

    assert x.dtype == torch.bfloat16, (

View on GitHub (pinned to 0132848349)

Solutions

1. Use a FP8-quantized checkpoint (Fp8MoEMethod) with hpc_ops
2. Switch to the default triton runner for non-FP8 models
3. Remove --moe-runner-backend hpc_ops entirely

Example fix

# before
--model Qwen_Qwen3-30B-A3B-Instruct-2507-AWQ --moe-runner-backend hpc_ops
# after
--model Qwen_Qwen3-30B-A3B-Instruct-2507-AWQ  (no hpc_ops flag)

from sglang.srt.layers.moe.moe_runner.hpc_ops import HpcOpsMoeQuantInfo
if server_args.moe_runner_backend == 'hpc_ops' and quant_method.__class__.__name__ != 'Fp8MoEMethod':
    raise SystemExit('hpc_ops requires an FP8 (Fp8MoEMethod) quantized model')

def ok_for_hpc_ops(quant_info) -> bool:
    from sglang.srt.layers.moe.moe_runner.hpc_ops import HpcOpsMoeQuantInfo
    return isinstance(quant_info, HpcOpsMoeQuantInfo)

try:
    fused_experts_none_to_hpc_ops(...)
except ValueError as e:
    if 'Fp8MoEMethod' in str(e):
        fallback_to_triton_runner(...)
    else:
        raise

When it happens

Trigger: Running --moe-runner-backend hpc_ops on a model with non-FP8 MoE quantization (BF16, AWQ, GPTQ, INT8, NVFP4 etc.), so fused_experts_none_to_hpc_ops receives e.g. TritonMoeQuantInfo or no quant info.

Common situations: Trying hpc_ops on an unquantized or differently-quantized checkpoint; combining --moe-runner-backend hpc_ops with --enable-ep-* flags on non-FP8 models.

Related errors

• The hpc_ops MoE runner backend does not support fused shared
• The hpc_ops MoE runner backend does not support apply_router
• The hpc_ops MoE runner backend does not support no_combine (
• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• quantize_and_serve requires ModelOpt quantization (set with

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/717c9f910850c412. Report an issue: GitHub.