sgl-project/sglang · error · ValueError

Humming does not support DeepEP

Error message

Humming does not support DeepEP {output_dtype} dispatch; use --deepep-dispatcher-output-dtype=bf16 or fp8.

What it means

configure_humming_deepep_dispatch resolves the DeepEP dispatcher output dtype (from --deepep-dispatcher-output-dtype, defaulting via the SGLANG_DEEPEP_BF16_DISPATCH env var) and only supports 'bf16' or 'fp8'. Any other string (typos like 'bf16 ', 'FP8', 'fp16', 'int8') raises this error during MoE weight processing or layer preparation.

Solutions

1. Set --deepep-dispatcher-output-dtype bf16 (or fp8) exactly
2. Or drop the flag and let it default via SGLANG_DEEPEP_BF16_DISPATCH
3. Check for trailing whitespace/case in script-provided dtype strings

Example fix

# before
--deepep-dispatcher-output-dtype fp16
# after
--deepep-dispatcher-output-dtype bf16

dtype = server_args.deepep_dispatcher_output_dtype or "auto"
assert dtype in ("auto", "bf16", "fp8"), f"unsupported dispatch dtype {dtype}"

When it happens

Trigger: Passing --deepep-dispatcher-output-dtype fp16 or any non-bf16/fp8 value on a Humming model with DeepEP enabled; env-var overrides that inject an invalid dtype string.

Common situations: Copy-pasting DeepEP flags tuned for other runtimes; assuming fp16 dispatch is supported because the dtype exists elsewhere in SGLang; case-sensitive typos.

Related errors

• DeepEP returned FP8 input while Humming is configured for…
• Humming expected DeepEP FP8 hidden states and group-128…
• Humming quantization for MoE only supports…
• Unsupported activation
• AfmoeConfig must define `num_experts`.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cf03ba383047f6ec. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/layers/quantization/humming_utils.py:62

    if dispatcher is None:
        return

    quant_config = dict(getattr(dispatcher, "quant_config", None) or {})
    quant_config["dispatcher_output_dtype"] = output_dtype
    dispatcher.set_quant_config(quant_config)


def configure_humming_deepep_dispatch(layer: torch.nn.Module) -> bool:
    if not get_moe_a2a_backend().is_deepep():
        layer._humming_uses_deepep_fp8_dispatch = False
        _set_humming_dispatcher_output_dtype(layer, "bf16")
        return False

    output_dtype = get_exec().moe.deepep_dispatcher_output_dtype
    if output_dtype == "auto":
        output_dtype = "bf16" if envs.SGLANG_DEEPEP_BF16_DISPATCH.get() else "fp8"
    if output_dtype not in ("bf16", "fp8"):
        raise ValueError(
            f"Humming does not support DeepEP {output_dtype} dispatch; "
            "use --deepep-dispatcher-output-dtype=bf16 or fp8."
        )

    _set_humming_dispatcher_output_dtype(layer, output_dtype)
    use_fp8 = output_dtype == "fp8"
    layer._humming_uses_deepep_fp8_dispatch = use_fp8
    return use_fp8


def make_humming_deepep_input_schema(
    sublayer_name: str, shape_k: int
) -> HummingInputSchema:
    if shape_k % 128 != 0:
        raise ValueError(
            f"Humming FP8 dispatch requires {sublayer_name} K={shape_k} "
            "to be divisible by 128."
        )

View on GitHub (pinned to 0132848349)