sgl-project/sglang · error · ValueError

Unsupported ascend_dispatcher_output_dtype: {self.ascend_dis

Error message

Unsupported ascend_dispatcher_output_dtype: {self.ascend_dispatcher_output_dtype}

What it means

Ascend (NPU) token dispatcher supports only a fixed set of dispatch-output dtypes (fp8, int8 via routing v2, MXFP8, bf16 default); set_ascend_dispatcher_output_dtype raises when configured with anything else, e.g. NVFP4. The method is invoked from __init__ and set_quant_config, so misconfiguration fails at dispatcher construction or when the quant config is applied.

Source

Thrown at python/sglang/srt/layers/moe/token_dispatcher/ascend_tp.py:93

    def set_ascend_dispatcher_output_dtype(self) -> None:
        """Choose init & finalize routing kernels based on quant config."""
        self.ascend_dispatcher_output_dtype = get_ascend_dispatcher_output_dtype(self)

        if self.ascend_dispatcher_output_dtype == DispatcherOutputDtype.BF16:
            self.init = NPUMoEInitRouting_v2(quant_mode=-1)
            self.finalize = NPUFinalizeRouting(drop_pad_mode=2)
            self.group_list_type = 1
        elif self.ascend_dispatcher_output_dtype == DispatcherOutputDtype.INT8:
            self.init = NPUMoEInitRouting_v2(quant_mode=1)
            self.finalize = NPUFinalizeRouting(drop_pad_mode=2)
            self.group_list_type = 1
        elif self.ascend_dispatcher_output_dtype == DispatcherOutputDtype.MXFP8:
            self.init = NPUMoEInitRouting_v2(quant_mode=MXFP8_QUANT_MODE)
            self.finalize = NPUFinalizeRouting(drop_pad_mode=2)
            self.group_list_type = 1
        else:
            raise ValueError(
                f"Unsupported ascend_dispatcher_output_dtype: {self.ascend_dispatcher_output_dtype}"
            )

    def dispatch(
        self, hidden_states: torch.Tensor, topk_output: TopKOutput
    ) -> AscendTPDispatchOutput:
        topk_weights, topk_ids, _ = topk_output
        topk_weights = topk_weights.to(hidden_states.dtype)
        topk_ids = topk_ids.to(torch.int32)
        top_k = topk_weights.shape[-1]

        (
            permuted_hidden_states,
            expanded_row_idx,
            expert_tokens,
            hidden_states_scale,
        ) = self.init._init_routing(
            hidden_states,

View on GitHub (pinned to 0132848349)

Solutions

1. Use a supported dtype for Ascend: bf16 (default), fp8, int8, or mxfp8
2. Remove the --ascend-dispatcher-output-dtype nvfp4 style flag from launch args
3. If you need nvfp4, check for a newer SGLang/Ascend version that adds support, or file a feature request

Example fix

# before
--ascend-dispatcher-output-dtype nvfp4  # ValueError

# after
--ascend-dispatcher-output-dtype fp8    # or omit the flag

ASCEND_OK = {"bf16", "fp8", "int8", "mxfp8"}
if dtype_str not in ASCEND_OK:
    raise SystemExit(f"{dtype_str} unsupported on Ascend; choose from {ASCEND_OK}")

When it happens

Trigger: Setting ascend_dispatcher_output_dtype (typically via server args like --ascend-dispatcher-output-dtype) to a value such as nvfp4 on Ascend hardware; or applying a quant config that resolves to an unsupported DispatcherOutputDtype enum member.

Common situations: Porting GPU-oriented configs (nvfp4/fp4 quantization recipes) to Ascend NPU deployments; version drift where a new dtype enum was added but Ascend support lags.

Related errors

• NPU packed attention requires q, k, and v with the same dtyp
• fuse_swiglu_interleaved set on an incompatible fused_moe cal
• combine() called before dispatch()
• Ascend A2/A3 NPU does not support nvfp4 deepep_dispatcher_ou
• The W8A8Int8 Fused MoE scheme is implemented only for NPU fo

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/333cfd34426350fe. Report an issue: GitHub.