{"record":{"id":"333cfd34426350fe","repo":"sgl-project/sglang","slug":"unsupported-ascend-dispatcher-output-dtype-self","errorCode":null,"errorMessage":"Unsupported ascend_dispatcher_output_dtype: {self.ascend_dispatcher_output_dtype}","messagePattern":"Unsupported ascend_dispatcher_output_dtype: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/moe/token_dispatcher/ascend_tp.py","lineNumber":93,"sourceCode":"\n    def set_ascend_dispatcher_output_dtype(self) -> None:\n        \"\"\"Choose init & finalize routing kernels based on quant config.\"\"\"\n        self.ascend_dispatcher_output_dtype = get_ascend_dispatcher_output_dtype(self)\n\n        if self.ascend_dispatcher_output_dtype == DispatcherOutputDtype.BF16:\n            self.init = NPUMoEInitRouting_v2(quant_mode=-1)\n            self.finalize = NPUFinalizeRouting(drop_pad_mode=2)\n            self.group_list_type = 1\n        elif self.ascend_dispatcher_output_dtype == DispatcherOutputDtype.INT8:\n            self.init = NPUMoEInitRouting_v2(quant_mode=1)\n            self.finalize = NPUFinalizeRouting(drop_pad_mode=2)\n            self.group_list_type = 1\n        elif self.ascend_dispatcher_output_dtype == DispatcherOutputDtype.MXFP8:\n            self.init = NPUMoEInitRouting_v2(quant_mode=MXFP8_QUANT_MODE)\n            self.finalize = NPUFinalizeRouting(drop_pad_mode=2)\n            self.group_list_type = 1\n        else:\n            raise ValueError(\n                f\"Unsupported ascend_dispatcher_output_dtype: {self.ascend_dispatcher_output_dtype}\"\n            )\n\n    def dispatch(\n        self, hidden_states: torch.Tensor, topk_output: TopKOutput\n    ) -> AscendTPDispatchOutput:\n        topk_weights, topk_ids, _ = topk_output\n        topk_weights = topk_weights.to(hidden_states.dtype)\n        topk_ids = topk_ids.to(torch.int32)\n        top_k = topk_weights.shape[-1]\n\n        (\n            permuted_hidden_states,\n            expanded_row_idx,\n            expert_tokens,\n            hidden_states_scale,\n        ) = self.init._init_routing(\n            hidden_states,","sourceCodeStart":75,"sourceCodeEnd":111,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/moe/token_dispatcher/ascend_tp.py#L75-L111","documentation":"Ascend (NPU) token dispatcher supports only a fixed set of dispatch-output dtypes (fp8, int8 via routing v2, MXFP8, bf16 default); set_ascend_dispatcher_output_dtype raises when configured with anything else, e.g. NVFP4. The method is invoked from __init__ and set_quant_config, so misconfiguration fails at dispatcher construction or when the quant config is applied.","triggerScenarios":"Setting ascend_dispatcher_output_dtype (typically via server args like --ascend-dispatcher-output-dtype) to a value such as nvfp4 on Ascend hardware; or applying a quant config that resolves to an unsupported DispatcherOutputDtype enum member.","commonSituations":"Porting GPU-oriented configs (nvfp4/fp4 quantization recipes) to Ascend NPU deployments; version drift where a new dtype enum was added but Ascend support lags.","solutions":["Use a supported dtype for Ascend: bf16 (default), fp8, int8, or mxfp8","Remove the --ascend-dispatcher-output-dtype nvfp4 style flag from launch args","If you need nvfp4, check for a newer SGLang/Ascend version that adds support, or file a feature request"],"exampleFix":"# before\n--ascend-dispatcher-output-dtype nvfp4  # ValueError\n\n# after\n--ascend-dispatcher-output-dtype fp8    # or omit the flag","handlingStrategy":"validation","validationCode":"ASCEND_OK = {\"bf16\", \"fp8\", \"int8\", \"mxfp8\"}\nif dtype_str not in ASCEND_OK:\n    raise SystemExit(f\"{dtype_str} unsupported on Ascend; choose from {ASCEND_OK}\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep per-platform dtype whitelists in deployment scripts","Never port nvfp4 GPU recipes to Ascend without checking the support matrix"],"tags":["ascend","npu","moe","dispatcher","dtype","quantization"],"backgroundTag":"unsupported-hardware-dtype","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}