{"record":{"id":"fb83c84be8af7a73","repo":"sgl-project/sglang","slug":"ascend-a2-a3-npu-does-not-support-nvfp4-deepep-dis","errorCode":null,"errorMessage":"Ascend A2/A3 NPU does not support nvfp4 deepep_dispatcher_output_dtype.","messagePattern":"Ascend A2/A3 NPU does not support nvfp4 deepep_dispatcher_output_dtype\\.","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/moe/token_dispatcher/deepep.py","lineNumber":484,"sourceCode":"        config = config_map[self.deepep_output_dtype]\n        self.use_fp8 = config[\"use_fp8\"]\n        self.use_nvfp4 = config[\"use_nvfp4\"]\n\n        # Handle environment variables\n        if _is_npu:\n            self._update_int8_quant_env()\n\n    def _validate_and_adjust_dtype(self) -> None:\n        \"\"\"Validate dtype against hardware and adjust if necessary.\"\"\"\n        if _is_npu:\n            if self.deepep_output_dtype == DispatcherOutputDtype.FP8:\n                logger.warning_once(\n                    \"Ascend A2/A3 NPU does not support fp8 \"\n                    \"deepep_dispatcher_output_dtype, switching to int8...\"\n                )\n                self.deepep_output_dtype = DispatcherOutputDtype.INT8\n            elif self.deepep_output_dtype == DispatcherOutputDtype.NVFP4:\n                raise RuntimeError(\n                    \"Ascend A2/A3 NPU does not support nvfp4 deepep_dispatcher_output_dtype.\"\n                )\n        else:\n            if self.deepep_output_dtype == DispatcherOutputDtype.INT8:\n                logger.warning_once(\n                    \"GPU does not support int8 \"\n                    \"deepep_dispatcher_output_dtype, switching to fp8...\"\n                )\n                self.deepep_output_dtype = DispatcherOutputDtype.FP8\n            # NVFP4 is supported on GPU, no adjustment needed\n\n    def _update_int8_quant_env(self) -> None:\n        \"\"\"TODO adapt different quantization schemes for base model and draft model on NPU\"\"\"\n        pass\n\n    def set_overlap_args(\n        self, combine_overlap_args: CombineOverlapArgs, meta_overlap_args: dict\n    ) -> None:","sourceCodeStart":466,"sourceCodeEnd":502,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/moe/token_dispatcher/deepep.py#L466-L502","documentation":"When quantizing dispatch output, DeepEP-style dispatchers validate the requested deepep_dispatcher_output_dtype against the hardware: on Ascend A2/A3 NPUs fp8 silently downgrades to int8 with a warning, but NVFP4 is a hard RuntimeError — Ascend NPUs have no fp4 support. Raised from _validate_and_adjust_dtype via set_deepep_dispatcher_dtype.","triggerScenarios":"Configuring deepep_dispatcher_output_dtype to nvfp4 (e.g. --deepep-dispatcher-output-dtype nvfp4, common for NVFP4-quantized models like some Blackwell recipes) while running on an Ascend A2/A3 NPU.","commonSituations":"Taking a GPU (Blackwell) nvfp4 serving recipe and running it on Huawei Ascend hardware; config templates copied across platforms without checking NPU support matrix.","solutions":["Use a supported dtype on Ascend: bf16, fp8 (auto-switches to int8 with warning), or int8","Don't run nvfp4-quantized MoE checkpoints on A2/A3 NPUs; use an fp8/int8-quantized checkpoint instead","Move the workload to supported NVIDIA hardware if nvfp4 is a hard requirement"],"exampleFix":"# before\n--deepep-dispatcher-output-dtype nvfp4  # on Ascend -> RuntimeError\n\n# after\n--deepep-dispatcher-output-dtype int8  # or fp8/bf16 on Ascend","handlingStrategy":"validation","validationCode":"import torch\nif getattr(torch, \"npu\", None) is not None and dtype == \"nvfp4\":\n    raise SystemExit(\"nvfp4 dispatch dtype unsupported on Ascend A2/A3; use int8/fp8/bf16\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Gate dispatcher dtype by hardware platform in launch scripts","Prefer int8 dispatch output on Ascend; fp8 silently downgrades"],"tags":["ascend","npu","deepep","nvfp4","quantization","hardware-support"],"backgroundTag":"unsupported-hardware-dtype","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}