{"record":{"id":"70e072f5fc32f001","repo":"sgl-project/sglang","slug":"flashinfer-cutedsl-fp4-moe-only-supports-deepep-lo","errorCode":null,"errorMessage":"flashinfer_cutedsl FP4 MoE only supports DeepEP low_latency dispatch (masked layout). DeepEP normal (prefill) dispatch has no CuteDSL FP4 handler. Pass --deepep-mode low_latency or auto.","messagePattern":"flashinfer_cutedsl FP4 MoE only supports DeepEP low_latency dispatch \\(masked layout\\)\\. DeepEP normal \\(prefill\\) dispatch has no CuteDSL FP4 handler\\. Pass --deepep-mode low_latency or auto\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":7677,"sourceCode":"        if cfg.enable_waterfill:\n            self._declare(\"_handle_a2a_moe\", enforce_shared_experts_fusion=True)\n            logger.info(f\"Waterfill is enabled with moe_a2a_backend='{a2a_backend}'.\")\n\n        if a2a_backend == \"deepep\":\n            if cfg.moe_runner_backend == \"flashinfer_cutedsl\":\n                if cfg.deepep_mode == \"auto\":\n                    self._declare(\n                        \"_handle_a2a_moe\",\n                        deepep_mode=\"low_latency\",\n                    )\n                    logger.warning(\n                        \"Forcing --deepep-mode low_latency: flashinfer_cutedsl \"\n                        \"FP4 MoE has no DeepEP normal-dispatch handler, so \"\n                        \"deepep auto mode would crash during prefill. \"\n                        \"low_latency covers both prefill and decode.\"\n                    )\n                elif cfg.deepep_mode == \"normal\":\n                    raise ValueError(\n                        \"flashinfer_cutedsl FP4 MoE only supports DeepEP \"\n                        \"low_latency dispatch (masked layout). DeepEP normal \"\n                        \"(prefill) dispatch has no CuteDSL FP4 handler. Pass \"\n                        \"--deepep-mode low_latency or auto.\"\n                    )\n            if cfg.deepep_mode == \"normal\":\n                logger.warning(\"Cuda graph is disabled because deepep_mode=`normal`\")\n                self._declare(\n                    \"_handle_a2a_moe\",\n                    cuda_graph_config=with_phase(\n                        cfg.cuda_graph_config, Phase.DECODE, backend=Backend.DISABLED\n                    ),\n                )\n                self._declare(\n                    \"_handle_a2a_moe\",\n                    cuda_graph_config=with_phase(\n                        cfg.cuda_graph_config, Phase.PREFILL, backend=Backend.DISABLED\n                    ),","sourceCodeStart":7659,"sourceCodeEnd":7695,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L7659-L7695","documentation":"Raised when --moe-runner-backend flashinfer_cutedsl FP4 MoE is combined with --deepep-mode normal. The CuteDSL FP4 kernels only implement the masked (low_latency) layout; DeepEP normal prefill dispatch would crash because no FP4 handler exists for it. The resolver auto-forces low_latency in 'auto' mode, so only an explicit 'normal' reaches this error.","triggerScenarios":"Launching with --moe-runner-backend flashinfer_cutedsl (FP4 quantized MoE) plus --deepep-mode normal.","commonSituations":"Copying a deepep normal-mode prefill config from an FP8/BF16 deployment onto an NVFP4 model; explicitly pinning --deepep-mode normal for prefill throughput on a cutedsl FP4 setup.","solutions":["Change --deepep-mode to low_latency (covers both prefill and decode)","Use --deepep-mode auto, which the resolver coerces to low_latency for this combo","Switch --moe-runner-backend away from flashinfer_cutedsl if normal-mode dispatch is required"],"exampleFix":"# before\n--moe-runner-backend flashinfer_cutedsl --deepep-mode normal\n# after\n--moe-runner-backend flashinfer_cutedsl --deepep-mode low_latency","handlingStrategy":"validation","validationCode":"if moe_runner_backend == \"flashinfer_cutedsl\" and deepep_mode == \"normal\":\n    raise SystemExit(\"cutedsl FP4 requires low_latency dispatch; fix flags\")\n# or simply coerce:\ndeep_mode = \"low_latency\" if moe_runner_backend == \"flashinfer_cutedsl\" else deepep_mode","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Default to --deepep-mode auto so the resolver coerces safely","Keep FP4/cutedsl configs on low_latency dispatch"],"tags":["moe","flashinfer","cutedsl","fp4","deepep-mode","server-args"],"backgroundTag":"unsupported-configuration-combo","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}