{"record":{"id":"f2ab1086485ba66c","repo":"sgl-project/sglang","slug":"dflash-speculative-decoding-only-supports-cuda-and","errorCode":null,"errorMessage":"DFLASH speculative decoding only supports CUDA and NPU devices.","messagePattern":"DFLASH speculative decoding only supports CUDA and NPU devices\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/speculative_hook.py","lineNumber":190,"sourceCode":"        if cfg.speculative_adaptive:\n            _init_adaptive_speculative_params(server_args)\n\n    if algo is not None:\n        # A registered algorithm's callback lives outside this tree and sets\n        # fields on the record, so the writes are captured around the call.\n        declare_direct_writes(\n            server_args,\n            \"handle_speculative_decoding.custom_algo\",\n            algo.handle_server_args,\n        )\n\n\ndef _handle_dflash(server_args: ServerArgs) -> None:\n    cfg = resolving_view(server_args)\n    from sglang.srt.arg_groups.overrides import resolved_view\n\n    if not (cfg.device.startswith(\"cuda\") or cfg.device == \"npu\"):\n        raise ValueError(\n            \"DFLASH speculative decoding only supports CUDA and NPU devices.\"\n        )\n\n    if resolved_view(server_args).enable_dp_attention:\n        raise ValueError(\n            \"Currently DFLASH speculative decoding does not support dp attention.\"\n        )\n\n    if cfg.pp_size != 1:\n        raise ValueError(\n            \"Currently DFLASH speculative decoding only supports pp_size == 1.\"\n        )\n\n    if cfg.speculative_draft_model_path is None:\n        raise ValueError(\n            \"DFLASH speculative decoding requires setting --speculative-draft-model-path.\"\n        )\n","sourceCodeStart":172,"sourceCodeEnd":208,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/speculative_hook.py#L172-L208","documentation":"The DFLASH speculative decoding implementation only has kernels for CUDA and NPU devices; selecting it on CPU, ROCm/HIP, or other backends fails during server-args handling before launch.","triggerScenarios":"--speculative-algorithm DFLASH with --device not starting with 'cuda' and not 'npu' (e.g. rocm, cpu).","commonSituations":"Running a DFLASH-configured launcher on AMD GPUs or CPU-only environments/CI.","solutions":["Run on CUDA GPUs or NPU","Or choose a speculative algorithm supported on your device (or disable speculation)"],"exampleFix":"# before (on ROCm)\n--speculative-algorithm DFLASH\n# after\n--speculative-algorithm EAGLE","handlingStrategy":"validation","validationCode":"if args.speculative_algorithm == \"DFLASH\":\n    assert args.device.startswith(\"cuda\") or args.device == \"npu\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Make device a first-class input to algorithm selection in your launcher"],"tags":["sglang","dflash","speculative-decoding","device-support","hardware-gpu"],"backgroundTag":"gpu-architecture-not-supported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}