sgl-project/sglang · error · ValueError
DSpark speculative decoding only supports CUDA or NPU…
Error message
DSpark speculative decoding only supports CUDA or NPU device.
What it means
Configuration validation error: the DSpark speculative decoding implementation only supports CUDA or NPU devices and is rejected elsewhere.
Solutions
- Run DSpark only on CUDA or NPU hardware
- Switch to a speculative algorithm supported on your device
- Check SGLANG_DEVICE / device override env vars are not forcing an unsupported device string
Example fix
# before (on ROCm) --speculative-algorithm DSPARK # after --speculative-algorithm EAGLE # or run on a CUDA/NPU host
Defensive patterns
Strategy: validation
Validate before calling
if not (args.device.startswith('cuda') or args.device.startswith('npu')):
raise SystemExit(f'DSPARK unsupported on device {args.device}') Prevention
- Gate speculative-algorithm selection on the detected device in cluster launchers
When it happens
Trigger: Launching with speculative_algorithm=DSPARK on a device string that does not start with 'cuda' or 'npu' (e.g. AMD GPUs where device resolves to 'rocm.hip').
Common situations: Running on AMD/ROCm clusters and reusing DSpark launch configs from NVIDIA machines; CPU-only smoke tests with spec decoding enabled.
Related errors
- candidates and next_token_logits must be on the same…
- Currently DSpark speculative decoding only supports pp_size…
- DFLASH speculative decoding only supports CUDA and NPU…
- DSPARK aux hidden capture requires PP=1.
- DSpark could not resolve speculative_num_draft_tokens; set…
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/716f03d38cebfdda.
Report an issue: GitHub.
Appendix: source
Thrown at python/sglang/srt/arg_groups/speculative_hook.py:351
)
logger.warning(
"Mixed chunked prefill is disabled because of using dflash speculative decoding."
)
def _target_checkpoint_bundles_dspark_draft(server_args: ServerArgs) -> bool:
from sglang.srt.speculative.dspark_components.dspark_config import (
checkpoint_bundles_dspark_draft,
)
return checkpoint_bundles_dspark_draft(server_args.get_model_config().hf_config)
def _handle_dspark(server_args: ServerArgs) -> None:
cfg = resolving_view(server_args)
_is_npu = cfg.device.startswith("npu")
if not cfg.device.startswith(("cuda", "npu")):
raise ValueError(
"DSpark speculative decoding only supports CUDA or NPU device."
)
# dp_size==1 with dp_attention is a degenerate flag under DSV4 CP; skip DP-only checks.
if cfg.enable_dp_attention and cfg.dp_size > 1:
if not cfg.enable_dp_lm_head:
raise ValueError("DSpark with dp attention requires --enable-dp-lm-head.")
if not _is_npu and cfg.moe_a2a_backend not in ("none", "megamoe"):
raise ValueError(
"DSpark with dp attention supports moe_a2a_backend 'none' "
"(built-in TP MoE) or 'megamoe', got "
f"{cfg.moe_a2a_backend!r}."
)
if not _is_npu and cfg.moe_a2a_backend != "none":
from sglang.srt.speculative.ragged_verify import (
RaggedVerifyMode,
read_ragged_verify_mode,
)View on GitHub (pinned to 0132848349)