sgl-project/sglang · error · ValueError

DSpark speculative decoding only supports CUDA or NPU device

Error message

DSpark speculative decoding only supports CUDA or NPU device.

What it means

DSpark speculative decoding is implemented only for CUDA and NPU accelerators. The hook checks cfg.device and rejects anything else (e.g. ROCm/HIP reported as 'rocm', xpu, cpu) at server-args time.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:351

        )
        logger.warning(
            "Mixed chunked prefill is disabled because of using dflash speculative decoding."
        )


def _target_checkpoint_bundles_dspark_draft(server_args: ServerArgs) -> bool:
    from sglang.srt.speculative.dspark_components.dspark_config import (
        checkpoint_bundles_dspark_draft,
    )

    return checkpoint_bundles_dspark_draft(server_args.get_model_config().hf_config)


def _handle_dspark(server_args: ServerArgs) -> None:
    cfg = resolving_view(server_args)
    _is_npu = cfg.device.startswith("npu")
    if not cfg.device.startswith(("cuda", "npu")):
        raise ValueError(
            "DSpark speculative decoding only supports CUDA or NPU device."
        )

    # dp_size==1 with dp_attention is a degenerate flag under DSV4 CP; skip DP-only checks.
    if cfg.enable_dp_attention and cfg.dp_size > 1:
        if not cfg.enable_dp_lm_head:
            raise ValueError("DSpark with dp attention requires --enable-dp-lm-head.")
        if not _is_npu and cfg.moe_a2a_backend not in ("none", "megamoe"):
            raise ValueError(
                "DSpark with dp attention supports moe_a2a_backend 'none' "
                "(built-in TP MoE) or 'megamoe', got "
                f"{cfg.moe_a2a_backend!r}."
            )
        if not _is_npu and cfg.moe_a2a_backend != "none":
            from sglang.srt.speculative.ragged_verify import (
                RaggedVerifyMode,
                read_ragged_verify_mode,
            )

View on GitHub (pinned to 0132848349)

Solutions

Run DSpark only on CUDA or NPU hardware
Switch to a speculative algorithm supported on your device
Check SGLANG_DEVICE / device override env vars are not forcing an unsupported device string

Example fix

# before (on ROCm)
--speculative-algorithm DSPARK
# after
--speculative-algorithm EAGLE  # or run on a CUDA/NPU host

Defensive patterns

Strategy: validation

Validate before calling

if not (args.device.startswith('cuda') or args.device.startswith('npu')):
    raise SystemExit(f'DSPARK unsupported on device {args.device}')

Prevention

Gate speculative-algorithm selection on the detected device in cluster launchers

When it happens

Trigger: Launching with speculative_algorithm=DSPARK on a device string that does not start with 'cuda' or 'npu' (e.g. AMD GPUs where device resolves to 'rocm.hip').

Common situations: Running on AMD/ROCm clusters and reusing DSpark launch configs from NVIDIA machines; CPU-only smoke tests with spec decoding enabled.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/716f03d38cebfdda. Report an issue: GitHub.