sgl-project/sglang · error · ValueError

DSpark speculative decoding only supports CUDA or NPU…

Error message

DSpark speculative decoding only supports CUDA or NPU device.

What it means

Configuration validation error: the DSpark speculative decoding implementation only supports CUDA or NPU devices and is rejected elsewhere.

Solutions

  1. Run DSpark only on CUDA or NPU hardware
  2. Switch to a speculative algorithm supported on your device
  3. Check SGLANG_DEVICE / device override env vars are not forcing an unsupported device string

Example fix

# before (on ROCm)
--speculative-algorithm DSPARK
# after
--speculative-algorithm EAGLE  # or run on a CUDA/NPU host
Defensive patterns

Strategy: validation

Validate before calling

if not (args.device.startswith('cuda') or args.device.startswith('npu')):
    raise SystemExit(f'DSPARK unsupported on device {args.device}')

Prevention

When it happens

Trigger: Launching with speculative_algorithm=DSPARK on a device string that does not start with 'cuda' or 'npu' (e.g. AMD GPUs where device resolves to 'rocm.hip').

Common situations: Running on AMD/ROCm clusters and reusing DSpark launch configs from NVIDIA machines; CPU-only smoke tests with spec decoding enabled.

Related errors


AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/716f03d38cebfdda. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:351

        )
        logger.warning(
            "Mixed chunked prefill is disabled because of using dflash speculative decoding."
        )


def _target_checkpoint_bundles_dspark_draft(server_args: ServerArgs) -> bool:
    from sglang.srt.speculative.dspark_components.dspark_config import (
        checkpoint_bundles_dspark_draft,
    )

    return checkpoint_bundles_dspark_draft(server_args.get_model_config().hf_config)


def _handle_dspark(server_args: ServerArgs) -> None:
    cfg = resolving_view(server_args)
    _is_npu = cfg.device.startswith("npu")
    if not cfg.device.startswith(("cuda", "npu")):
        raise ValueError(
            "DSpark speculative decoding only supports CUDA or NPU device."
        )

    # dp_size==1 with dp_attention is a degenerate flag under DSV4 CP; skip DP-only checks.
    if cfg.enable_dp_attention and cfg.dp_size > 1:
        if not cfg.enable_dp_lm_head:
            raise ValueError("DSpark with dp attention requires --enable-dp-lm-head.")
        if not _is_npu and cfg.moe_a2a_backend not in ("none", "megamoe"):
            raise ValueError(
                "DSpark with dp attention supports moe_a2a_backend 'none' "
                "(built-in TP MoE) or 'megamoe', got "
                f"{cfg.moe_a2a_backend!r}."
            )
        if not _is_npu and cfg.moe_a2a_backend != "none":
            from sglang.srt.speculative.ragged_verify import (
                RaggedVerifyMode,
                read_ragged_verify_mode,
            )

View on GitHub (pinned to 0132848349)