sgl-project/sglang · error · ValueError

Currently DSpark speculative decoding only supports pp_size

Error message

Currently DSpark speculative decoding only supports pp_size == 1.

What it means

DSpark speculative decoding only supports a single pipeline stage. The hook rejects cfg.pp_size != 1 because DSpark's draft/verify coordination is not implemented across pipeline ranks.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:394

                )
        if cfg.attn_cp_size > 1:
            raise ValueError(
                "DSpark with dp attention does not support context parallel "
                f"(attn_cp_size={cfg.attn_cp_size})."
            )
        if (
            not _is_npu
            and cfg.speculative_moe_a2a_backend is not None
            and cfg.speculative_moe_a2a_backend != cfg.moe_a2a_backend
        ):
            raise ValueError(
                "DSpark ignores --speculative-moe-a2a-backend; with dp attention it "
                f"must match the target moe_a2a_backend={cfg.moe_a2a_backend!r} "
                f"(got {cfg.speculative_moe_a2a_backend!r})."
            )

    if cfg.pp_size != 1:
        raise ValueError(
            "Currently DSpark speculative decoding only supports pp_size == 1."
        )

    if cfg.speculative_draft_model_path is None:
        if _target_checkpoint_bundles_dspark_draft(server_args):
            declare_resolution(
                server_args,
                "_handle_dspark",
                speculative_draft_model_path=cfg.model_path,
            )
            declare_resolution(
                server_args,
                "_handle_dspark",
                speculative_draft_model_revision=cfg.revision,
            )
            logger.info(
                "DSpark draft weights are bundled in the target checkpoint; "
                "defaulting --speculative-draft-model-path to --model-path (%s).",

View on GitHub (pinned to 0132848349)

Solutions

Set --pipeline-parallel-size 1
Use TP/EP to reach the desired GPU count instead of PP
Pick a PP-compatible speculative algorithm if PP is mandatory

Example fix

# before
--speculative-algorithm DSPARK --pp-size 2
# after
--speculative-algorithm DSPARK --pp-size 1 --tp-size 8

Defensive patterns

Strategy: validation

Validate before calling

if args.speculative_algorithm == 'DSPARK' and args.pp_size != 1:
    raise SystemExit('DSPARK requires pp_size == 1')

Prevention

Use TP/EP instead of PP for DSpark multi-GPU scaling

When it happens

Trigger: Launching with speculative_algorithm=DSPARK and --pipeline-parallel-size > 1.

Common situations: Multi-node PP deployments of large MoE models; adding DSpark to an existing pp=2/4 launch script.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1c1312322871144e. Report an issue: GitHub.