sgl-project/sglang · error · ValueError

DeepEP v2 MoE is not validated as a speculative draft backen

Error message

DeepEP v2 MoE is not validated as a speculative draft backend. Select another --speculative-moe-a2a-backend.

What it means

Raised when the resolved speculative draft path would use moe_a2a_backend == 'deepep_v2'. DeepEP v2 has not been validated as a speculative draft backend, so draft models cannot run on it. The check resolves the SpeculativeAlgorithm; non-ngram algorithms inherit the target's moe_a2a_backend as the draft backend, which trips the guard.

Source

Thrown at python/sglang/srt/server_args.py:7632

            raise ValueError(
                f"DeepEP v2 MoE is not validated for {architecture!r}; supported "
                f"architectures are {sorted(validated_architectures)}. "
                "Other model workflows may require an all-reduce after A2A "
                "combine. Use --moe-a2a-backend deepep."
            )

    def _validate_deepep_v2_speculative_draft(self) -> None:
        """Reject an explicit or inherited DeepEP v2 draft backend."""
        view = resolved_view(self)
        draft_backend = view.speculative_moe_a2a_backend
        if draft_backend is None and view.speculative_algorithm:
            from sglang.srt.speculative.spec_info import SpeculativeAlgorithm

            algorithm = SpeculativeAlgorithm.from_string(view.speculative_algorithm)
            if not algorithm.is_ngram():
                draft_backend = view.moe_a2a_backend
        if draft_backend == "deepep_v2":
            raise ValueError(
                "DeepEP v2 MoE is not validated as a speculative draft backend. "
                "Select another --speculative-moe-a2a-backend."
            )

    def _handle_a2a_moe(self):
        # The backend overrides and the ep_size=tp_size adjustments moved to
        # the resolution pipeline (arg_groups/overrides.py:
        # _a2a_backend_overrides / _a2a_ep_size); the per-backend logs,
        # asserts, fusion/deepep_mode/env/cuda-graph writes stay below.
        cfg = resolving_view(self)
        from sglang.srt.arg_groups.overrides import (
            _a2a_backend_overrides,
            _a2a_ep_size,
            _a2a_fusion_adjustments,
            run_post_process_pass,
        )

        run_post_process_pass(self, _a2a_backend_overrides)

View on GitHub (pinned to 0132848349)

Solutions

Set --speculative-moe-a2a-backend deepep (or another supported backend) explicitly for the draft
Use --speculative-algorithm NGRAM, which does not use a draft MoE backend
Drop --moe-a2a-backend deepep_v2 and run the whole stack on deepep

Example fix

# before
--speculative-algorithm EAGLE --moe-a2a-backend deepep_v2
# after
--speculative-algorithm EAGLE --moe-a2a-backend deepep_v2 --speculative-moe-a2a-backend deepep

Defensive patterns

Strategy: validation

Validate before calling

if spec_algorithm not in (None, "NGRAM") and a2a_backend == "deepep_v2":
    draft_a2a = draft_a2a or "deepep"
    cmd += ["--speculative-moe-a2a-backend", draft_a2a]

Prevention

Always set --speculative-moe-a2a-backend explicitly when enabling speculative decoding on MoE models
Prefer NGRAM if you only need lightweight speculation on deepep_v2

When it happens

Trigger: Enabling --speculative-algorithm EAGLE (or any non-ngram algorithm) together with --moe-a2a-backend deepep_v2 without setting a separate --speculative-moe-a2a-backend; the draft inherits deepep_v2 from the target model.

Common situations: Turning on speculative decoding on a DeepSeek/Qwen3 MoE deployment that was upgraded to deepep_v2 for throughput; inheriting the a2a backend implicitly instead of setting a draft-specific one.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3fe9ad44079a5344. Report an issue: GitHub.