vllm-project/vllm · error · ValueError

Currently, async scheduling is only supported with EAGLE/MTP

Error message

Currently, async scheduling is only supported with EAGLE/MTP/Draft Model/NGram GPU/DSpark kind of speculative decoding

What it means

vLLM's asynchronous scheduler currently only supports a fixed set of speculative decoding methods: EAGLE family (incl. MTP), NGRAM GPU variants, `draft_model`, and `dspark`. If async scheduling is explicitly enabled together with any other speculative method, config validation rejects it at startup.

Source

Thrown at vllm/config/vllm.py:1170

        if self.scheduler_config.async_scheduling:
            # Async scheduling explicitly enabled, hard fail any incompatibilities.
            # Currently, async scheduling only support eagle speculative
            # decoding.
            if uses_rocm_deepep_ht_dbo:
                raise ValueError(
                    "Async scheduling is not compatible with ROCm DeepEP "
                    "high-throughput DBO. Please use --no-async-scheduling or "
                    "select a different all2all backend."
                )
            if self.speculative_config is not None:
                if (
                    self.speculative_config.method not in get_args(EagleModelTypes)
                    and self.speculative_config.method not in get_args(NgramGPUTypes)
                    and self.speculative_config.method != "draft_model"
                    and self.speculative_config.method != "dspark"
                ):
                    raise ValueError(
                        "Currently, async scheduling is only supported "
                        "with EAGLE/MTP/Draft Model/NGram GPU/DSpark kind of "
                        "speculative decoding"
                    )
                if self.speculative_config.disable_padded_drafter_batch:
                    raise ValueError(
                        "Async scheduling is not compatible with "
                        "disable_padded_drafter_batch=True."
                    )
            if not executor_supports_async_sched:
                raise ValueError(
                    f"`{executor_backend}` does not support async scheduling yet."
                )
        elif self.scheduler_config.async_scheduling is None:
            # Enable async scheduling unless there is an incompatible option.
            if (
                self.model_config is not None
                and self.model_config.runner_type == "pooling"

View on GitHub (pinned to c794754062)

Solutions

Switch to a supported speculative method (e.g. `--speculative-config '{"method":"eagle",...}'`, `ngram` GPU variant, `draft_model`, or `dspark`).
Or disable async scheduling with `--no-async-scheduling` to keep the chosen spec-decode method.
Check the installed vLLM version's supported list via `vllm.speculative.speculative_config` (EagleModelTypes/NgramGPUTypes) before configuring.

Example fix

# before
vllm serve meta-llama/Llama-3.1-8B-Instruct \
  --async-scheduling \
  --speculative-config '{"method": "med", "num_speculative_tokens": 4}'

# after
vllm serve meta-llama/Llama-3.1-8B-Instruct \
  --async-scheduling \
  --speculative-config '{"method": "eagle", "model": ".../Llama-3.1-8B-Instruct-eagle"}'

Defensive patterns

Strategy: validation

Validate before calling

from typing import get_args
from vllm.speculative import EagleModelTypes, NgramGPUTypes  # names per installed version
ok = {"draft_model", "dspark", *get_args(EagleModelTypes), *get_args(NgramGPUTypes)}
if async_scheduling and spec_method not in ok:
    async_scheduling = False

Try / catch

try:
    LLM(**args)
except ValueError as e:
    if "only supported with EAGLE/MTP" in str(e):
        args["async_scheduling"] = False  # retry once
    else:
        raise

Prevention

Pin the accepted spec-decode method list per vLLM version
Pre-validate speculative_config against the async-scheduling allowlist

When it happens

Trigger: Launching with `--async-scheduling` plus a speculative config whose method is outside EagleModelTypes/NgramGPUTypes/draft_model/dspark (e.g. `med`, `ngram` CPU variant, or other lookahead methods).

Common situations: Users experimenting with newer or CPU-side spec-decode algorithms while keeping async scheduling on; upgrading vLLM where the accepted method list changed.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/388c35e8fd9944be. Report an issue: GitHub.