vllm-project/vllm · error · ValueError

{self.communicator} communicator is incompatible with async

Error message

{self.communicator} communicator is incompatible with async EPLB due to NCCL multi-stream conflicts. Use 'torch_gloo' or 'nixl' instead, or leave communicator unset for automatic selection.

What it means

EPLBConfig's validator forbids use_async=True with communicator 'torch_nccl' or 'pynccl'. Both communicators issue NCCL operations on the device stream, and async EPLB moves expert weights while compute streams are active, causing NCCL multi-stream conflicts (deadlocks/corruption). Use 'torch_gloo' or 'nixl', or leave the communicator unset for auto-selection (which prefers nixl, falling back to torch_gloo).

Source

Thrown at vllm/config/parallel.py:107

    policy: EPLBPolicyOption = "default"
    """The policy type for expert parallel load balancing (EPLB)."""

    communicator: EPLBCommunicatorBackend | None = None
    """
    Backend for EPLB expert weight communication:
    - "torch_nccl": Use torch.distributed on the device process group
    - "torch_gloo": Use torch.distributed gloo with CPU staging
    - "nixl": Use NIXL with staged send/recv buffers
    - "pynccl": Use PyNccl send/recv
    - None: Auto-select backend (prefers "nixl", falls back to "torch_gloo")
    """

    @model_validator(mode="after")
    def _validate_eplb_config(self) -> Self:
        if self.use_async and self.policy != "default":
            raise ValueError("Async EPLB is only supported with the default policy.")
        if self.use_async and self.communicator in ("torch_nccl", "pynccl"):
            raise ValueError(
                f"{self.communicator} communicator is incompatible with "
                "async EPLB due to NCCL multi-stream conflicts. Use "
                "'torch_gloo' or 'nixl' instead, or leave communicator "
                "unset for automatic selection."
            )
        if self.log_balancedness and self.log_balancedness_interval <= 0:
            raise ValueError("log_balancedness_interval must be greater than 0.")
        return self


@config
class ParallelConfig:
    """Configuration for the distributed execution."""

    pipeline_parallel_size: int = Field(default=1, ge=1)
    """Number of pipeline parallel groups."""
    tensor_parallel_size: int = Field(default=1, ge=1)
    """Number of tensor parallel groups."""

View on GitHub (pinned to c794754062)

Solutions

1. Remove --eplb-communicator so vLLM auto-selects (nixl preferred, torch_gloo fallback).
2. Or set --eplb-communicator torch_gloo or --eplb-communicator nixl explicitly.
3. If you truly need torch_nccl/pynccl, disable --async-eplb.

Example fix

# before
vllm serve DeepSeek-ai/DeepSeek-V3 --enable-eplb --async-eplb --eplb-communicator torch_nccl

# after
vllm serve DeepSeek-ai/DeepSeek-V3 --enable-eplb --async-eplb --eplb-communicator nixl

def check_eplb_communicator(use_async: bool, communicator: str | None) -> None:
    if use_async and communicator in ("torch_nccl", "pynccl"):
        raise SystemExit("Use torch_gloo/nixl (or unset) with async EPLB")

When it happens

Trigger: Passing --enable-eplb --async-eplb --eplb-communicator torch_nccl (or pynccl); or a config file where communicator was pinned for the sync path and --async-eplb was added later.

Common situations: Reusing a working sync-EPLB config when enabling async EPLB; explicitly choosing torch_nccl believing it is the highest-performance option.

Related errors

• Async EPLB is only supported with the default policy.
• log_balancedness_interval must be greater than 0.
• Image generation should not fail
• cannot continue the final message when the last message is n
• multimodal preprocessing error: {0}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/290c572c51aa73ca. Report an issue: GitHub.