vllm-project/vllm · error · NotImplementedError

Elastic EP is not compatible with data_parallel_external_lb

Error message

Elastic EP is not compatible with data_parallel_external_lb or data_parallel_hybrid_lb. Elastic EP relies on a single API server and core client to coordinate scale up/down.

What it means

Elastic EP relies on one API server and core client to coordinate scaling, which is incompatible with the external/hybrid data-parallel load balancer modes where request routing is owned outside the engine. This path raises NotImplementedError, marking an unimplemented combination rather than a user typo.

Source

Thrown at vllm/config/parallel.py:852

            self.pipeline_parallel_size
            * self.tensor_parallel_size
            * self.prefill_context_parallel_size
        )

        if self.distributed_executor_backend == "external_launcher":
            logger.info("Using external launcher for distributed inference.")
            self.world_size *= self.data_parallel_size

        if self.enable_elastic_ep:
            if not self.enable_eplb:
                raise ValueError("Elastic EP is only supported with enable_eplb=True.")
            if self.pipeline_parallel_size > 1:
                raise ValueError(
                    "Elastic EP is not supported with pipeline parallelism "
                    f"(pipeline_parallel_size={self.pipeline_parallel_size})."
                )
            if self.data_parallel_external_lb or self.data_parallel_hybrid_lb:
                raise NotImplementedError(
                    "Elastic EP is not compatible with data_parallel_external_lb "
                    "or data_parallel_hybrid_lb. Elastic EP relies on a single API "
                    "server and core client to coordinate scale up/down."
                )
            if self.eplb_config.use_async:
                from vllm.distributed.nixl_utils import is_nixl_available

                if not is_nixl_available():
                    raise ValueError(
                        "Elastic EP with async EPLB requires the NIXL "
                        "package. Either install NIXL or set "
                        "--eplb-config.use_async=false."
                    )

        if self.data_parallel_size > 1 or self.data_parallel_size_local == 0:
            # Data parallel was specified in the engine args.
            if self.distributed_executor_backend == "external_launcher":
                # For external launcher,

View on GitHub (pinned to c794754062)

Solutions

1. Disable the external/hybrid LB flags (run with a single API server routing to DP ranks internally) when using elastic EP.
2. Or disable --enable-elastic-ep and keep the external LB architecture with static EP sizing.
3. Watch vLLM release notes; this combination may be implemented later.

Example fix

# before
vllm serve model --enable-elastic-ep --data-parallel-external-lb
# after
vllm serve model --enable-elastic-ep

def elastic_ep_lb_valid(enable_elastic_ep: bool, ext_lb: bool, hybrid_lb: bool) -> bool:
    return not enable_elastic_ep or not (ext_lb or hybrid_lb)

assert elastic_ep_lb_valid(True, False, False)

When it happens

Trigger: Launching with --enable-elastic-ep together with --data-parallel-external-lb or --data-parallel-hybrid-lb.

Common situations: Production DP deployments fronted by an external LB attempting to add elastic MoE scaling; config templates combining all 'advanced' flags at once.

Related errors

• data_parallel_external_lb can only be set when data_parallel
• external coordinator mode is not implemented yet
• data parallel rank {rank} is not connected to this frontend;
• data parallel size must be at least 1
• data parallel size ({}) exceeds the two-byte engine identity

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/1248d32ec7635451. Report an issue: GitHub.