vllm-project/vllm · error · ValueError

Fault tolerance requires a single API server process (--api-

Error message

Fault tolerance requires a single API server process (--api-server-count=1), but got {self._api_process_count}. The FT system assumes one AsyncMPClient manages all engines.

What it means

The fault-tolerance (FT) feature assumes exactly one AsyncMPClient owns all engine processes, so ParallelConfig refuses enable_fault_tolerance together with more than one API server process. Setting --api-server-count > 1 with FT enabled fails at config validation time, before any engine starts.

Source

Thrown at vllm/config/parallel.py:461

                    continue
                start_str, end_str = part.split("-", 1)
                if int(start_str) > int(end_str):
                    raise ValueError(
                        f"numa_bind_cpus ranges must be ascending, but got '{cpuset}'."
                    )
        return value

    @model_validator(mode="after")
    def _validate_parallel_config(self) -> Self:
        if self._api_process_rank >= self._api_process_count:
            raise ValueError(
                "Invalid value of `_api_process_rank`. "
                f"Expected to be `-1` or `[0, {self._api_process_count})`, "
                f"but found: {self._api_process_rank}"
            )

        if self.enable_fault_tolerance and self._api_process_count > 1:
            raise ValueError(
                "Fault tolerance requires a single API server process "
                f"(--api-server-count=1), but got {self._api_process_count}. "
                "The FT system assumes one AsyncMPClient manages all engines."
            )

        if self.all2all_backend in ["pplx", "naive"]:
            logger.warning(
                "The '%s' all2all backend has been removed. "
                "Falling back to 'allgather_reducescatter'.",
                self.all2all_backend,
            )
            self.all2all_backend = "allgather_reducescatter"

        if self.data_parallel_size_local > self.data_parallel_size:
            raise ValueError(
                f"data_parallel_size_local ({self.data_parallel_size_local}) "
                f"must be <= data_parallel_size ({self.data_parallel_size})"
            )

View on GitHub (pinned to c794754062)

Solutions

1. Set --api-server-count=1 (or drop the flag) so FT has its single API server.
2. If HTTP throughput is the concern, put an external reverse proxy/load balancer in front of one API server instead of multiple API processes.
3. If multi-API-server scaling is mandatory, disable --enable-fault-tolerance and rely on an external orchestrator (e.g. Kubernetes) for restarts.

Example fix

# before
vllm serve model --enable-fault-tolerance --api-server-count 2
# after
vllm serve model --enable-fault-tolerance --api-server-count 1

def ft_config_valid(enable_fault_tolerance: bool, api_server_count: int) -> bool:
    return not enable_fault_tolerance or api_server_count == 1

assert ft_config_valid(True, 1)

When it happens

Trigger: vLLM startup with both --enable-fault-tolerance and --api-server-count=2 (or higher).

Common situations: Trying to scale HTTP throughput with multiple API server processes while also wanting FT restarts; enabling FT globally in a shared config template that already sets api-server-count for other deployments.

Related errors

• Invalid value of `_api_process_rank`. Expected to be `-1` or
• numa_bind_cpus ranges must be ascending, but got '{cpuset}'.
• data_parallel_size_local ({self.data_parallel_size_local}) m
• data_parallel_external_lb can only be set when data_parallel
• EPLB requires tensor, prefill-context, or data parallelism,

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/7afca971b11799ae. Report an issue: GitHub.