{"record":{"id":"7afca971b11799ae","repo":"vllm-project/vllm","slug":"fault-tolerance-requires-a-single-api-server-proce","errorCode":null,"errorMessage":"Fault tolerance requires a single API server process (--api-server-count=1), but got {self._api_process_count}. The FT system assumes one AsyncMPClient manages all engines.","messagePattern":"Fault tolerance requires a single API server process \\(--api-server-count=1\\), but got (.+?)\\. The FT system assumes one AsyncMPClient manages all engines\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/parallel.py","lineNumber":461,"sourceCode":"                    continue\n                start_str, end_str = part.split(\"-\", 1)\n                if int(start_str) > int(end_str):\n                    raise ValueError(\n                        f\"numa_bind_cpus ranges must be ascending, but got '{cpuset}'.\"\n                    )\n        return value\n\n    @model_validator(mode=\"after\")\n    def _validate_parallel_config(self) -> Self:\n        if self._api_process_rank >= self._api_process_count:\n            raise ValueError(\n                \"Invalid value of `_api_process_rank`. \"\n                f\"Expected to be `-1` or `[0, {self._api_process_count})`, \"\n                f\"but found: {self._api_process_rank}\"\n            )\n\n        if self.enable_fault_tolerance and self._api_process_count > 1:\n            raise ValueError(\n                \"Fault tolerance requires a single API server process \"\n                f\"(--api-server-count=1), but got {self._api_process_count}. \"\n                \"The FT system assumes one AsyncMPClient manages all engines.\"\n            )\n\n        if self.all2all_backend in [\"pplx\", \"naive\"]:\n            logger.warning(\n                \"The '%s' all2all backend has been removed. \"\n                \"Falling back to 'allgather_reducescatter'.\",\n                self.all2all_backend,\n            )\n            self.all2all_backend = \"allgather_reducescatter\"\n\n        if self.data_parallel_size_local > self.data_parallel_size:\n            raise ValueError(\n                f\"data_parallel_size_local ({self.data_parallel_size_local}) \"\n                f\"must be <= data_parallel_size ({self.data_parallel_size})\"\n            )","sourceCodeStart":443,"sourceCodeEnd":479,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/parallel.py#L443-L479","documentation":"The fault-tolerance (FT) feature assumes exactly one AsyncMPClient owns all engine processes, so ParallelConfig refuses enable_fault_tolerance together with more than one API server process. Setting --api-server-count > 1 with FT enabled fails at config validation time, before any engine starts.","triggerScenarios":"vLLM startup with both --enable-fault-tolerance and --api-server-count=2 (or higher).","commonSituations":"Trying to scale HTTP throughput with multiple API server processes while also wanting FT restarts; enabling FT globally in a shared config template that already sets api-server-count for other deployments.","solutions":["Set --api-server-count=1 (or drop the flag) so FT has its single API server.","If HTTP throughput is the concern, put an external reverse proxy/load balancer in front of one API server instead of multiple API processes.","If multi-API-server scaling is mandatory, disable --enable-fault-tolerance and rely on an external orchestrator (e.g. Kubernetes) for restarts."],"exampleFix":"# before\nvllm serve model --enable-fault-tolerance --api-server-count 2\n# after\nvllm serve model --enable-fault-tolerance --api-server-count 1","handlingStrategy":"validation","validationCode":"def ft_config_valid(enable_fault_tolerance: bool, api_server_count: int) -> bool:\n    return not enable_fault_tolerance or api_server_count == 1\n\nassert ft_config_valid(True, 1)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep FT deployments on a dedicated config with --api-server-count pinned to 1.","Lint shared launch templates for mutually exclusive flags before rollout."],"tags":["fault-tolerance","api-server","configuration","parallelism"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}