{"record":{"id":"1dead2eee421a58d","repo":"apache/beam","slug":"failed-to-start-vllm-server-process-status-process-status","errorCode":null,"errorMessage":"Failed to start vLLM server. Process status: {process_status}. Next time a request is tried, the server will be restarted","messagePattern":"Failed to start vLLM server\\. Process status: (.+?)\\. Next time a request is tried, the server will be restarted","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"critical","filePath":"sdks/python/apache_beam/ml/inference/vllm_inference.py","lineNumber":353,"sourceCode":"             self._server_process.poll() is None and\n             (self._dynamo_process is None or\n              self._dynamo_process.poll() is None) and\n             (self._etcd_process is None or self._etcd_process.poll() is None)):\n        try:\n          models = client.models.list().data\n          logging.info('models: %s' % models)\n          if len(models) > 0:\n            self._server_started = True\n            return\n        except:  # pylint: disable=bare-except\n          pass\n        # Sleep while bringing up the process\n        time.sleep(5)\n\n      process_status = self._process_status()\n      self._stop_processes()\n      if retries == 0:\n        raise Exception(\n            \"Failed to start vLLM server. Process status: \"\n            f\"{process_status}. Next time a request is tried, the server \"\n            \"will be restarted\")\n      else:\n        self.start_server(retries - 1)\n\n\nclass VLLMCompletionsModelHandler(ModelHandler[str,\n                                               PredictionResult,\n                                               _VLLMModelServer]):\n  def __init__(\n      self,\n      model_name: str,\n      vllm_server_kwargs: Optional[dict[str, Optional[str]]] = None,\n      *,\n      use_dynamo: bool = False,\n      dynamo_frontend_kwargs: Optional[dict[str, Optional[str]]] = None,\n      min_batch_size: Optional[int] = None,","sourceCodeStart":335,"sourceCodeEnd":371,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/inference/vllm_inference.py#L335-L371","documentation":"check_connectivity probes the vLLM server process; if the server never becomes reachable and retries are exhausted (retries == 0), it stops the processes and raises an exception noting the server failed to start and will be restarted on the next request.","triggerScenarios":"Calling start_server or _async_run_inference when the vLLM server subprocess fails health checks repeatedly — bad model path/HF model id, OOM on GPU, missing model weights, or server taking longer than the retry budget.","commonSituations":"Wrong model name or local path; GPU out-of-memory when loading large models; cold-start latency exceeding the connectivity retries; vLLM version incompatibility.","solutions":["Check the vLLM server process logs for the root startup failure (OOM, model not found)","Increase the retry count passed to start_server(retries=N) for slow model cold starts","Validate the model path/HF id and ensure GPU memory is sufficient for the model","Ensure vLLM and its CUDA dependencies are correctly installed in the worker image"],"exampleFix":"// before\nserver.start_server()  # default retries, slow model load fails\n// after\nserver.start_server(retries=5)  # allow longer cold start","handlingStrategy":"retry","validationCode":"# preflight: confirm model loads and GPU memory is available\nassert os.path.exists(model_path) or model_id.startswith(('google/', 'meta-'))\nassert torch.cuda.mem_get_info()[0] > estimated_model_bytes","typeGuard":null,"tryCatchPattern":"try:\n    server.start_server()\nexcept Exception as e:\n    if 'Failed to start vLLM server' in str(e):\n        time.sleep(30)          # backoff for cold start / transient issues\n        server.start_server()   # per source, the server is restarted on next try\n    else:\n        raise","preventionTips":["Increase start_server(retries=N) for large-model cold starts","Monitor worker GPU memory to avoid OOM during model load","Validate model path / HF model id before launching the pipeline","Check vLLM server logs immediately after this error for the true root cause"],"tags":["apache-beam","vllm","server-startup","gpu"],"backgroundTag":"connection-refused","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}