apache/beam · critical · Exception

Failed to start vLLM server. Process status

Error message

Failed to start vLLM server. Process status: {process_status}. Next time a request is tried, the server will be restarted

What it means

check_connectivity probes the vLLM server process; if the server never becomes reachable and retries are exhausted (retries == 0), it stops the processes and raises an exception noting the server failed to start and will be restarted on the next request.

Solutions

Check the vLLM server process logs for the root startup failure (OOM, model not found)
Increase the retry count passed to start_server(retries=N) for slow model cold starts
Validate the model path/HF id and ensure GPU memory is sufficient for the model
Ensure vLLM and its CUDA dependencies are correctly installed in the worker image

Example fix

// before
server.start_server()  # default retries, slow model load fails
// after
server.start_server(retries=5)  # allow longer cold start

Defensive patterns

Strategy: retry

Validate before calling

# preflight: confirm model loads and GPU memory is available
assert os.path.exists(model_path) or model_id.startswith(('google/', 'meta-'))
assert torch.cuda.mem_get_info()[0] > estimated_model_bytes

Try / catch

try:
    server.start_server()
except Exception as e:
    if 'Failed to start vLLM server' in str(e):
        time.sleep(30)          # backoff for cold start / transient issues
        server.start_server()   # per source, the server is restarted on next try
    else:
        raise

Prevention

Increase start_server(retries=N) for large-model cold starts
Monitor worker GPU memory to avoid OOM during model load
Validate model path / HF model id before launching the pipeline
Check vLLM server logs immediately after this error for the true root cause

When it happens

Trigger: Calling start_server or _async_run_inference when the vLLM server subprocess fails health checks repeatedly — bad model path/HF model id, OOM on GPU, missing model weights, or server taking longer than the retry budget.

Common situations: Wrong model name or local path; GPU out-of-memory when loading large models; cold-start latency exceeding the connectivity retries; vLLM version incompatibility.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1dead2eee421a58d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/inference/vllm_inference.py:353

             self._server_process.poll() is None and
             (self._dynamo_process is None or
              self._dynamo_process.poll() is None) and
             (self._etcd_process is None or self._etcd_process.poll() is None)):
        try:
          models = client.models.list().data
          logging.info('models: %s' % models)
          if len(models) > 0:
            self._server_started = True
            return
        except:  # pylint: disable=bare-except
          pass
        # Sleep while bringing up the process
        time.sleep(5)

      process_status = self._process_status()
      self._stop_processes()
      if retries == 0:
        raise Exception(
            "Failed to start vLLM server. Process status: "
            f"{process_status}. Next time a request is tried, the server "
            "will be restarted")
      else:
        self.start_server(retries - 1)


class VLLMCompletionsModelHandler(ModelHandler[str,
                                               PredictionResult,
                                               _VLLMModelServer]):
  def __init__(
      self,
      model_name: str,
      vllm_server_kwargs: Optional[dict[str, Optional[str]]] = None,
      *,
      use_dynamo: bool = False,
      dynamo_frontend_kwargs: Optional[dict[str, Optional[str]]] = None,
      min_batch_size: Optional[int] = None,

View on GitHub (pinned to 12126d8942)