{"record":{"id":"85843d532474b93c","repo":"apache/beam","slug":"embedded-dynamo-mode-requires-etcd-when-etcd-endpoints-is","errorCode":null,"errorMessage":"Embedded Dynamo mode requires etcd when ETCD_ENDPOINTS is not set. Install etcd in the worker container or set ETCD_ENDPOINTS to an external etcd service.","messagePattern":"Embedded Dynamo mode requires etcd when ETCD_ENDPOINTS is not set\\. Install etcd in the worker container or set ETCD_ENDPOINTS to an external etcd service\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"sdks/python/apache_beam/ml/inference/vllm_inference.py","lineNumber":251,"sourceCode":"      try:\n        with urllib.request.urlopen(health_url, timeout=2) as response:\n          if response.status < 500:\n            return\n      except Exception:  # pylint: disable=broad-except\n        time.sleep(1)\n\n    process_status = self._process_status()\n    self._stop_processes()\n    raise RuntimeError(\n        \"Failed to start embedded etcd for Dynamo. Process status: \"\n        f\"{process_status}. Install etcd in the worker container or set \"\n        \"ETCD_ENDPOINTS to an external etcd service.\")\n\n  def _ensure_etcd(self) -> None:\n    if not self._uses_embedded_etcd():\n      return\n    if shutil.which('etcd') is None:\n      raise RuntimeError(\n          \"Embedded Dynamo mode requires etcd when ETCD_ENDPOINTS is not \"\n          \"set. Install etcd in the worker container or set ETCD_ENDPOINTS \"\n          \"to an external etcd service.\")\n\n    etcd_name = f'beam-dynamo-etcd-{uuid.uuid4().hex}'\n    self._etcd_data_dir = f'/tmp/{etcd_name}'\n    peer_port, = subprocess_server.pick_port(None)\n    etcd_cmd = [\n        'etcd',\n        '--name',\n        etcd_name,\n        '--listen-client-urls',\n        'http://127.0.0.1:{{PORT}}',\n        '--advertise-client-urls',\n        'http://127.0.0.1:{{PORT}}',\n        '--listen-peer-urls',\n        f'http://127.0.0.1:{peer_port}',\n        '--initial-advertise-peer-urls',","sourceCodeStart":233,"sourceCodeEnd":269,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/inference/vllm_inference.py#L233-L269","documentation":"_ensure_etcd checks that when embedded Dynamo mode is used (ETCD_ENDPOINTS not set), an etcd binary is available on PATH. If shutil.which('etcd') finds nothing, it raises RuntimeError telling you to install etcd or configure external endpoints.","triggerScenarios":"Calling start_server with embedded Dynamo mode enabled in a container where the etcd binary is not installed and ETCD_ENDPOINTS is unset.","commonSituations":"Minimal worker images without etcd; relying on the host's etcd not present in the job container; forgetting to set ETCD_ENDPOINTS when etcd is provided externally.","solutions":["Install etcd in the worker container image (apt-get install etcd-server or equivalent)","Set ETCD_ENDPOINTS to a running external etcd service so embedded etcd is not needed","Verify etcd is on the PATH for the worker process (which etcd resolves)"],"exampleFix":"# before\nexport APACHE_BEAM_VLLM_DYNAMO=1  # embedded mode, no etcd\n# after\nexport ETCD_ENDPOINTS=etcd-host:2379  # or install etcd in the image","handlingStrategy":"validation","validationCode":"import os, shutil\nif not os.environ.get('ETCD_ENDPOINTS') and shutil.which('etcd') is None:\n    raise RuntimeError('Install etcd or set ETCD_ENDPOINTS before enabling embedded Dynamo mode')","typeGuard":null,"tryCatchPattern":"try:\n    server.start_server()\nexcept RuntimeError as e:\n    if 'requires etcd' in str(e):\n        os.environ['ETCD_ENDPOINTS'] = 'etcd-host:2379'\n    raise","preventionTips":["Run `which etcd` in the worker image during CI to catch missing binaries","Set ETCD_ENDPOINTS whenever etcd is provided externally","Keep a documented Dockerfile step installing etcd for vLLM workloads"],"tags":["apache-beam","vllm","etcd","dynamo","missing-binary"],"backgroundTag":"missing-dependency","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}