{"record":{"id":"ebe78607ec94089e","repo":"apache/beam","slug":"worker-subprocess-exited-with-return-code-s","errorCode":null,"errorMessage":"Worker subprocess exited with return code %s","messagePattern":"Worker subprocess exited with return code (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/portability/local_job_service.py","lineNumber":224,"sourceCode":"        endpoints_pb2.ApiServiceDescriptor(url=self._control_address))\n    pipeline_options = json_format.MessageToJson(\n        self._provision_info.provision_info.pipeline_options)\n\n    env_dict = dict(\n        os.environ,\n        CONTROL_API_SERVICE_DESCRIPTOR=control_descriptor,\n        LOGGING_API_SERVICE_DESCRIPTOR=logging_descriptor,\n        PIPELINE_OPTIONS=pipeline_options)\n    # only add worker_id when it is set.\n    if self._worker_id:\n      env_dict['WORKER_ID'] = self._worker_id\n\n    with worker_handlers.SUBPROCESS_LOCK:\n      p = subprocess.Popen(self._worker_command_line, shell=True, env=env_dict)\n    try:\n      p.wait()\n      if p.returncode:\n        raise RuntimeError(\n            'Worker subprocess exited with return code %s' % p.returncode)\n    finally:\n      if p.poll() is None:\n        p.kill()\n      logging_server.stop(0)\n\n\nclass BeamJob(abstract_job_service.AbstractBeamJob):\n  \"\"\"This class handles running and managing a single pipeline.\n\n    The current state of the pipeline is available as self.state.\n    \"\"\"\n  def __init__(\n      self,\n      job_id: str,\n      pipeline,\n      options,\n      provision_info: fn_runner.ExtendedProvisionInfo,","sourceCodeStart":206,"sourceCodeEnd":242,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/portability/local_job_service.py#L206-L242","documentation":"The WorkerHandler used by the embedded local runner launches the SDK harness as a shell subprocess and blocks on p.wait(). If the process exits with a nonzero return code, the runner raises RuntimeError reporting that code, since the worker died abnormally instead of completing its work.","triggerScenarios":"The SDK harness subprocess started by run() crashes: unhandled exception in user code at harness startup, import errors, OOM kill (return code -9/137), or bad worker command line/environment.","commonSituations":"Missing dependencies in the worker environment; container/OS OOM killer terminating the harness; python version mismatch; failing --worker_command_line overrides; exceptions during pipeline deserialization (pickled DoFn unavailable on worker).","solutions":["Check the logging server output above the error for the harness's actual traceback.","Install missing dependencies in the worker environment (or bundle with --worker_options / requirements).","Fix user code that raises at import/Deserialization time on the worker (use cloudpickle-compatible, importable code).","Investigate return codes: 137/-9 indicates OOM — increase memory or reduce parallelism."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"env_ok = all(dep_importable(d) for d in required_deps)","typeGuard":null,"tryCatchPattern":"try:\n    handler.run(worker_command)\nexcept RuntimeError as e:\n    rc = int(str(e).rsplit(' ', 1)[-1])\n    if abs(rc) in (9, 137): investigate_oom()\n    else: inspect_harness_logs()","preventionTips":["Ensure worker env has all pipeline dependencies","Watch for return codes 137/-9 (OOM)","Keep worker python version compatible with the pipeline","Test pickled DoFns import cleanly on workers"],"tags":["subprocess","worker-lifecycle","crash","beam"],"backgroundTag":"process-exited-nonzero","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}