apache/beam · error · RuntimeError

Shared Server Process died unexpectedly with exit code

Error message

Shared Server Process died unexpectedly with exit code {exit_code}

What it means

Raised in _create_server when the spawned server process exits with a nonzero exit code before it finished initializing. Unlike errorIndex 4021 (the server wrote a traceback), here the process died without producing an error file, so Beam reports the raw exit code.

Solutions

Check system logs (dmesg/journalctl) for OOM kills and increase memory limits
Identify and fix native-library crashes by importing/exercising the shared object in a standalone subprocess
Check the reported exit code: 137/-9 means SIGKILL (usually OOM), -11 means SIGSEGV (native crash)
Pin/upgrade libraries whose import crashes and ensure the environment matches your tested configuration

Defensive patterns

Strategy: retry

Validate before calling

# smoke-test the shared object creation in isolation
proc = subprocess.Popen([sys.executable, '-c', 'obj = create_shared_object()'])
rc = proc.wait(60)
assert rc == 0, f'init subprocess died with {rc}'

Try / catch

try:
    handle = mps.acquire()
except RuntimeError as e:
    if 'died unexpectedly with exit code' in str(e):
        code = int(e.args[0].rsplit(' ', 1)[-1]) if e.args else -1
        if code in (-9, 137):
            raise MemoryError('Shared server OOM-killed; increase memory') from e
        raise

Prevention

Provision enough memory: exit code -9/137 usually means OOM kill
Test native-library imports in a clean subprocess before deploying
Avoid sys.exit()/crashes in module-level code of the shared object's module
Check dmesg/journalctl for SIGSEGV/SIGKILL of the child when diagnosing

When it happens

Trigger: The shared-object server process is terminated or crashes hard (segfault, OOM-kill, sys.exit, external terminate) during the startup wait loop in _get_manager.

Common situations: OOM killer killing the child; native code (numpy/tensorflow extensions) segfaulting at import; the process being killed by a resource manager or watchdog; exit() called in module-level code of the shared object's module.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a6d9f4a1cc07beee. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/utils/multi_process_shared.py:508

      last_log = start_time
      while True:
        if os.path.exists(address_file):
          break

        if os.path.exists(error_file):
          with open(error_file, 'r') as f:
            error_msg = f.read()
          try:
            os.remove(error_file)
          except OSError:
            pass

          if p.is_alive(): p.terminate()
          raise RuntimeError(f"Shared Server Process crashed:\n{error_msg}")

        if not p.is_alive():
          exit_code = p.exitcode
          raise RuntimeError(
              "Shared Server Process died unexpectedly"
              f" with exit code {exit_code}")

        if time.time() - last_log > 300:
          logging.warning(
              "Still waiting for %s to initialize... %ss elapsed)",
              self._tag,
              int(time.time() - start_time))
          last_log = time.time()

        time.sleep(0.05)

      logging.info('External process successfully started for %s', self._tag)
    else:
      # We need to be able to authenticate with both the manager
      # and the process.
      self._serving_manager = _SingletonRegistrar(
          address=('localhost', 0), authkey=AUTH_KEY)

View on GitHub (pinned to 12126d8942)