{"record":{"id":"4a8f32e231c7f543","repo":"apache/beam","slug":"shared-server-process-crashed-error-msg","errorCode":null,"errorMessage":"Shared Server Process crashed:\n{error_msg}","messagePattern":"Shared Server Process crashed:\n(.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/utils/multi_process_shared.py","lineNumber":504,"sourceCode":"\n      atexit.register(cleanup_process)\n\n      start_time = time.time()\n      last_log = start_time\n      while True:\n        if os.path.exists(address_file):\n          break\n\n        if os.path.exists(error_file):\n          with open(error_file, 'r') as f:\n            error_msg = f.read()\n          try:\n            os.remove(error_file)\n          except OSError:\n            pass\n\n          if p.is_alive(): p.terminate()\n          raise RuntimeError(f\"Shared Server Process crashed:\\n{error_msg}\")\n\n        if not p.is_alive():\n          exit_code = p.exitcode\n          raise RuntimeError(\n              \"Shared Server Process died unexpectedly\"\n              f\" with exit code {exit_code}\")\n\n        if time.time() - last_log > 300:\n          logging.warning(\n              \"Still waiting for %s to initialize... %ss elapsed)\",\n              self._tag,\n              int(time.time() - start_time))\n          last_log = time.time()\n\n        time.sleep(0.05)\n\n      logging.info('External process successfully started for %s', self._tag)\n    else:","sourceCodeStart":486,"sourceCodeEnd":522,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/utils/multi_process_shared.py#L486-L522","documentation":"Raised in _create_server while _get_manager waits for the newly spawned shared-object server process to initialize. The server process wrote its traceback to an error file before dying, so Beam surfaces that message wrapped in this RuntimeError and removes the temp file.","triggerScenarios":"Starting a shared object via multi_process_shared when the server process's initialization (e.g. the entry function constructing the object, or the module import in the child) raises an exception before it begins serving.","commonSituations":"The callable that creates the shared object raises (bad config, missing resource); a module imported by the child fails; unpicklable/incorrect initialization args; missing dependencies in the worker environment (esp. on Dataflow/containers).","solutions":["Read the embedded error_msg in the exception: fix the underlying exception raised during server initialization","Verify all imports/dependencies needed by the shared object are installed in the runtime environment","Ensure the initialization callable and its arguments are picklable and side-effect free","Test constructing the object in a fresh subprocess locally to reproduce the child's error"],"exampleFix":"// before\nshared = multi_process_shared.MultiProcessShared(MyObj, name='x')  # MyObj ctor reads missing file\n// after\npath = ensure_file_exists()\nshared = multi_process_shared.MultiProcessShared(lambda: MyObj(path), name='x')","handlingStrategy":"try-catch","validationCode":"import importlib, pickle\nmod = importlib.import_module(init_callable.__module__)  # import must not fail\npickle.dumps(init_callable)  # must be picklable\ninit_callable()  # must construct locally without raising","typeGuard":null,"tryCatchPattern":"try:\n    handle = mps.acquire()\nexcept RuntimeError as e:\n    if str(e).startswith('Shared Server Process crashed'):\n        logging.error('Server init failed: %s', e)\n        # fix dependency/config, then retry\n    else:\n        raise","preventionTips":["Verify the shared object's initializer runs cleanly in a plain subprocess","Ensure all dependencies are installed in the runtime/container environment","Keep initialization arguments picklable and side-effect free","Capture and log the embedded error_msg — it contains the child's real traceback"],"tags":["python","multiprocessing","process-crash","initialization"],"backgroundTag":"module-init-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}