{"record":{"id":"22107a3412a267ef","repo":"apache/beam","slug":"service-failed-to-start-up-with-error-s","errorCode":null,"errorMessage":"Service failed to start up with error %s","messagePattern":"Service failed to start up with error (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/utils/subprocess_server.py","lineNumber":229,"sourceCode":"            (\"grpc.max_send_message_length\", -1),\n            # Default: 20000ms (20s), increased to 10 minutes for stability\n            (\"grpc.keepalive_timeout_ms\", 600_000),\n            # Default: 2, set to 0 to allow unlimited pings without data\n            (\"grpc.http2.max_pings_without_data\", 0),\n            # Default: False, set to True to allow keepalive pings when no calls\n            (\"grpc.keepalive_permit_without_calls\", True),\n            # Default: 2, set to 0 to allow unlimited ping strikes\n            (\"grpc.http2.max_ping_strikes\", 0),\n            # Default: 0 (disabled), enable socket reuse for better handling\n            (\"grpc.so_reuseport\", 1),\n        ]\n        self._grpc_channel = grpc.insecure_channel(\n            endpoint, options=channel_options)\n        channel_ready = grpc.channel_ready_future(self._grpc_channel)\n        while True:\n          if process is not None and process.poll() is not None:\n            _LOGGER.error(\"Failed to start job service with %s\", process.args)\n            raise RuntimeError(\n                'Service failed to start up with error %s' % process.poll())\n          try:\n            channel_ready.result(timeout=wait_secs)\n            break\n          except (grpc.FutureTimeoutError, grpc.RpcError):\n            wait_secs *= 1.2\n            logging.log(\n                logging.WARNING if wait_secs > 1 else logging.DEBUG,\n                'Waiting for grpc channel to be ready at %s.',\n                endpoint)\n        return self._stub_class(self._grpc_channel)\n      except Exception as e:\n        _LOGGER.warning(\n            \"Error bringing up service (attempt %d of %d): %s\",\n            attempt + 1,\n            max_retries,\n            e)\n        self.stop_force()","sourceCodeStart":211,"sourceCodeEnd":247,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/utils/subprocess_server.py#L211-L247","documentation":"start() launches the Beam Java/Flink job service subprocess and waits for its gRPC channel to become ready. If the process exits (poll() is not None) before the channel is ready, the loop aborts and raises with the process exit code. This means the server died during startup rather than merely being slow.","triggerScenarios":"The spawned java subprocess terminates during startup — bad JVM flags, missing/corrupt jar, port conflicts, OOM kill, or incompatible Java version — detected via process.poll() while polling grpc.channel_ready_future().","commonSituations":"JAVA_HOME pointing to an incompatible JDK; invalid channel options; a crashed or outdated job-server jar; container memory limits killing the JVM; port already in use.","solutions":["Read the surrounding _LOGGER.error('Failed to start job service with %s', process.args) output and the subprocess stdout/stderr to find the actual JVM failure.","Verify the jar exists and matches the Beam version; run it manually with java -jar to see the error.","Check JAVA_HOME / java -version compatibility with the Beam job server.","Free the port or fix gRPC endpoint configuration; check available memory and container limits.","Retry the start — transient resource contention can kill startup."],"exampleFix":"// before\nwith subprocess_server.JavaJarServer.create('flink', jar_path, []) as srv:\n    ...\n// after\nimport subprocess, shutil\nsubprocess.run(['java', '-jar', jar_path, '--help'], check=True)  # surface JVM errors first\nwith subprocess_server.JavaJarServer.create('flink', jar_path, []) as srv:\n    ...","handlingStrategy":"try-catch","validationCode":"import subprocess\nsubprocess.run(['java', '-version'], capture_output=True, check=True)  # JVM sanity check before start()","typeGuard":null,"tryCatchPattern":"try:\n    server.start()\nexcept RuntimeError as e:\n    if 'Service failed to start up' in str(e):\n        inspect_process_stderr_and_retry()\n    else:\n        raise","preventionTips":["Run the job-server jar manually once to surface JVM errors","Match Java version to Beam requirements","Check port availability and memory limits before starting","Retry transient startup failures with backoff"],"tags":["python","subprocess","grpc","startup-failure","apache-beam"],"backgroundTag":"service-startup-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}