{"record":{"id":"27fffbf098708fa5","repo":"apache/seatunnel","slug":"failed-to-start-python-source-process-with-executa","errorCode":null,"errorMessage":"Failed to start python source process with executable [{}] and script [{}]","messagePattern":"Failed to start python source process with executable \\[(.+?)\\] and script \\[(.+?)\\]","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java","lineNumber":149,"sourceCode":"                PythonSourceExecutionPolicy.resolveExecutable(sourceConfig.getPythonExecutable());\n        LOG.warn(\n                \"Python source runs unsandboxed external code. Resolved executable='{}', scriptOrigin='python.script.path={}', guarded by system properties '{}','{}'.\",\n                resolvedExecutable,\n                scriptPath,\n                PythonSourceExecutionPolicy.PYTHON_SOURCE_ENABLED_PROPERTY,\n                PythonSourceExecutionPolicy.PYTHON_ALLOWED_EXECUTABLES_PROPERTY);\n        ProcessBuilder processBuilder =\n                new ProcessBuilder(resolvedExecutable.toString(), scriptPath.toString());\n        configureWorkingDirectory(processBuilder, scriptPath);\n\n        synchronized (lifecycleLock) {\n            if (closeRequested) {\n                throw new IOException(\"Python source reader has already been closed\");\n            }\n            try {\n                this.process = processBuilder.start();\n            } catch (IOException e) {\n                throw new IOException(\n                        \"Failed to start python source process with executable [\"\n                                + resolvedExecutable\n                                + \"] and script [\"\n                                + scriptPath\n                                + \"]\",\n                        e);\n            }\n\n            try {\n                startStderrPump();\n                startStdoutPump();\n                startStdinWriter(sourceConfig.getPythonScriptConfig());\n                waitForInitialConfigWrite();\n            } catch (Exception e) {\n                try {\n                    close();\n                } catch (IOException closeException) {\n                    e.addSuppressed(closeException);","sourceCodeStart":131,"sourceCodeEnd":167,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java#L131-L167","documentation":"Wraps the IOException from processBuilder.start() with context about the resolved executable and script path. Thrown when the OS refuses to spawn the python process — e.g. fork/exec failure, missing executable at start time, or resource exhaustion.","triggerScenarios":"processBuilder.start() throwing IOException during PythonSourceReader.open(): executable vanished between validation and start, OS-level limits (too many processes/fds), or permission revoked.","commonSituations":"ulimit/max user processes reached on busy workers; temp dir or working directory missing; interpreter deleted mid-run; container memory limits blocking fork.","solutions":["Read the wrapped cause 'e' for the underlying OS reason","Check ulimit -u (nproc) and file descriptor limits on the worker","Confirm the resolved executable and working directory still exist and are accessible to the runtime user","Retry the task once resources are freed"],"exampleFix":"// before\n// cause ignored, only wrapper message seen\n// after\ncatch (IOException e) { LOG.error(\"spawn failed\", e.getCause()); /* inspect cause */ }","handlingStrategy":"try-catch","validationCode":"// preflight: new ProcessBuilder(exe, \"--version\").start().waitFor()","typeGuard":null,"tryCatchPattern":"try {\n    reader.open();\n} catch (java.io.IOException e) {\n    if (e.getMessage().startsWith(\"Failed to start python source process\")) {\n        Throwable cause = e.getCause();\n        // log cause, check ulimits/executable existence, retry after backoff\n    } else throw e;\n}","preventionTips":["Monitor nproc and fd ulimits on workers","Preflight the interpreter with a --version spawn","Keep the interpreter and working directory stable during job lifetime"],"tags":["python","process","os","io"],"backgroundTag":"process-spawn-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}