apache/seatunnel · error · IOException

Failed to start python source process with executable [{}] a

Error message

Failed to start python source process with executable [{}] and script [{}]

What it means

Wraps the IOException from processBuilder.start() with context about the resolved executable and script path. Thrown when the OS refuses to spawn the python process — e.g. fork/exec failure, missing executable at start time, or resource exhaustion.

Source

Thrown at seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java:149

                PythonSourceExecutionPolicy.resolveExecutable(sourceConfig.getPythonExecutable());
        LOG.warn(
                "Python source runs unsandboxed external code. Resolved executable='{}', scriptOrigin='python.script.path={}', guarded by system properties '{}','{}'.",
                resolvedExecutable,
                scriptPath,
                PythonSourceExecutionPolicy.PYTHON_SOURCE_ENABLED_PROPERTY,
                PythonSourceExecutionPolicy.PYTHON_ALLOWED_EXECUTABLES_PROPERTY);
        ProcessBuilder processBuilder =
                new ProcessBuilder(resolvedExecutable.toString(), scriptPath.toString());
        configureWorkingDirectory(processBuilder, scriptPath);

        synchronized (lifecycleLock) {
            if (closeRequested) {
                throw new IOException("Python source reader has already been closed");
            }
            try {
                this.process = processBuilder.start();
            } catch (IOException e) {
                throw new IOException(
                        "Failed to start python source process with executable ["
                                + resolvedExecutable
                                + "] and script ["
                                + scriptPath
                                + "]",
                        e);
            }

            try {
                startStderrPump();
                startStdoutPump();
                startStdinWriter(sourceConfig.getPythonScriptConfig());
                waitForInitialConfigWrite();
            } catch (Exception e) {
                try {
                    close();
                } catch (IOException closeException) {
                    e.addSuppressed(closeException);

View on GitHub (pinned to cf67b549a7)

Solutions

1. Read the wrapped cause 'e' for the underlying OS reason
2. Check ulimit -u (nproc) and file descriptor limits on the worker
3. Confirm the resolved executable and working directory still exist and are accessible to the runtime user
4. Retry the task once resources are freed

Example fix

// before
// cause ignored, only wrapper message seen
// after
catch (IOException e) { LOG.error("spawn failed", e.getCause()); /* inspect cause */ }

// preflight: new ProcessBuilder(exe, "--version").start().waitFor()

try {
    reader.open();
} catch (java.io.IOException e) {
    if (e.getMessage().startsWith("Failed to start python source process")) {
        Throwable cause = e.getCause();
        // log cause, check ulimits/executable existence, retry after backoff
    } else throw e;
}

When it happens

Trigger: processBuilder.start() throwing IOException during PythonSourceReader.open(): executable vanished between validation and start, OS-level limits (too many processes/fds), or permission revoked.

Common situations: ulimit/max user processes reached on busy workers; temp dir or working directory missing; interpreter deleted mid-run; container memory limits blocking fork.

Related errors

• Timed out after {} seconds while writing python.script.confi
• Python script exited before reading its first stdin line; py
• Failed to write python.script.config
• Python source process exited with code {}{}
• Python source stderr: {}

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/27fffbf098708fa5. Report an issue: GitHub.