Python source process did not terminate after forced…

Error message

Python source process did not terminate after forced shutdown

What it means

During close(), after the Python process fails to terminate gracefully (interrupt signal), the reader calls destroyForcibly() and waits. If the process is still alive after the forced-shutdown wait times out, this warning is logged — the OS-level process could not be killed within the timeout window.

Solutions

Investigate the Python script for uninterruptible operations (D-state I/O, native calls)
Kill the process manually at the OS level (kill -9 <pid>) and check for zombie/descendant processes
Increase the destroy timeout on slow hosts
Avoid Python scripts that fork detached children which survive cancellation

Defensive patterns

Strategy: fallback

Validate before calling

// verify the script handles SIGTERM and exits within a few seconds before deploying
// timeout 5s python3 script.py < /dev/null && echo exits-cleanly

Try / catch

try { reader.close(); } catch (IOException e) { /* surfaced close failure */ log.warn("python process cleanup incomplete: {}", e.getMessage()); }

Prevention

Avoid native/blocking calls that ignore signals
Kill orphaned processes after jobs (process supervisor)
Test script termination behavior under load
Increase timeouts on slow/loaded hosts

When it happens

Trigger: close() -> destroyProcess(): runningProcess.destroy() then destroyForcibly() both fail to end the process within timeoutMillis, detected via waitFor returning false.

Common situations: Python process stuck in uninterruptible I/O or ignoring SIGTERM/SIGKILL delivery windows; D-state processes on overloaded hosts; detached grandchildren keeping the session busy.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/25c104673900f8c1. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java:753

    private static void destroyProcess(Process runningProcess, long timeoutMillis) {
        if (!runningProcess.isAlive()) {
            return;
        }

        boolean interrupted = false;
        runningProcess.destroy();
        try {
            if (!runningProcess.waitFor(timeoutMillis, TimeUnit.MILLISECONDS)) {
                runningProcess.destroyForcibly();
            }
        } catch (InterruptedException e) {
            interrupted = true;
            runningProcess.destroyForcibly();
        }
        if (runningProcess.isAlive()) {
            try {
                if (!runningProcess.waitFor(timeoutMillis, TimeUnit.MILLISECONDS)) {
                    LOG.warn("Python source process did not terminate after forced shutdown");
                }
            } catch (InterruptedException e) {
                interrupted = true;
            }
        }
        if (interrupted) {
            Thread.currentThread().interrupt();
        }
    }

    /**
     * Closes all Java pipe endpoints so blocked reader and writer threads can terminate.
     *
     * <p>Bounded on purpose. When a child process inherited stdout, the pipe stays open on the
     * child's side and closing the read end blocks behind the pump thread's pending read until that
     * child exits. On Windows that wait is not interruptible by the close itself, so performing it
     * inline would let cancellation exceed the reader's bounded shutdown window. The close is
     * handed to a daemon thread instead: the process has already been destroyed by this point, so

View on GitHub (pinned to cf67b549a7)