Python source stream closer did not finish within

Error message

Python source stream closer did not finish within {}ms; process stream handles may remain open until it completes or the JVM exits

What it means

closeProcessStreams() joins the stream-closer thread with a bounded timeout; if that thread is still alive after the join, this warning is logged. The process's stdout/stderr handles may remain open until the closer finishes or the JVM exits, risking file-descriptor leaks.

Solutions

Ensure the Python process and all children terminate so the pipes close and EOF is reached
Increase the close timeout to let stream draining finish
Check the script for spawned child processes inheriting stdout/stderr
If leaks accumulate, recycle the task/JVM or close handles at OS level

Defensive patterns

Strategy: fallback

Try / catch

try { reader.close(); } catch (IOException e) { /* close failure precedence */ log.warn("python stream close incomplete: {}", e.getMessage()); }

Prevention

Ensure script and children close stdout/stderr on exit
Avoid scripts that fork background processes inheriting pipes
Raise close timeout if scripts emit large buffered output
Watch JVM fd counts (ulimit/lsof) for leaks

When it happens

Trigger: close() -> closeProcessStreams(): streamCloser.join(timeoutMillis) returns and streamCloser.isAlive() is still true — the closer is blocked, typically on reading streams that never reach EOF because the process (or a child holding the pipe) is still running.

Common situations: Python process or its descendants keeping stdout/stderr pipes open after close was requested; large buffered stream output still being drained; blocked read() on a pipe nobody closes.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/4f2cadd8b786d0e4. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java:791

     */
    private static void closeProcessStreams(Process runningProcess, long timeoutMillis) {
        Thread streamCloser =
                new Thread(
                        () -> {
                            closeQuietly(runningProcess.getOutputStream());
                            closeQuietly(runningProcess.getInputStream());
                            closeQuietly(runningProcess.getErrorStream());
                        },
                        "python-source-stream-closer");
        streamCloser.setDaemon(true);
        streamCloser.start();
        try {
            streamCloser.join(timeoutMillis);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        if (streamCloser.isAlive()) {
            LOG.warn(
                    "Python source stream closer did not finish within {}ms; process stream"
                            + " handles may remain open until it completes or the JVM exits",
                    timeoutMillis);
        }
    }

    /** Stream cleanup is best effort because process termination is the authoritative boundary. */
    private static void closeQuietly(Closeable closeable) {
        try {
            closeable.close();
        } catch (IOException e) {
            LOG.debug("Failed to close Python source process stream", e);
        }
    }

    private static boolean isAlive(Thread thread) {
        return thread != null && thread.isAlive();
    }

View on GitHub (pinned to cf67b549a7)