{"record":{"id":"7e118d3dff244176","repo":"apache/seatunnel","slug":"timed-out-waiting-for-python-source-stdout-to-clos","errorCode":null,"errorMessage":"Timed out waiting for Python source stdout to close after the process exited; ensure child processes do not inherit stdout","messagePattern":"Timed out waiting for Python source stdout to close after the process exited; ensure child processes do not inherit stdout","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java","lineNumber":563,"sourceCode":"     * Gives stdout one short drain interval per poll after the direct process exits.\n     *\n     * @return true when stdout reached EOF with no buffered rows, or false when polling must\n     *     continue or engine cancellation won the race\n     */\n    private boolean awaitStdoutPumpAfterProcessExit() throws IOException {\n        if (closeRequested) {\n            return false;\n        }\n        if (stdoutCloseDeadlineNanos == 0L) {\n            stdoutCloseDeadlineNanos =\n                    System.nanoTime() + TimeUnit.SECONDS.toNanos(PROCESS_DESTROY_TIMEOUT_SECONDS);\n        }\n        if (!isAlive(stdoutPumpThread)) {\n            return !closeRequested && stdoutLines.isEmpty();\n        }\n        if (!stdoutLines.isEmpty()) {\n            if (System.nanoTime() >= stdoutCloseDeadlineNanos) {\n                throw inheritedStdoutTimeout();\n            }\n            return false;\n        }\n\n        long remainingNanos = stdoutCloseDeadlineNanos - System.nanoTime();\n        if (remainingNanos <= 0) {\n            throw inheritedStdoutTimeout();\n        }\n        long waitMillis =\n                Math.min(\n                        PROCESS_EXIT_CHECK_TIMEOUT_MILLIS,\n                        Math.max(1L, TimeUnit.NANOSECONDS.toMillis(remainingNanos)));\n        try {\n            stdoutPumpThread.join(waitMillis);\n        } catch (InterruptedException e) {\n            Thread.currentThread().interrupt();\n            throw new IOException(\n                    \"Interrupted while draining Python source stdout after process exit\", e);","sourceCodeStart":545,"sourceCodeEnd":581,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java#L545-L581","documentation":"After the Python process exits, the source waits for the stdout pump thread to drain and close. If stdout is still open past the deadline, a child process inherited the stdout pipe, keeping it open forever; the reader throws this timeout error instead of hanging.","triggerScenarios":"Python source process spawns child processes (workers, subprocess.Popen without stdout redirect) that inherit the parent's stdout; stdout pump thread never sees EOF before stdoutCloseDeadlineNanos elapses.","commonSituations":"Python scripts using multiprocessing/Dask/Spark-like children; invoking shell commands from the Python source that inherit descriptors; long-lived daemons forked by the script.","solutions":["Fix the Python script so child processes redirect/close stdout/stderr (subprocess.Popen(..., stdout=DEVNULL) or pass close_fds=True)","Ensure spawned workers use their own pipes/queues rather than the parent's stdout protocol channel","As a workaround, suppress or detach children (daemonize with os.setsid) so stdout EOF arrives when the main process exits"],"exampleFix":"# before: children inherit stdout, blocking EOF\ncmd = [\"worker\"]\nsubprocess.Popen(cmd)\n# after\nsubprocess.Popen(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, close_fds=True)","handlingStrategy":"try-catch","validationCode":"# audit your Python source for child processes that inherit stdout\nimport subprocess, sys\nsubprocess.Popen(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, close_fds=True)","typeGuard":null,"tryCatchPattern":"try {\n    reader.awaitTermination();\n} catch (Exception e) {\n    if (e.getMessage().contains(\"stdout to close\")) {\n        // fix the Python script's child process stdio handling and rerun\n    }\n    throw e;\n}","preventionTips":["Never let child processes inherit the stdout used by the SeaTunnel protocol","Redirect children's stdout/stderr to DEVNULL, files, or pipes","Avoid forked daemons in Python source scripts; daemonize with new session fds","Test scripts under multiprocessing to confirm stdout EOF arrives at exit"],"tags":["python","subprocess","stdout","timeout"],"backgroundTag":"request-timeout","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}