{"record":{"id":"dadc16b3a25132c6","repo":"apache/seatunnel","slug":"suppressing-further-python-source-stderr-lines-for","errorCode":null,"errorMessage":"Suppressing further Python source stderr lines for this process","messagePattern":"Suppressing further Python source stderr lines for this process","errorType":"console","errorClass":null,"httpStatus":null,"severity":"info","filePath":"seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java","lineNumber":402,"sourceCode":"    }\n\n    private void startStderrPump() {\n        stderrPumpThread =\n                new Thread(\n                        () -> {\n                            try (BufferedReader stderrReader =\n                                    new BufferedReader(\n                                            new InputStreamReader(\n                                                    process.getErrorStream(),\n                                                    StandardCharsets.UTF_8))) {\n                                String line;\n                                int loggedLines = 0;\n                                while ((line = stderrReader.readLine()) != null) {\n                                    appendStderrLine(line);\n                                    if (loggedLines < STDERR_HISTORY_LIMIT) {\n                                        LOG.warn(\"Python source stderr: {}\", line);\n                                    } else if (loggedLines == STDERR_HISTORY_LIMIT) {\n                                        LOG.warn(\n                                                \"Suppressing further Python source stderr lines for this process\");\n                                    }\n                                    loggedLines++;\n                                }\n                            } catch (IOException e) {\n                                if (!closeRequested && !stderrShutdownRequested) {\n                                    stderrPumpFailure = e;\n                                }\n                            }\n                        },\n                        \"python-source-stderr-pump\");\n        stderrPumpThread.setDaemon(true);\n        stderrPumpThread.start();\n    }\n\n    private void startStdoutPump() {\n        stdoutPumpThread =\n                new Thread(","sourceCodeStart":384,"sourceCodeEnd":420,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java#L384-L420","documentation":"PythonSourceReader caps stderr logging at STDERR_HISTORY_LIMIT lines per process; once the limit is hit it logs this single suppression notice and keeps consuming (and buffering) further stderr lines without logging them. This prevents log flooding from chatty Python processes while preserving the tail history for error messages.","triggerScenarios":"open()'s stderr pump reads more than STDERR_HISTORY_LIMIT lines from the Python process's stderr within one process lifetime — the line at index == limit triggers this one-time message.","commonSituations":"Verbose Python libraries logging everything to stderr; a script with a loop printing diagnostics; a misbehaving dependency emitting continuous warnings; a stuck Python process spamming errors repeatedly.","solutions":["Suppress the noise at its source: configure the Python script/libraries to log less or to a file instead of stderr.","Fix the underlying Python issue causing repeated stderr output (trace the first logged lines, which precede this message).","Increase STDERR_HISTORY_LIMIT in PythonSourceReader if full stderr capture is genuinely required for debugging.","Rely on appendStderrLine history, which is included when the reader raises its terminal error, rather than the per-line logs."],"exampleFix":"// before (Python script spamming stderr)\nfor i in range(100000): print('debug', i, file=sys.stderr)\n// after\nimport logging; logging.basicConfig(filename='app.log'); logging.debug('debug %d', i)","handlingStrategy":"validation","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep Python scripts quiet on stderr","Configure library log levels/destinations inside the script","Check the first few stderr lines for the real problem; later lines are suppressed"],"tags":["python","stderr","log-flood","throttling"],"backgroundTag":"log-flood-suppression","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}