Suppressing further Python source stderr lines for this proc

Error message

Suppressing further Python source stderr lines for this process

What it means

PythonSourceReader caps stderr logging at STDERR_HISTORY_LIMIT lines per process; once the limit is hit it logs this single suppression notice and keeps consuming (and buffering) further stderr lines without logging them. This prevents log flooding from chatty Python processes while preserving the tail history for error messages.

Source

Thrown at seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java:402

    }

    private void startStderrPump() {
        stderrPumpThread =
                new Thread(
                        () -> {
                            try (BufferedReader stderrReader =
                                    new BufferedReader(
                                            new InputStreamReader(
                                                    process.getErrorStream(),
                                                    StandardCharsets.UTF_8))) {
                                String line;
                                int loggedLines = 0;
                                while ((line = stderrReader.readLine()) != null) {
                                    appendStderrLine(line);
                                    if (loggedLines < STDERR_HISTORY_LIMIT) {
                                        LOG.warn("Python source stderr: {}", line);
                                    } else if (loggedLines == STDERR_HISTORY_LIMIT) {
                                        LOG.warn(
                                                "Suppressing further Python source stderr lines for this process");
                                    }
                                    loggedLines++;
                                }
                            } catch (IOException e) {
                                if (!closeRequested && !stderrShutdownRequested) {
                                    stderrPumpFailure = e;
                                }
                            }
                        },
                        "python-source-stderr-pump");
        stderrPumpThread.setDaemon(true);
        stderrPumpThread.start();
    }

    private void startStdoutPump() {
        stdoutPumpThread =
                new Thread(

View on GitHub (pinned to cf67b549a7)

Solutions

Suppress the noise at its source: configure the Python script/libraries to log less or to a file instead of stderr.
Fix the underlying Python issue causing repeated stderr output (trace the first logged lines, which precede this message).
Increase STDERR_HISTORY_LIMIT in PythonSourceReader if full stderr capture is genuinely required for debugging.
Rely on appendStderrLine history, which is included when the reader raises its terminal error, rather than the per-line logs.

Example fix

// before (Python script spamming stderr)
for i in range(100000): print('debug', i, file=sys.stderr)
// after
import logging; logging.basicConfig(filename='app.log'); logging.debug('debug %d', i)

Defensive patterns

Strategy: validation

Prevention

Keep Python scripts quiet on stderr
Configure library log levels/destinations inside the script
Check the first few stderr lines for the real problem; later lines are suppressed

When it happens

Trigger: open()'s stderr pump reads more than STDERR_HISTORY_LIMIT lines from the Python process's stderr within one process lifetime — the line at index == limit triggers this one-time message.

Common situations: Verbose Python libraries logging everything to stderr; a script with a loop printing diagnostics; a misbehaving dependency emitting continuous warnings; a stuck Python process spamming errors repeatedly.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/dadc16b3a25132c6. Report an issue: GitHub.