apache/seatunnel · error · IOException

Failed to deserialize python source stdout line [{}]

Error message

Failed to deserialize python source stdout line [{}]

What it means

Each stdout line from the Python child is fed to the configured DeserializationSchema to produce a SeaTunnelRow. If deserialization throws for a given line, the reader wraps it in an IOException naming the offending line, failing the task rather than silently dropping data.

Source

Thrown at seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java:461

                        "python-source-stdout-pump");
        stdoutPumpThread.setDaemon(true);
        stdoutPumpThread.start();
    }

    private void offerStdoutLine(String line) throws InterruptedException {
        while (!closeRequested) {
            if (stdoutLines.offer(line, QUEUE_OFFER_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)) {
                return;
            }
        }
    }

    private void emitRow(String line, Collector<SeaTunnelRow> output) throws IOException {
        SeaTunnelRow row;
        try {
            row = deserializationSchema.deserialize(line.getBytes(StandardCharsets.UTF_8));
        } catch (Exception e) {
            throw new IOException(
                    "Failed to deserialize python source stdout line [" + line + "]", e);
        }

        if (row != null) {
            output.collect(row);
        }
    }

    private void finishIfProcessCompleted() throws Exception {
        if (closeRequested || processExitVerified) {
            return;
        }

        if (process.isAlive()) {
            return;
        }

        if (!verifyProcessExit()) {

View on GitHub (pinned to cf67b549a7)

Solutions

  1. Move all logging/debug prints in the Python script to stderr
  2. Print exactly one JSON object per row, matching the declared schema
  3. Validate a sample output line with the same deserializer locally
  4. Check the offending line in the message for schema mismatches

Example fix

# before
print('loaded model')
print(json.dumps(row))
# after
print('loaded model', file=sys.stderr)
print(json.dumps(row))
Defensive patterns

Strategy: try-catch

Validate before calling

import json
line = get_sample_stdout_line()
json.loads(line)  # must succeed and match the declared SeaTunnel schema

Try / catch

try {
    reader.pollNext(output);
} catch (IOException e) {
    if (e.getMessage().startsWith("Failed to deserialize python source stdout line")) {
        // offending line is in e.getMessage(); fix script output or schema
    }
}

Prevention

When it happens

Trigger: The Python script prints a line that the DeserializationSchema cannot parse — wrong JSON structure, non-UTF8-safe control characters, partial/truncated line, debug print statements mixed into stdout, or a schema/type change on the Python side.

Common situations: Script prints logs or banners to stdout instead of stderr; Python emits JSON with fields/types that don't match the declared SeaTunnel schema; script output truncated on crash mid-line; version drift between script output format and job schema.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/d116e0eb0c117bae. Report an issue: GitHub.