apache/seatunnel · error · SeaTunnelAvroFormatException

SERIALIZATION_ERROR

SERIALIZATION_ERROR

Error message

Serialization error on record : ${element}

What it means

SeaTunnelAvroFormatException with code SERIALIZATION_ERROR thrown by AvroSerializationSchema.serialize when the Avro writer fails to encode a converted GenericRecord with an IOException. The RowConverter has already produced the GenericRecord; the failure occurs at the encoder/writer level (schema mismatch, I/O on the output stream).

Source

Thrown at seatunnel-formats/seatunnel-format-avro/src/main/java/org/apache/seatunnel/format/avro/AvroSerializationSchema.java:59

    private final RowToAvroConverter converter;
    private final DatumWriter<GenericRecord> writer;

    public AvroSerializationSchema(SeaTunnelRowType rowType) {
        this.out = new ByteArrayOutputStream();
        this.encoder = EncoderFactory.get().binaryEncoder(out, null);
        this.converter = new RowToAvroConverter(rowType);
        this.writer = this.converter.getWriter();
    }

    @Override
    public byte[] serialize(SeaTunnelRow element) {
        GenericRecord record = converter.convertRowToGenericRecord(element);
        try {
            writer.write(record, encoder);
            encoder.flush();
            return out.toByteArray();
        } catch (IOException e) {
            throw new SeaTunnelAvroFormatException(
                    AvroFormatErrorCode.SERIALIZATION_ERROR,
                    "Serialization error on record : " + element);
        } finally {
            out.reset();
        }
    }
}

View on GitHub (pinned to cf67b549a7)

Solutions

1. Compare the SeaTunnelRowType against the Avro schema and make fields nullable (union with NULL) where nulls occur
2. Regenerate the schema from the current row type so they match
3. Log/print the failing record to identify the offending field
4. Catch SeaTunnelAvroFormatException and route bad records to a dead-letter path

Example fix

// before
Schema schema = SchemaBuilder.builder().stringType(); // non-nullable field
row.setField(0, null);
// after
Schema schema = SchemaBuilder.builder().nullable().stringType();
row.setField(0, null); // now valid

// pre-validate nulls against the schema
rowType.getFieldTypes().forEach((i, t) -> {
    if (row.getField(i) == null && !isNullable(schema, i))
        throw new IllegalArgumentException("null in non-nullable field " + i);
});

try {
    byte[] bytes = avroSerializationSchema.serialize(row);
} catch (SeaTunnelAvroFormatException e) {
    LOG.error("Avro serialization failed, code={}", e.getFormatErrorCode(), e);
    deadLetterSink.send(row); // don't crash the pipeline
}

When it happens

Trigger: Calling serialize(SeaTunnelRow) where writer.write(record, encoder) or encoder.flush() throws IOException — typically a record not matching the Avro schema (wrong field type/null in non-nullable field).

Common situations: Row schema changed after the Avro schema was created; null values in fields the schema marks non-nullable; numeric type mismatches after a connector change; downstream reuse of a closed/reset stream.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• UNSUPPORTED_DATA_TYPE
• UNSUPPORTED_DATA_TYPE
• Fail to serialize at field: ${fieldName}.
• CONFIG_VALIDATION_FAILED
• CommonErrorCode.UNSUPPORTED_DATA_TYPE

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/5d97bcf0e2bbc448. Report an issue: GitHub.