apache/flink · error · RuntimeException

Fail to serialize at field: %s.

Error message

Fail to serialize at field: %s.

What it means

The row converter wraps per-field conversion so that any failure inside a field's converter is rethrown as RuntimeException('Fail to serialize at field: %s.') naming the offending Avro field. The original error (null in non-nullable field, wrong physical type, nested union problem, decimal overflow) is the cause.

Source

Thrown at flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/RowDataToAvroConverters.java:291

        final int length = rowType.getFieldCount();

        return new RowDataToAvroConverter() {
            private static final long serialVersionUID = 1L;

            @Override
            public Object convert(Schema schema, Object object) {
                final RowData row = (RowData) object;
                final List<Schema.Field> fields = schema.getFields();
                final GenericRecord record = new GenericData.Record(schema);
                for (int i = 0; i < length; ++i) {
                    final Schema.Field schemaField = fields.get(i);
                    try {
                        Object avroObject =
                                fieldConverters[i].convert(
                                        schemaField.schema(), fieldGetters[i].getFieldOrNull(row));
                        record.put(i, avroObject);
                    } catch (Throwable t) {
                        throw new RuntimeException(
                                String.format(
                                        "Fail to serialize at field: %s.", schemaField.name()),
                                t);
                    }
                }
                return record;
            }
        };
    }

    private static RowDataToAvroConverter createArrayConverter(
            ArrayType arrayType, boolean legacyTimestampMapping) {
        LogicalType elementType = arrayType.getElementType();
        final ArrayData.ElementGetter elementGetter = ArrayData.createElementGetter(elementType);
        final RowDataToAvroConverter elementConverter =
                createConverter(arrayType.getElementType(), legacyTimestampMapping);

        return new RowDataToAvroConverter() {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Read the field name from the message and the root cause from getCause(); fix that specific field's nullability, type, or precision.
2. Make the Avro field nullable (["null",T]) if nulls are expected, or coalesce them away before the sink.
3. Align declared decimal precision/scale between the table schema and the Avro schema.

Example fix

// error: Fail to serialize at field: amount.
// cause: null into non-nullable bytes

// before
{"name":"amount","type":{"type":"bytes","logicalType":"decimal","precision":10,"scale":2}}

// after
{"name":"amount","type":["null",{"type":"bytes","logicalType":"decimal","precision":10,"scale":2}],"default":null}

// validate nullability of each value against its field schema before convert
for (int i = 0; i < rowType.getFieldCount(); i++) {
    if (!rowType.getTypeAt(i).isNullable() && row.isNullAt(i)) {
        throw new IllegalArgumentException("Null in non-nullable field: " + rowType.getFieldNames().get(i));
    }
}

try {
    record = (GenericRecord) converter.convert(schema, row);
} catch (RuntimeException e) {
    // message: 'Fail to serialize at field: %s.' -> identify field, fix data or schema
    String field = e.getMessage().replace("Fail to serialize at field: ", "").replace(".", "");
    ctx.output(dlqTag, row);
}

When it happens

Trigger: RowData->Avro conversion where field i's value cannot be converted against schemaField.schema(): null for a non-nullable field, decimal precision/scale mismatch, bad union shape (see the not-a-nullable-type error), or unsupported nested type.

Common situations: Schema drift between the SQL table and the Avro sink schema; upstream operators introducing nulls; decimal(38,10) values hitting a schema declared with smaller precision.

Related errors

• Failed to serialize row.
• Schema must be set when using Generic Record
• Failed to deserialize Avro record.
• Failed to serialize schema registry.
• Failed to create Avro encoder.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/6b2c0faca21c550f. Report an issue: GitHub.