apache/flink · error · WrappingRuntimeException

Failed to create Avro encoder.

Error message

Failed to create Avro encoder.

What it means

Thrown from AvroSerializationSchema.checkAvroInitialized when EncoderFactory.get().jsonEncoder(schema, out) raises IOException while constructing a JSON encoder for AvroEncoding.JSON. The schema was already parsed successfully (or the parse error surfaces separately); encoder creation fails for malformed schema structures that Avro's JsonEncoder rejects.

Source

Thrown at flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/AvroSerializationSchema.java:200

        }
        ClassLoader cl = Thread.currentThread().getContextClassLoader();
        if (SpecificRecord.class.isAssignableFrom(recordClazz)) {
            Schema schema = SpecificData.get().getSchema(recordClazz);
            this.datumWriter = new SpecificDatumWriter<>(schema);
            this.schema = schema;
        } else {
            this.schema = new Schema.Parser().parse(this.schemaString);
            GenericData genericData = new GenericData(cl);

            this.datumWriter = new GenericDatumWriter<>(schema, genericData);
        }

        this.arrayOutputStream = new ByteArrayOutputStream();
        if (encoding == AvroEncoding.JSON) {
            try {
                this.encoder = EncoderFactory.get().jsonEncoder(this.schema, arrayOutputStream);
            } catch (IOException e) {
                throw new WrappingRuntimeException("Failed to create Avro encoder.", e);
            }
        } else {
            this.encoder = EncoderFactory.get().directBinaryEncoder(arrayOutputStream, null);
        }
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) {
            return true;
        }
        if (o == null || getClass() != o.getClass()) {
            return false;
        }
        AvroSerializationSchema<?> that = (AvroSerializationSchema<?>) o;
        return recordClazz.equals(that.recordClazz) && Objects.equals(schema, that.schema);
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Validate the schema independently: new Schema.Parser().setValidate(true).parse(schemaString) and EncoderFactory.get().jsonEncoder(schema, new ByteArrayOutputStream()) in a unit test before deploying.
2. Fix the .avsc (name the nested records properly, correct malformed unions).
3. If JSON output is optional, fall back to AvroEncoding.BINARY which uses directBinaryEncoder and does not hit this path.

Example fix

// before
new AvroSerializationSchema<>(clazz, schemaString, AvroEncoding.JSON) // schema broken for JSON

// after
// pre-validate in a test:
Schema s = new Schema.Parser().parse(schemaString);
EncoderFactory.get().jsonEncoder(s, new ByteArrayOutputStream()); // throws early, fix schema here

Schema s = new Schema.Parser().parse(schemaString);
try {
    EncoderFactory.get().jsonEncoder(s, new ByteArrayOutputStream());
} catch (IOException e) {
    throw new IllegalArgumentException("Schema unusable for JSON encoding: " + e.getMessage(), e);
}

When it happens

Trigger: Building AvroSerializationSchema with encoding=AvroEncoding.JSON and a schema string whose record/field structure is invalid for JSON encoding (e.g. unnamed nested types Avro cannot name in JSON output).

Common situations: Switching from binary to JSON encoding for debugging/inspection; hand-written .avsc with subtle structural errors that a lenient parser accepts but JsonEncoder rejects.

Related errors

• Schema must be set when using Generic Record
• Failed to deserialize Avro record.
• Failed to serialize row.
• Failed to serialize schema registry.
• Unsupported type: {}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/0264d869d9b72021. Report an issue: GitHub.