apache/flink · error · WrappingRuntimeException

Failed to serialize schema registry.

Error message

Failed to serialize schema registry.

What it means

RegistryAvroSerializationSchema.serialize throws WrappingRuntimeException('Failed to serialize schema registry.') when any IOException occurs while writing the schema id header via schemaCoder.writeSchema() or encoding the record. Unlike the plain AvroSerializationSchema, here this genuinely includes Confluent Schema Registry interactions (e.g. registering/looking up the schema id) failing.

Source

Thrown at flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/RegistryAvroSerializationSchema.java:125

    }

    @Override
    public byte[] serialize(T object) {
        checkAvroInitialized();

        if (object == null) {
            return null;
        } else {
            try {
                ByteArrayOutputStream outputStream = getOutputStream();
                outputStream.reset();
                Encoder encoder = getEncoder();
                schemaCoder.writeSchema(getSchema(), outputStream);
                getDatumWriter().write(object, encoder);
                encoder.flush();
                return outputStream.toByteArray();
            } catch (IOException e) {
                throw new WrappingRuntimeException("Failed to serialize schema registry.", e);
            }
        }
    }

    @Override
    protected void checkAvroInitialized() {
        super.checkAvroInitialized();
        if (schemaCoder == null) {
            schemaCoder = schemaCoderProvider.get();
        }
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) {
            return true;
        }
        if (o == null || getClass() != o.getClass()) {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Unwrap e.getCause(): if it is a RegistryRetrieverException/RetriableException/ConnectException, fix registry connectivity/credentials (URL, basic.auth.credentials.source, TLS truststore).
2. If registration was rejected, resolve schema compatibility in the registry (evolve the schema compatibly or adjust subject compatibility mode deliberately).
3. Make the registry client resilient: increase timeouts/retries on the cached schema coder provider; ensure the schema id is pre-registered so writeSchema() only reads.
4. Verify the record matches getSchema() to rule out a plain encode failure.

Example fix

// before
RegistryAvroSerializationSchema.forGeneric(topic, false).serialize(record);

// after
// pre-register the schema once from the client, so the hot path only writes:
CachedSchemaCoderProvider provider = new CachedSchemaCoderProvider(registryUrl);
// + fix env: schema.registry.url / basic.auth.user-info set correctly

// fail fast on registry reachability before the job starts
try (SchemaRegistryClient c = new CachedSchemaRegistryClient(registryUrl, 100)) {
    c.getAllSubjects(); // throws if URL/auth wrong
} catch (IOException e) {
    throw new IllegalStateException("Schema Registry not reachable: " + registryUrl, e);
}

try {
    bytes = serializer.serialize(record);
} catch (WrappingRuntimeException e) {
    Throwable root = ExceptionUtils.stripExecution(e);
    if (root instanceof RetriableException) {
        // transient registry hiccup: retry with backoff
        throw new Retry Later (backoff) exception;
    }
    throw e; // schema/data errors are not retriable
}

When it happens

Trigger: Schema Registry unreachable/timeout, auth (TLS/basic auth) misconfigured, schema registration rejected (incompatible), or the record not conforming to the writer schema — any of these surfaces as this wrapping exception during serialize().

Common situations: Network flakiness between TaskManager and the registry; wrong registry URL or missing credentials; schema compatibility mode rejecting a new version; kafka topic + registry mismatch after schema evolution.

Related errors

• Could not find schema with id %s in registry
• Unknown data format. Magic number does not match
• Could not register schema in registry
• Option %s.%s is required for serialization
• Failed to serialize row.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/7bd38208f3fc4cdb. Report an issue: GitHub.