{"record":{"id":"e56b301560c8a22a","repo":"apache/flink","slug":"failed-to-deserialize-avro-record","errorCode":null,"errorMessage":"Failed to deserialize Avro record.","messagePattern":"Failed to deserialize Avro record\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/AvroRowDataDeserializationSchema.java","lineNumber":143,"sourceCode":"        this.typeInfo = typeInfo;\n        this.runtimeConverter = runtimeConverter;\n    }\n\n    @Override\n    public void open(InitializationContext context) throws Exception {\n        this.nestedSchema.open(context);\n    }\n\n    @Override\n    public RowData deserialize(@Nullable byte[] message) throws IOException {\n        if (message == null) {\n            return null;\n        }\n        try {\n            GenericRecord deserialize = nestedSchema.deserialize(message);\n            return (RowData) runtimeConverter.convert(deserialize);\n        } catch (Exception e) {\n            throw new IOException(\"Failed to deserialize Avro record.\", e);\n        }\n    }\n\n    @Override\n    public boolean isEndOfStream(RowData nextElement) {\n        return false;\n    }\n\n    @Override\n    public TypeInformation<RowData> getProducedType() {\n        return typeInfo;\n    }\n\n    @Override\n    public boolean equals(Object o) {\n        if (this == o) {\n            return true;\n        }","sourceCodeStart":125,"sourceCodeEnd":161,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/AvroRowDataDeserializationSchema.java#L125-L161","documentation":"AvroRowDataDeserializationSchema.deserialize wraps any failure of the nested Avro decoder or the GenericRecord->RowData runtime converter in an IOException. The root cause is in the suppressed exception: usually bytes that do not match the reader schema, or a value the converter cannot map to the declared logical type.","triggerScenarios":"Calling deserialize() on bytes produced with a different/evolved Avro schema (field added/removed/type changed), corrupt or truncated Kafka payload, non-Avro bytes fed to the format, or an Avro logical type value the RowData converter does not recognize.","commonSituations":"Schema evolution without compatible reader/writer schemas in the Avro format DDL; producer writing JSON-encoded Avro while consumer expects binary; topic mix-up; toggling 'avro.timestamp.logical-type.mapping.legacy' between producer and consumer.","solutions":["Inspect the cause chain (e.getCause()) to see whether it is a decode error (schema mismatch) or a converter error (value type), and align the consumer's Avro schema with the producer's writer schema.","Verify the incoming payload is Avro binary (not JSON/Avro JSON) and matches the schema registered/configured in the AvroRowDataDeserializationSchema.","If schemas legitimately evolved, configure the deserializer with the reader schema that is compatible with the writer schema used by the producer.","Add a dirty-record side output: catch IOException in a flatMap before the format and route bad bytes to a DLQ instead of failing the job."],"exampleFix":"// before\nRowData row = deserializationSchema.deserialize(bytes);\n\n// after\nRowData row;\ntry {\n    row = deserializationSchema.deserialize(bytes);\n} catch (IOException e) {\n    log.warn(\"Bad Avro record: {}\", e.getCause().getMessage());\n    ctx.output(DLQ_TAG, bytes);\n    return;\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    RowData row = deserializer.deserialize(bytes);\n    collect(row);\n} catch (IOException e) {\n    // e.getCause() distinguishes decode failure vs converter failure\n    ctx.output(deadLetterTag, bytes); // route raw bytes to DLQ\n}","preventionTips":["Pin producer and consumer to the same Avro schema version (or register both in a schema registry).","Add a canary deserialization test over sample production payloads in CI.","Keep legacy timestamp mapping flags identical on write and read paths."],"tags":["avro","flink","deserialization","schema-evolution","kafka"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}