{"record":{"id":"867ee5d6e303ce4a","repo":"apache/druid","slug":"failed-to-decode-avro-message-for-schema-id-s","errorCode":null,"errorMessage":"Failed to decode Avro message for schema id[%s]","messagePattern":"Failed to decode Avro message for schema id\\[(.+?)\\]","errorType":"exception","errorClass":"ParseException","httpStatus":null,"severity":"error","filePath":"extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/SchemaRegistryBasedAvroBytesDecoder.java","lineNumber":179,"sourceCode":"      // For all other errors, just include the code and message received from the library.\n      throw new ParseException(\n          null,\n          ex2,\n          \"Failed to fetch Avro schema id[%s] from registry. Error code[%s] and message[%s].\",\n          id,\n          ex2.getErrorCode(),\n          ex2.getMessage()\n      );\n    }\n    if (schema == null) {\n      throw new ParseException(null, \"No Avro schema id[%s] in registry\", id);\n    }\n    DatumReader<GenericRecord> reader = new GenericDatumReader<>(schema);\n    try {\n      return reader.read(null, DecoderFactory.get().binaryDecoder(bytes.array(), offset, length, null));\n    }\n    catch (Exception e) {\n      throw new ParseException(null, e, \"Failed to decode Avro message for schema id[%s]\", id);\n    }\n  }\n\n  @Override\n  public boolean equals(Object o)\n  {\n    if (this == o) {\n      return true;\n    }\n    if (o == null || getClass() != o.getClass()) {\n      return false;\n    }\n    SchemaRegistryBasedAvroBytesDecoder that = (SchemaRegistryBasedAvroBytesDecoder) o;\n    return capacity == that.capacity\n           && Objects.equals(url, that.url)\n           && Objects.equals(urls, that.urls)\n           && Objects.equals(config, that.config)\n           && Objects.equals(headers, that.headers);","sourceCodeStart":161,"sourceCodeEnd":197,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/SchemaRegistryBasedAvroBytesDecoder.java#L161-L197","documentation":"Thrown when the Avro schema was successfully fetched from the registry but the binary message still fails to decode via GenericDatumReader.read(). The payload does not conform to the schema registered under its id.","triggerScenarios":"parse(ByteBuffer bytes) calls reader.read() with the registry schema and it throws — message bytes corrupted, written with a different/incompatible schema version than the one registered under that id, or truncation (e.g., oversized record cut off).","commonSituations":"Schema registry entries overwritten with non-backwards-compatible schemas; message truncation by broker/proxy (fetch.max.bytes limits); producer bugs writing malformed records; mixing Avro serialization configs (e.g., different normalize settings).","solutions":["Compare the writer schema of the failing messages with the registry schema for that id and fix any incompatible change (use full/backward compatibility mode on the registry subject)","Check for message truncation: increase Kafka max.message.bytes / fetch sizes if records are large","Inspect a raw failing message with a standalone Avro decoder to isolate the byte offset","Reproduce with the same GenericDatumReader to validate the fix offline"],"exampleFix":"null","handlingStrategy":"try-catch","validationCode":"// decode offline to reproduce and localize the failure\nDatumReader<GenericRecord> r = new GenericDatumReader<>(registrySchema);\nDecoder d = DecoderFactory.get().binaryDecoder(rawBytes, null);\nr.read(null, d); // throws here if bytes don't match schema","typeGuard":null,"tryCatchPattern":"try {\n  GenericRecord record = decoder.parse(bytes);\n} catch (ParseException e) {\n  if (e.getMessage().startsWith(\"Failed to decode Avro message\")) {\n    LOG.warn(e, \"Message does not conform to registered schema; sending to DLQ\");\n  }\n}","preventionTips":["Enable FULL_TRANSITIVE compatibility on registry subjects","Compare fetched schema byte-for-byte against producer .avsc in CI","Watch for truncation: align max.message.bytes across brokers/clients","Keep a dead-letter topic for undecodable messages"],"tags":["avro","deserialization","schema-registry"],"backgroundTag":"schema-validation-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}