{"record":{"id":"f1234ac42c4be94b","repo":"apache/druid","slug":"failed-to-read-avro-message-with-schema-id-s","errorCode":null,"errorMessage":"Failed to read Avro message with schema id[%s]","messagePattern":"Failed to read Avro message with schema id\\[(.+?)\\]","errorType":"exception","errorClass":"ParseException","httpStatus":null,"severity":"error","filePath":"extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/InlineSchemasAvroBytesDecoder.java","lineNumber":122,"sourceCode":"    }\n\n    byte version = bytes.get();\n    if (version != V1) {\n      throw new ParseException(null, \"Found record of arbitrary version[%s]\", version);\n    }\n\n    int schemaId = bytes.getInt();\n    Schema schemaObj = schemaObjs.get(schemaId);\n    if (schemaObj == null) {\n      throw new ParseException(null, \"Failed to find schema for id[%s]\", schemaId);\n    }\n\n    DatumReader<GenericRecord> reader = new GenericDatumReader<>(schemaObj);\n    try (ByteBufferInputStream inputStream = new ByteBufferInputStream(Collections.singletonList(bytes))) {\n      return reader.read(null, DecoderFactory.get().binaryDecoder(inputStream, null));\n    }\n    catch (Exception e) {\n      throw new ParseException(null, e, \"Failed to read Avro message with schema id[%s]\", schemaId);\n    }\n  }\n}\n","sourceCodeStart":104,"sourceCodeEnd":126,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/InlineSchemasAvroBytesDecoder.java#L104-L126","documentation":"This ParseException is thrown by InlineSchemasAvroBytesDecoder.parse when the Avro binary payload cannot be decoded into a GenericRecord using the schema looked up from the inline schema map for the given schema id. The message bytes themselves are malformed relative to the schema, or the configured inline schema does not match the data.","triggerScenarios":"Calling parse(ByteBuffer bytes) where reader.read() fails: payload bytes corrupted/truncated, wrong schema registered for the schema id in inlineSchemas, or payload was not Avro binary-encoded.","commonSituations":"Kafka topics with multiple producers using different schemas under the same id; schema evolved but the inline schema map was not updated; non-Avro bytes sent to the topic by mistake.","solutions":["Verify the schema registered in inlineSchemas for the reported schema id matches the producer's schema (use the producer's .avsc)","Inspect raw message bytes to confirm they are valid Avro binary data","Ensure producer and consumer use compatible Avro writer/reader schema evolution (e.g., add fields with defaults)","Re-ingest the offending messages after fixing the schema configuration"],"exampleFix":"// before\n\"inlineSchemas\": {\"1\": \"{\\\"namespace\\\":\\\"old\\\",...}\"}\n// after\n\"inlineSchemas\": {\"1\": \"{\\\"namespace\\\":\\\"new\\\",\\\"fields\\\":[{\\\"name\\\":\\\"extraField\\\",\\\"type\\\":\\\"string\\\",\\\"default\\\":\\\"\\\"}, ...]}\"}","handlingStrategy":"try-catch","validationCode":"// before ingestion, verify schema compatibility offline\nSchema writer = new Schema.Parser().parse(new File(\"writer.avsc\"));\nSchema reader = new Schema.Parser().parse(configuredInlineSchema);\nif (!writer.getFullName().equals(reader.getFullName()) || !reader.isCompatible(writer)) {\n  throw new IllegalArgumentException(\"Inline schema does not match producer schema\");\n}","typeGuard":"boolean isValidAvroPayload(ByteBuffer bytes) {\n  return bytes != null && bytes.remaining() > 0;\n}","tryCatchPattern":"try {\n  GenericRecord record = decoder.parse(bytes);\n} catch (ParseException e) {\n  LOG.warn(e, \"Dropping malformed Avro message\");\n  metrics.counter(\"avro-parse-failures\").inc();\n  // skip or route to DLQ\n}","preventionTips":["Keep inlineSchemas in sync with producer schema versions via CI check","Enforce schema compatibility (backward) on the producer side","Validate messages offline with avro-tools before wiring ingestion","Monitor parse failure rates to catch schema drift early"],"tags":["avro","deserialization","kafka-ingestion"],"backgroundTag":"schema-validation-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}