{"record":{"id":"4b083e18b34aaac7","repo":"apache/druid","slug":"found-record-of-arbitrary-version-s","errorCode":null,"errorMessage":"Found record of arbitrary version[%s]","messagePattern":"Found record of arbitrary version\\[(.+?)\\]","errorType":"exception","errorClass":"ParseException","httpStatus":null,"severity":"error","filePath":"extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/InlineSchemasAvroBytesDecoder.java","lineNumber":108,"sourceCode":"  public Map<String, Map<String, Object>> getSchemas()\n  {\n    return schemas;\n  }\n\n  // It is assumed that record has following format.\n  // byte 1 : version, static 0x1\n  // byte 2-5 : int schemaId\n  // remaining bytes would have avro data\n  @Override\n  public GenericRecord parse(ByteBuffer bytes)\n  {\n    if (bytes.remaining() < 5) {\n      throw new ParseException(null, \"Record must have at least 5 bytes carrying version and schemaId\");\n    }\n\n    byte version = bytes.get();\n    if (version != V1) {\n      throw new ParseException(null, \"Found record of arbitrary version[%s]\", version);\n    }\n\n    int schemaId = bytes.getInt();\n    Schema schemaObj = schemaObjs.get(schemaId);\n    if (schemaObj == null) {\n      throw new ParseException(null, \"Failed to find schema for id[%s]\", schemaId);\n    }\n\n    DatumReader<GenericRecord> reader = new GenericDatumReader<>(schemaObj);\n    try (ByteBufferInputStream inputStream = new ByteBufferInputStream(Collections.singletonList(bytes))) {\n      return reader.read(null, DecoderFactory.get().binaryDecoder(inputStream, null));\n    }\n    catch (Exception e) {\n      throw new ParseException(null, e, \"Failed to read Avro message with schema id[%s]\", schemaId);\n    }\n  }\n}\n","sourceCodeStart":90,"sourceCodeEnd":126,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/InlineSchemasAvroBytesDecoder.java#L90-L126","documentation":"The first byte of each record is a format version byte; currently only version V1 (the Confluent-compatible wire format) is understood. A record whose version byte differs is rejected with a ParseException naming the offending version.","triggerScenarios":"parse(ByteBuffer) reads a record whose first byte != V1 — messages written by a newer Confluent/serializer wire-format version, a completely different serialization whose first byte collides differently (e.g. raw Avro binary data starting with a non-magic byte), or byte-offset corruption shifting the header.","commonSituations":"Upgrading serializers (schema registry clients) to a wire format the Druid decoder predates; mixed-format topics where some messages aren't Confluent-framed; misaligned reads after earlier parse bugs consumed wrong byte counts.","solutions":["Confirm the producer's wire-format magic byte is 0 (Confluent v1); reconfigure non-standard serializers to the Confluent wire format.","Upgrade Druid/avro-extensions to a version that supports the record's wire-format version if a newer format is in use.","Audit the topic for mixed serialization; split foreign-format records to another topic.","Validate a failing message offline: dump the first bytes and check the expected 0 + 4-byte schema-id layout."],"exampleFix":"// before (raw Avro binary, no header)\nDatumWriter w = new GenericDatumWriter(schema); w.write(record, enc); // first byte is Avro data\n// after\nout.write(0); // Confluent magic/version byte V1\nout.writeIntBE(schemaId);\nDatumWriter w = new GenericDatumWriter(schema); w.write(record, enc);","handlingStrategy":"validation","validationCode":"// Check the version byte before decoding\nbyte version = msg[0];\nif (version != 0 /* V1 */) {\n  throw new IllegalArgumentException(\"Unsupported Avro wire-format version \" + version + \"; expected Confluent V1 (0)\");\n}","typeGuard":"boolean isV1WireFormat(byte[] msg) {\n  return msg != null && msg.length > 0 && msg[0] == 0;\n}","tryCatchPattern":"try {\n  GenericRecord r = decoder.parse(ByteBuffer.wrap(msg));\n} catch (ParseException e) {\n  if (String.valueOf(e.getMessage()).contains(\"arbitrary version\")) {\n    deadLetter(msg, \"unsupported wire-format version\");\n  }\n}","preventionTips":["Pin serializer versions so the wire format stays Confluent V1-compatible.","Audit topics for mixed serializers/protocols before enabling Avro ingestion.","Upgrade Druid avro-extensions if your registry client emits a newer wire-format version.","Dump and verify the first byte of sample messages when changing producers."],"tags":["java","avro","wire-format","version-mismatch"],"backgroundTag":"invalid-enum-value","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}