What causes "Failed to read Avro message" in apache/druid?

Verify the schema stored in the schema repo for the subject/id matches the producer's writer schema

apache/druid · error · ParseException

Failed to read Avro message

Error message

Failed to read Avro message

What it means

Thrown by SchemaRepoBasedAvroBytesDecoder.parse when the Avro message cannot be decoded after the schema has been resolved from the subject/id repository. Same family as the other Avro read failures but for the (older) SchemaRepo-based decoder.

Solutions

Verify the schema stored in the schema repo for the subject/id matches the producer's writer schema
Update or re-register the correct schema version in the repo
Validate the raw message with a standalone Avro tool to detect corruption
Consider migrating to the schema_registry decoder if actually using Confluent wire format

Example fix

null

Defensive patterns

Strategy: try-catch

Validate before calling

// verify subject/id resolution before decode
Schema s = typedRepository.getSchema(subject, id);
if (s == null) throw new IllegalStateException("No schema in repo for " + subject + "/" + id);

Try / catch

try {
  GenericRecord record = decoder.parse(bytes);
} catch (ParseException e) {
  LOG.warn(e, "SchemaRepo Avro decode failed; routing to DLQ");
}

Prevention

Publish every producer schema version to the repo before rollout
Migrate to schema_registry decoder if using Confluent wire format
Checksum-validate large messages end-to-end
Keep writer/reader schema compatibility tests in CI

When it happens

Trigger: parse(ByteBuffer bytes) resolves subject and id via typedRepository, then reader.read() throws — schema mismatch between writer and repository schema, corrupted bytes, or truncated message in the ByteBufferInputStream.

Common situations: Messages written under an evolved schema not present/updated in the schema repo; corrupted records on the topic; using this legacy decoder with Confluent wire-format messages it does not understand.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/e7261c4a4b13799d. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/SchemaRepoBasedAvroBytesDecoder.java:85

  }

  @JsonProperty
  public SubjectAndIdConverter<SUBJECT, ID> getSubjectAndIdConverter()
  {
    return subjectAndIdConverter;
  }

  @Override
  public GenericRecord parse(ByteBuffer bytes)
  {
    Pair<SUBJECT, ID> subjectAndId = subjectAndIdConverter.getSubjectAndId(bytes);
    Schema schema = typedRepository.getSchema(subjectAndId.lhs, subjectAndId.rhs);
    DatumReader<GenericRecord> reader = new GenericDatumReader<>(schema);
    try (ByteBufferInputStream inputStream = new ByteBufferInputStream(Collections.singletonList(bytes))) {
      return reader.read(null, DecoderFactory.get().binaryDecoder(inputStream, null));
    }
    catch (Exception e) {
      throw new ParseException(null, e, "Failed to read Avro message");
    }
  }

  @Override
  public boolean equals(Object o)
  {
    if (this == o) {
      return true;
    }
    if (o == null || getClass() != o.getClass()) {
      return false;
    }

    SchemaRepoBasedAvroBytesDecoder<?, ?> that = (SchemaRepoBasedAvroBytesDecoder<?, ?>) o;

    if (!Objects.equals(subjectAndIdConverter, that.subjectAndIdConverter)) {
      return false;
    }

View on GitHub (pinned to 9b90983fd2)