apache/druid · error · ParseException

Failed to find schema for id[%s]

Error message

Failed to find schema for id[%s]

What it means

After validating the version byte and reading the 4-byte schema id, the decoder looks the id up in the schemas configured inline in the ingestion spec. If no schema is registered for that id, decoding is impossible and a ParseException naming the id is thrown.

Source

Thrown at extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/InlineSchemasAvroBytesDecoder.java:114

  // byte 1 : version, static 0x1
  // byte 2-5 : int schemaId
  // remaining bytes would have avro data
  @Override
  public GenericRecord parse(ByteBuffer bytes)
  {
    if (bytes.remaining() < 5) {
      throw new ParseException(null, "Record must have at least 5 bytes carrying version and schemaId");
    }

    byte version = bytes.get();
    if (version != V1) {
      throw new ParseException(null, "Found record of arbitrary version[%s]", version);
    }

    int schemaId = bytes.getInt();
    Schema schemaObj = schemaObjs.get(schemaId);
    if (schemaObj == null) {
      throw new ParseException(null, "Failed to find schema for id[%s]", schemaId);
    }

    DatumReader<GenericRecord> reader = new GenericDatumReader<>(schemaObj);
    try (ByteBufferInputStream inputStream = new ByteBufferInputStream(Collections.singletonList(bytes))) {
      return reader.read(null, DecoderFactory.get().binaryDecoder(inputStream, null));
    }
    catch (Exception e) {
      throw new ParseException(null, e, "Failed to read Avro message with schema id[%s]", schemaId);
    }
  }
}

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Add the missing schema (matching the id in the error) to the multiple_schemas decoder's schema list in the ingestion spec and restart/redo the supervision.
2. Prefer the schema_registry decoder so ids resolve dynamically against the registry instead of a static inline list.
3. Coordinate schema evolution: register the new version, update the ingestion spec, then switch producers to it.
4. Log/alert on schema-id drift and validate the failing message's id against the registry to confirm which schema to add.

Example fix

// before
{"type":"multiple_schemas","schemaInfos":[{"schema":"...v1...","id":1}]}
// after
{"type":"multiple_schemas","schemaInfos":[{"schema":"...v1...","id":1},{"schema":"...v2...","id":2}]}

// Verify every schema id in use is registered in the decoder spec
int schemaId = ByteBuffer.wrap(msg, 1, 4).getInt();
if (!declaredSchemaIds.contains(schemaId)) {
  throw new IllegalStateException("Schema id " + schemaId + " missing from multiple_schemas spec; add it before producing with that schema");
}

try {
  GenericRecord r = decoder.parse(ByteBuffer.wrap(msg));
} catch (ParseException e) {
  if (String.valueOf(e.getMessage()).contains("Failed to find schema for id")) {
    alert("Producer using unregistered schema id — update ingestion spec");
    deadLetter(msg);
  }
}

When it happens

Trigger: A message on the topic carries a schema id that is not listed in the decoder's "schemaInfos"/inline schema list — e.g. the producer registered a new schema version and started stamping its id while the Druid spec still lists only older ids.

Common situations: Producers evolve the schema (new versions) after the ingestion spec was authored; multiple topics share a decoder whose id list covers only one topic; ids copied from the schema registry don't match the inline-listed ids.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Avro + JQ not supported
• Avro + nested tree extraction not supported
• avroBytesDecoder is required to decode Avro records
• Failed to read Avro message
• Record must have at least 5 bytes carrying version and schem

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/cd867909e98566e3. Report an issue: GitHub.