apache/druid · error · IllegalArgumentException

avroBytesDecoder is required to decode Avro records

Error message

avroBytesDecoder is required to decode Avro records

What it means

AvroStreamInputFormat cannot decode Avro records without knowing how the raw bytes were serialized, so the avroBytesDecoder property is mandatory. Constructing the input format (e.g. from an inputFormat spec) with a null decoder throws IllegalArgumentException.

Source

Thrown at extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/AvroStreamInputFormat.java:53

public class AvroStreamInputFormat extends NestedInputFormat
{
  private final boolean binaryAsString;
  private final boolean extractUnionsByType;

  private final AvroBytesDecoder avroBytesDecoder;

  @JsonCreator
  public AvroStreamInputFormat(
      @JsonProperty("flattenSpec") @Nullable JSONPathSpec flattenSpec,
      @JsonProperty("avroBytesDecoder") AvroBytesDecoder avroBytesDecoder,
      @JsonProperty("binaryAsString") @Nullable Boolean binaryAsString,
      @JsonProperty("extractUnionsByType") @Nullable Boolean extractUnionsByType
  )
  {
    super(flattenSpec);
    if (avroBytesDecoder == null) {
      throw new IAE("avroBytesDecoder is required to decode Avro records");
    }
    this.avroBytesDecoder = avroBytesDecoder;
    this.binaryAsString = binaryAsString != null && binaryAsString;
    this.extractUnionsByType = extractUnionsByType != null && extractUnionsByType;
  }

  @Override
  public boolean isSplittable()
  {
    return false;
  }

  @JsonProperty
  public AvroBytesDecoder getAvroBytesDecoder()
  {
    return avroBytesDecoder;
  }

View on GitHub (pinned to 9b90983fd2)

Solutions

Add a valid avroBytesDecoder spec to the inputFormat, e.g. {"type":"schema_registry","url":"http://registry:8081"} or {"type":"inline_schema"} / {"type":"multiple_schemas", ...}.
Verify the decoder "type" value matches a registered AvroAvroBytesDecoder subclass; fix typos so Jackson binds it instead of leaving the field null.
If constructing in Java, pass a non-null AvroBytesDecoder implementation (e.g. new SchemaRegistryAvroBytesDecoder(...)).

Example fix

// before
"inputFormat": {"type": "avro", "flattenSpec": {...}}
// after
"inputFormat": {"type": "avro", "avroBytesDecoder": {"type": "schema_registry", "url": "http://schema-registry:8081"}, "flattenSpec": {...}}

Defensive patterns

Strategy: validation

Validate before calling

// Validate inputFormat spec before submission
final Object decoder = inputFormat.get("avroBytesDecoder");
if (decoder == null || ((Map<String,Object>) decoder).get("type") == null) {
  throw new IllegalArgumentException("avroBytesDecoder with a valid type is required for the avro inputFormat");
}

Type guard

boolean hasAvroBytesDecoder(Map<String,Object> inputFormat) {
  return inputFormat.get("avroBytesDecoder") instanceof Map
      && ((Map<?,?>) inputFormat.get("avroBytesDecoder")).get("type") instanceof String;
}

Try / catch

try {
  new AvroStreamInputFormat(flattenSpec, decoder, binaryAsString, extractUnionsByType);
} catch (IllegalArgumentException e) {
  // surface a spec-validation error naming the missing avroBytesDecoder before runtime
}

Prevention

Always include avroBytesDecoder with a correct "type" (schema_registry, inline_schema, multiple_schemas) in avro inputFormats.
Double-check the decoder type spelling; a typo makes Jackson bind null.
Validate ingestion specs with a linter/test-submit before production.

When it happens

Trigger: Creating an AvroStreamInputFormat where the "avroBytesDecoder" field is absent or null in the JSON inputFormat spec — e.g. an avro bytes decoder spec missing its "type" so it deserializes to null, or the field omitted entirely.

Common situations: Typos in the decoder type name (e.g. "avro_bytes_decoder") causing failed JSON binding; copying an old inputFormat example that predates the avroBytesDecoder requirement; programmatically building the object without passing a decoder.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/793edf7912ffea52. Report an issue: GitHub.