apache/druid · error · IllegalArgumentException

Either [columns] or [findColumnsFromHeader] must be set

Error message

Either [columns] or [findColumnsFromHeader] must be set

What it means

FlatTextInputFormat requires a way to learn the column names: either an explicit 'columns' list or header detection via 'findColumnsFromHeader'/'hasHeaderRow'. When neither is provided it cannot build a signature and throws IAE in the constructor.

Source

Thrown at processing/src/main/java/org/apache/druid/data/input/impl/FlatTextInputFormat.java:70

      @Nullable List<String> columns,
      @Nullable String listDelimiter,
      String delimiter,
      @Nullable Boolean hasHeaderRow,
      @Nullable Boolean findColumnsFromHeader,
      int skipHeaderRows,
      @Nullable Boolean tryParseNumbers
  )
  {
    this.columns = columns == null ? Collections.emptyList() : columns;
    this.listDelimiter = listDelimiter;
    this.delimiter = Preconditions.checkNotNull(delimiter, "delimiter");
    if (columns == null || columns.isEmpty()) {
      if (hasHeaderRow != null && findColumnsFromHeader != null) {
        // User provided both hasHeaderRow and findColumnsFromHeader.
        throw new IAE("Cannot accept both [findColumnsFromHeader] and [hasHeaderRow]");
      } else if (hasHeaderRow == null && findColumnsFromHeader == null) {
        // User provided neither columns, nor one of the header-related parameters.
        throw new IAE("Either [columns] or [findColumnsFromHeader] must be set");
      } else {
        // User provided one of hasHeaderRow or findColumnsFromHeader. Take the one they provided.
        this.findColumnsFromHeader = hasHeaderRow != null ? hasHeaderRow : findColumnsFromHeader;
      }
    } else {
      this.findColumnsFromHeader = findColumnsFromHeader == null ? false : findColumnsFromHeader;
    }
    this.skipHeaderRows = skipHeaderRows;
    Preconditions.checkArgument(
        !delimiter.equals(listDelimiter),
        "Cannot have same delimiter and list delimiter of [%s]",
        delimiter
    );
    this.tryParseNumbers = tryParseNumbers == null ? false : tryParseNumbers;

    if (!this.columns.isEmpty()) {
      for (String column : this.columns) {
        Preconditions.checkArgument(

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Add "findColumnsFromHeader": true to read column names from the file's first row
2. Or supply an explicit "columns": [...] array in the inputFormat
3. Or set the legacy "hasHeaderRow": true if on an older API surface

Example fix

// before
{"type":"csv"}
// after
{"type":"csv","findColumnsFromHeader":true}

// Before submitting the spec
Map<String,Object> fmt = (Map<String,Object>) spec.get("inputFormat");
boolean hasColumns = fmt.get("columns") instanceof List && !((List<?>) fmt.get("columns")).isEmpty();
boolean hasHeader = fmt.get("findColumnsFromHeader") != null || fmt.get("hasHeaderRow") != null;
if (!hasColumns && !hasHeader) { throw new IllegalArgumentException("csv/tsv inputFormat needs columns or findColumnsFromHeader"); }

try { buildFormat(cfg); } catch (IAE e) { cfg.put("findColumnsFromHeader", true); buildFormat(cfg); }

When it happens

Trigger: Creating FlatTextInputFormat with columns null or empty while hasHeaderRow and findColumnsFromHeader are both null (e.g. an inputFormat spec with just {"type":"csv"}).

Common situations: Minimal CSV ingestion specs that forgot header configuration; generated specs where the columns field was dropped by a templating step; users assuming the first row is used as a header by default.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Empty columns
• Cannot accept both [findColumnsFromHeader] and [hasHeaderRow
• The gRPC query server requires either a Basic or Anonymous a
• Metric [%s] not whitelisted.
• Can't load TrustStore. Truststore path or password is not se

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/4177b9ff57f3c233. Report an issue: GitHub.