apache/druid · error · IllegalArgumentException

If an external table provides a format, it must also…

Error message

If an external table provides a format, it must also provide columns

What it means

The mirror condition of the columns-without-format check: FormattedInputSourceDefn.validate rejects an external table that declares an input format but no columns. For formatted sources Druid requires the schema to be supplied alongside the format.

Solutions

1. Add column definitions to the table spec matching the data.
2. Run schema discovery/sampling to populate columns before saving.
3. Remove the format if columns will be inferred elsewhere (only if the source defn allows).
4. Validate the spec locally before submission: columns XOR format is never accepted for formatted sources.

Example fix

// before
{"inputFormat":{"type":"json"}}
// after
{"inputFormat":{"type":"json"},"columns":[{"name":"a","type":"bigint"}]}

boolean hasCols = spec.getColumns() != null && !spec.getColumns().isEmpty();
boolean hasFmt = spec.getInputFormat() != null;
if (!hasCols && hasFmt) throw new IllegalArgumentException("format requires column definitions");

try { saveExternalTable(spec); } catch (IAE e) { if (e.getMessage().contains("must also provide columns")) { runSchemaDiscovery(spec); saveExternalTable(spec); } }

When it happens

Trigger: Validating an external table where inputFormatMap is non-null but spec().columns() is empty; e.g. saving a spec with {"inputFormat":{"type":"json"}} and no column definitions.

Common situations: Providing format via an API that does not auto-populate columns; skipping the schema-discovery step in a wizard; hand-written specs that include format but omit columns.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• If an external table provides columns, it must also provide…
• Actual Row count mismatch. Expected
• Aggregation [ ] does not support column [ ] of type [ ]…
• at least 2 bins expected
• at least 2 bins expected

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/638cd580422227e2. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/catalog/model/table/FormattedInputSourceDefn.java:69

  private Map<String, InputFormatDefn> formats;

  @Override
  public void bind(TableDefnRegistry registry)
  {
    formats = registry.formats();
    super.bind(registry);
  }

  @Override
  public void validate(ResolvedExternalTable table)
  {
    final boolean hasColumns = !CollectionUtils.isNullOrEmpty(table.resolvedTable().spec().columns());
    final boolean hasFormat = table.inputFormatMap != null;
    if (hasColumns && !hasFormat) {
      throw new IAE("If an external table provides columns, it must also provide a format");
    }
    if (!hasColumns && hasFormat) {
      throw new IAE("If an external table provides a format, it must also provide columns");
    }
    super.validate(table);
  }

  @Override
  protected AdHocTableFunction defineAdHocTableFunction()
  {
    List<ParameterDefn> fullTableParams = adHocTableFnParameters();
    List<ParameterDefn> allParams = addFormatParameters(fullTableParams);
    return new AdHocTableFunction(allParams);
  }

  /**
   * Overridden by subclasses to provide the list of table function parameters for
   * this specific input format. This list is combined with parameters for input
   * formats. The method is called only once per run.
   */
  protected abstract List<ParameterDefn> adHocTableFnParameters();

View on GitHub (pinned to 9b90983fd2)