apache/druid · error · IllegalArgumentException

Duplicate column name:

Error message

Duplicate column name:

What it means

Druid's catalog table validation rejects a table spec whose column list contains two columns with the same name. TableDefn.validateColumns() iterates the columns and, if a name is already seen in the HashSet, throws IAE. Column names within a table must be unique to keep the column index map well-defined.

Solutions

  1. Remove or rename the duplicate column so all column names in the spec are unique
  2. Review the spec JSON (or update payload) and find which column name appears more than once
  3. If updating an existing column, send the updated column in the update list with the same name (merge replaces it) rather than duplicating it in the base spec

Example fix

// before
"columns": [{"name": "ts", "type": "long"}, {"name": "ts", "type": "string"}]
// after
"columns": [{"name": "ts", "type": "long"}, {"name": "ts_str", "type": "string"}]
Defensive patterns

Strategy: validation

Validate before calling

Set<String> seen = new HashSet<>();
for (ColumnSpec c : spec.columns()) {
  if (!seen.add(c.name())) { throw new IllegalArgumentException("duplicate column: " + c.name()); }
}

Type guard

boolean hasUniqueColumns(List<ColumnSpec> cols) {
  return cols.stream().map(ColumnSpec::name).filter(Objects::nonNull).distinct().count() == cols.size();
}

Try / catch

try { resolvedTable.validate(); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("Duplicate column name")) { /* fix spec */ } else { throw e; } }

Prevention

When it happens

Trigger: Calling validate(), applyUpdateColumns(), or merge() on a TableSpec whose columns list contains two ColumnSpec entries with the identical name.

Common situations: Hand-editing a table spec JSON and duplicating a column; a column-update payload that re-adds an existing column name instead of replacing it; programmatic spec generation that appends columns without deduplication.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/482110dbbdd2db2d. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/catalog/model/TableDefn.java:101

   * Validate a table spec using the table, field and column definitions defined
   * here. The column definitions validate the type of each property value using
   * the object mapper.
   */
  public void validate(ResolvedTable table)
  {
    validate(table.properties(), table.jsonMapper());
    validateColumns(table.spec().columns());
  }

  public void validateColumns(List<ColumnSpec> columns)
  {
    if (columns == null) {
      return;
    }
    Set<String> names = new HashSet<>();
    for (ColumnSpec colSpec : columns) {
      if (!names.add(colSpec.name())) {
        throw new IAE("Duplicate column name: " + colSpec.name());
      }
      colSpec.validate();
      validateColumn(colSpec);
    }
  }

  /**
   * Table-specific validation of a column spec. Override for table definitions
   * that need table-specific validation rules.
   * <p>
   * A column declared without a type is legal (the type is resolved from the physical schema, the ingestion query, or
   * the input format), but a declared type must parse to a Druid type, otherwise we would silently substitute STRING
   * for the unrecognized declaration. This runs at catalog write time only: reads never validate, so tables stored
   * before this rule keep resolving (though editing them surfaces the invalid type). {@link Columns#druidType} maps
   * {@code __time} to LONG regardless of the declared type, which {@link ColumnSpec#validate} already restricts to
   * LONG or untyped.
   */
  protected void validateColumn(ColumnSpec colSpec)

View on GitHub (pinned to 9b90983fd2)