apache/druid · error · ParseException

Duplicate column entries found

Error message

Duplicate column entries found : %s

What it means

ParserUtils.validateFields() checks the supplied field name list for duplicates using findDuplicates and throws a ParseException listing the duplicated column names. Druid parsers require unique column names per row schema.

Solutions

1. Remove duplicate entries from the columns/dimensions list so each column appears once
2. Deduplicate programmatically, e.g. new ArrayList<>(new LinkedHashSet<>(fieldNames))
3. Validate the spec JSON before submission to catch duplicated column names early

Example fix

// before
"columns": ["ts", "user", "user"]
// after
"columns": ["ts", "user"]

List<String> deduped = new ArrayList<>(new LinkedHashSet<>(columns));
if (deduped.size() != columns.size()) throw new IllegalArgumentException("duplicate columns: " + (columns.size() - deduped.size()));
ParserUtils.validateFields(deduped);

try { parser.setFieldNames(fieldNames); } catch (ParseException e) { if (e.getMessage().startsWith("Duplicate column entries")) { /* fix spec */ } throw e; }

When it happens

Trigger: Calling ParserUtils.validateFields(fieldNames) (directly or via setFieldNames/parseSpec initialization) with an Iterable<String> containing repeated column names.

Common situations: parseSpec 'columns' list with the same column listed twice; dimensionsSpec dimensions duplicated; programmatically assembling field names from multiple sources without deduplication.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• A local input source accepts only one of
• A local input source can set parameter
• A local input source requires one parameter of
• A local input source requires one property of
• A table definition must include a table spec.

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/fed3d6de84fae0a2. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/java/util/common/parsers/ParserUtils.java:141

  {
    Set<String> duplicates = new HashSet<>();
    Set<String> uniqueNames = new HashSet<>();

    for (String fieldName : fieldNames) {
      if (uniqueNames.contains(fieldName)) {
        duplicates.add(fieldName);
      }
      uniqueNames.add(fieldName);
    }

    return duplicates;
  }

  public static void validateFields(Iterable<String> fieldNames)
  {
    Set<String> duplicates = findDuplicates(fieldNames);
    if (!duplicates.isEmpty()) {
      throw new ParseException(null, "Duplicate column entries found : %s", duplicates.toString());
    }
  }

  public static String stripQuotes(String input)
  {
    input = input.trim();
    if (input.charAt(0) == '\"' && input.charAt(input.length() - 1) == '\"') {
      input = input.substring(1, input.length() - 1).trim();
    }
    return input;
  }

  @Nullable
  public static DateTimeZone getDateTimeZone(String timeZone)
  {
    return TIMEZONE_LOOKUP.get(timeZone);
  }

View on GitHub (pinned to 9b90983fd2)