{"record":{"id":"fed3d6de84fae0a2","repo":"apache/druid","slug":"duplicate-column-entries-found-s","errorCode":null,"errorMessage":"Duplicate column entries found : %s","messagePattern":"Duplicate column entries found : (.+?)","errorType":"validation","errorClass":"ParseException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/java/util/common/parsers/ParserUtils.java","lineNumber":141,"sourceCode":"  {\n    Set<String> duplicates = new HashSet<>();\n    Set<String> uniqueNames = new HashSet<>();\n\n    for (String fieldName : fieldNames) {\n      if (uniqueNames.contains(fieldName)) {\n        duplicates.add(fieldName);\n      }\n      uniqueNames.add(fieldName);\n    }\n\n    return duplicates;\n  }\n\n  public static void validateFields(Iterable<String> fieldNames)\n  {\n    Set<String> duplicates = findDuplicates(fieldNames);\n    if (!duplicates.isEmpty()) {\n      throw new ParseException(null, \"Duplicate column entries found : %s\", duplicates.toString());\n    }\n  }\n\n  public static String stripQuotes(String input)\n  {\n    input = input.trim();\n    if (input.charAt(0) == '\\\"' && input.charAt(input.length() - 1) == '\\\"') {\n      input = input.substring(1, input.length() - 1).trim();\n    }\n    return input;\n  }\n\n  @Nullable\n  public static DateTimeZone getDateTimeZone(String timeZone)\n  {\n    return TIMEZONE_LOOKUP.get(timeZone);\n  }\n","sourceCodeStart":123,"sourceCodeEnd":159,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/java/util/common/parsers/ParserUtils.java#L123-L159","documentation":"ParserUtils.validateFields() checks the supplied field name list for duplicates using findDuplicates and throws a ParseException listing the duplicated column names. Druid parsers require unique column names per row schema.","triggerScenarios":"Calling ParserUtils.validateFields(fieldNames) (directly or via setFieldNames/parseSpec initialization) with an Iterable<String> containing repeated column names.","commonSituations":"parseSpec 'columns' list with the same column listed twice; dimensionsSpec dimensions duplicated; programmatically assembling field names from multiple sources without deduplication.","solutions":["Remove duplicate entries from the columns/dimensions list so each column appears once","Deduplicate programmatically, e.g. new ArrayList<>(new LinkedHashSet<>(fieldNames))","Validate the spec JSON before submission to catch duplicated column names early"],"exampleFix":"// before\n\"columns\": [\"ts\", \"user\", \"user\"]\n// after\n\"columns\": [\"ts\", \"user\"]","handlingStrategy":"validation","validationCode":"List<String> deduped = new ArrayList<>(new LinkedHashSet<>(columns));\nif (deduped.size() != columns.size()) throw new IllegalArgumentException(\"duplicate columns: \" + (columns.size() - deduped.size()));\nParserUtils.validateFields(deduped);","typeGuard":null,"tryCatchPattern":"try { parser.setFieldNames(fieldNames); } catch (ParseException e) { if (e.getMessage().startsWith(\"Duplicate column entries\")) { /* fix spec */ } throw e; }","preventionTips":["Deduplicate columns lists with LinkedHashSet before constructing parseSpecs","Check dimensionsSpec for repeated dimension names","Validate spec JSON for duplicate column names at submission time"],"tags":["parser","duplicate-columns","validation"],"backgroundTag":"schema-validation-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}