apache/druid · error · IAE

Cannot have duplicate field definition: %s

Error message

Cannot have duplicate field definition: %s

What it means

ObjectFlatteners.create() keeps a Map of field name -> extractor; putting a second extractor under a name that already exists returns non-null and triggers this IAE. Two fieldSpec entries in the same flattenSpec must not share the same 'name'.

Source

Thrown at processing/src/main/java/org/apache/druid/java/util/common/parsers/ObjectFlatteners.java:77

      switch (fieldSpec.getType()) {
        case ROOT:
          extractor = obj -> flattenerMaker.getRootField(obj, fieldSpec.getExpr());
          break;
        case PATH:
          extractor = flattenerMaker.makeJsonPathExtractor(fieldSpec.getExpr());
          break;
        case JQ:
          extractor = flattenerMaker.makeJsonQueryExtractor(fieldSpec.getExpr());
          break;
        case TREE:
          extractor = flattenerMaker.makeJsonTreeExtractor(fieldSpec.getNodes());
          break;
        default:
          throw new UOE("Unsupported field type[%s]", fieldSpec.getType());
      }

      if (extractors.put(fieldSpec.getName(), extractor) != null) {
        throw new IAE("Cannot have duplicate field definition: %s", fieldSpec.getName());
      }
    }

    return new ObjectFlattener<>()
    {
      @Override
      public Map<String, Object> flatten(final T obj)
      {
        return new AbstractMap<>()
        {
          @Override
          public int size()
          {
            return keySet().size();
          }

          @Override
          public boolean isEmpty()

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Remove or rename the duplicate fieldSpec entry so every fieldSpec has a unique 'name'
2. Deduplicate the flattener list before building the spec (e.g. by field name, keeping the last/first occurrence intentionally)
3. Log or assert spec uniqueness in spec-generation tooling

Example fix

// before
[{"type":"path","name":"val","expr":"$.a"},{"type":"path","name":"val","expr":"$.b"}]
// after
[{"type":"path","name":"val_a","expr":"$.a"},{"type":"path","name":"val_b","expr":"$.b"}]

Set<String> seen = new HashSet<>();
for (FieldSpec fs : flattenSpec.getFlattener()) {
  if (!seen.add(fs.getName())) throw new IllegalArgumentException("duplicate fieldSpec name: " + fs.getName());
}

try { flattener = ObjectFlatteners.create(spec, maker); } catch (IAE e) { throw new SpecValidationException("Duplicate field in flattenSpec", e); }

When it happens

Trigger: Passing a flattenSpec whose 'flattener' list contains two or more fieldSpecs with identical 'name' values to ObjectFlatteners.create().

Common situations: Copy-pasted fieldSpec entries with the name not updated after changing the expr; programmatically merging specs from two sources that both define the same output column; templated spec generation producing duplicate names.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• ORC flattener does not support JQ
• Unsupported field type[%s]
• ColumnCapacityExceededException
• Bloom filter aggregators are query-time only
• Unsupported keyFormat. KafkaInputformat only supports input

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/296c3fe46a6cb1b1. Report an issue: GitHub.