apache/iceberg · error · IllegalArgumentException

Cannot add duplicate partition field name: %s

Error message

Cannot add duplicate partition field name: %s

What it means

When adding a partition field via BaseUpdatePartitionSpec.addField, the resulting field name must be unique among live spec fields. If an existing field (not marked for deletion in this update) already uses the name and is not a void transform eligible for auto-rename, the update throws IllegalArgumentException to prevent ambiguous partition field names.

Source

Thrown at core/src/main/java/org/apache/iceberg/BaseUpdatePartitionSpec.java:229

    PartitionField newField = recycleOrCreatePartitionField(sourceTransform, name);
    if (newField.name() == null) {
      String partitionName =
          PartitionSpecVisitor.visit(schema, newField, PartitionNameGenerator.INSTANCE);
      newField =
          new PartitionField(
              newField.sourceId(), newField.fieldId(), partitionName, newField.transform());
    }

    checkForRedundantAddedPartitions(newField);
    transformToAddedField.put(validationKey, newField);

    PartitionField existingField = nameToField.get(newField.name());
    if (existingField != null && !deletes.contains(existingField.fieldId())) {
      if (isVoidTransform(existingField)) {
        // rename the old deleted field that is being replaced by the new field
        renameField(existingField.name(), existingField.name() + "_" + existingField.fieldId());
      } else {
        throw new IllegalArgumentException(
            String.format("Cannot add duplicate partition field name: %s", name));
      }
    } else if (existingField != null && deletes.contains(existingField.fieldId())) {
      renames.put(existingField.name(), existingField.name() + "_" + existingField.fieldId());
    }

    nameToAddedField.put(newField.name(), newField);

    adds.add(newField);

    return this;
  }

  @Override
  public BaseUpdatePartitionSpec removeField(String name) {
    PartitionField alreadyAdded = nameToAddedField.get(name);
    Preconditions.checkArgument(
        alreadyAdded == null, "Cannot delete newly added field: %s", alreadyAdded);

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Remove the existing field (removeField) in the same update before adding the new one, or pick a distinct name
2. Check current spec field names before calling addField
3. If the old field is a void transform, the add will auto-rename it — verify that rename is acceptable

Example fix

// before
specUpdate.addField("days(ts)", Transforms.day(tsType)); // name already exists
// after
if (table.spec().fields().stream().noneMatch(f -> f.name().equals("days_ts_v2"))) {
  specUpdate.addField("days_ts_v2", Transforms.day(tsType));
}

Set<String> names = table.spec().fields().stream().map(PartitionField::name).collect(Collectors.toSet());
if (names.contains(newName)) { throw new IllegalArgumentException("duplicate partition field: " + newName); }

catch (IllegalArgumentException e) { /* use a unique field name or skip the add */ }

When it happens

Trigger: addField("days(ts)", ...) when a live field named days(ts) already exists and was not removed in the same update; re-adding a deleted field's name without the void-transform rename path applying.

Common situations: Migrating specs where a field was dropped and re-added in separate transactions; two engineers independently adding the same partition field; automated schema-evolution jobs replaying duplicate addField calls.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Cannot update partition spec with unknown transform:
• Cannot find unpartitioned spec in specs:
• Unsorted order ID must be 0
• Sort order ID 0 is reserved for unsorted order
• this.getClass().getName() + " doesn't implement addNonDefaul

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/59c9a95c3466bd8c. Report an issue: GitHub.