apache/iceberg · error · java.lang.UnsupportedOperationException

Cannot add column since setting default values in Spark is…

Error message

Cannot add column %s since setting default values in Spark is currently unsupported

What it means

Iceberg supports column default values, but the Spark DDL integration in Spark3Util does not: when an AddColumn change carries a non-null default value it throws UnsupportedOperationException because Spark cannot yet express the default in a way this path supports.

Solutions

Remove the DEFAULT clause from the ADD COLUMN statement and set defaults afterwards via a write or update path
Issue the add-column without a default, then apply the default using an Iceberg API that supports it outside this Spark path
Upgrade Iceberg — newer versions may add default-value support in Spark DDL conversion

Example fix

// before
ALTER TABLE t ADD COLUMN c INT DEFAULT 5;
// after
ALTER TABLE t ADD COLUMN c INT;
-- then backfill/apply default separately, e.g. UPDATE or rewrite defaults via Iceberg API

Defensive patterns

Strategy: validation

Validate before calling

TableChange.AddColumn add = ...;
if (add.defaultValue() != null) {
  throw new IllegalArgumentException("Strip DEFAULT before ADD COLUMN via Spark3Util: " + add);
}

Try / catch

try {
  Spark3Util.applySchemaChanges(table, changes);
} catch (UnsupportedOperationException e) {
  if (e.getMessage().contains("default values")) {
    // retry with defaultValue stripped / apply defaults separately
  }
}

Prevention

Never emit DEFAULT clauses in ADD COLUMN targeting Iceberg tables via Spark
Apply defaults with a separate write/backfill step
Check Spark/Iceberg release notes for default-value DDL support before using it

When it happens

Trigger: ALTER TABLE ... ADD COLUMN c INT DEFAULT 5 (or equivalent AddColumn with defaultValue() != null) processed through Spark3Util.applySchemaChanges.

Common situations: Spark 3.x DDL with DEFAULT clauses added in newer Spark releases; SQL generators that emit defaults on every column; migrating DDL from engines that support defaults.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/dd1c87bb828c4e8a. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java:235

      TableChange.After after = (TableChange.After) update.position();
      String referenceField = peerName(update.fieldNames(), after.column());
      pendingUpdate.moveAfter(DOT.join(update.fieldNames()), referenceField);

    } else if (update.position() instanceof TableChange.First) {
      pendingUpdate.moveFirst(DOT.join(update.fieldNames()));

    } else {
      throw new IllegalArgumentException("Unknown position for reorder: " + update.position());
    }
  }

  private static void apply(UpdateSchema pendingUpdate, TableChange.AddColumn add) {
    Preconditions.checkArgument(
        add.isNullable(),
        "Incompatible change: cannot add required column: %s",
        leafName(add.fieldNames()));
    if (add.defaultValue() != null) {
      throw new UnsupportedOperationException(
          String.format(
              "Cannot add column %s since setting default values in Spark is currently unsupported",
              leafName(add.fieldNames())));
    }

    Type type = SparkSchemaUtil.convert(add.dataType());
    pendingUpdate.addColumn(
        parentName(add.fieldNames()), leafName(add.fieldNames()), type, add.comment());

    if (add.position() instanceof TableChange.After) {
      TableChange.After after = (TableChange.After) add.position();
      String referenceField = peerName(add.fieldNames(), after.column());
      pendingUpdate.moveAfter(DOT.join(add.fieldNames()), referenceField);

    } else if (add.position() instanceof TableChange.First) {
      pendingUpdate.moveFirst(DOT.join(add.fieldNames()));

    } else {

View on GitHub (pinned to 86d9c8fc54)