apache/iceberg · error · UnsupportedOperationException

Cannot add column since setting default values in Spark is…

Error message

Cannot add column %s since setting default values in Spark is currently unsupported

What it means

Thrown when a Spark ADD COLUMN change carries a default value. Setting column default values through Spark DDL is not supported by Iceberg's Spark3Util schema-change path, so this UnsupportedOperationException is raised.

Solutions

1. Remove the DEFAULT clause and backfill values after adding the column
2. Set default values directly through Iceberg's UpdateSchema API (setDefault) instead of Spark DDL
3. Upgrade Iceberg if a newer release adds Spark default-value support

Example fix

-- before
ALTER TABLE t ADD COLUMN c INT DEFAULT 5;
-- after
ALTER TABLE t ADD COLUMN c INT;
UPDATE t SET c = 5 WHERE c IS NULL;

if (add.defaultValue() != null) { throw new IllegalArgumentException("Remove DEFAULT before adding the column: " + String.join(".", add.fieldNames())); }

boolean hasNoDefault(TableChange.AddColumn a) { return a.defaultValue() == null; }

try { spark.sql(ddl); } catch (UnsupportedOperationException e) { if (e.getMessage().contains("setting default values in Spark is currently unsupported")) { // strip DEFAULT clause and backfill via UPDATE spark.sql(stripDefault(ddl)); } else throw e; }

When it happens

Trigger: ALTER TABLE ... ADD COLUMN c INT DEFAULT 5 (or any AddColumn with defaultValue() != null) against an Iceberg table.

Common situations: DDL scripts generated for engines supporting defaults (e.g. newer Spark, Delta); migration tooling preserving DEFAULT clauses.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Cannot add column since setting default values in Spark is…
• Cannot add column since setting default values in Spark is…
• Cannot add column since setting default values in Spark is…
• Cannot convert unsupported type to Spark
• Cannot drop identifier fields in non-Iceberg table: $table

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/640f57d8405aecf9. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java:242

      TableChange.After after = (TableChange.After) update.position();
      String referenceField = peerName(update.fieldNames(), after.column());
      pendingUpdate.moveAfter(DOT.join(update.fieldNames()), referenceField);

    } else if (update.position() instanceof TableChange.First) {
      pendingUpdate.moveFirst(DOT.join(update.fieldNames()));

    } else {
      throw new IllegalArgumentException("Unknown position for reorder: " + update.position());
    }
  }

  private static void apply(UpdateSchema pendingUpdate, TableChange.AddColumn add) {
    Preconditions.checkArgument(
        add.isNullable(),
        "Incompatible change: cannot add required column: %s",
        leafName(add.fieldNames()));
    if (add.defaultValue() != null) {
      throw new UnsupportedOperationException(
          String.format(
              "Cannot add column %s since setting default values in Spark is currently unsupported",
              leafName(add.fieldNames())));
    }

    Type type = SparkSchemaUtil.convert(add.dataType());
    pendingUpdate.addColumn(
        parentName(add.fieldNames()), leafName(add.fieldNames()), type, add.comment());

    if (add.position() instanceof TableChange.After) {
      TableChange.After after = (TableChange.After) add.position();
      String referenceField = peerName(add.fieldNames(), after.column());
      pendingUpdate.moveAfter(DOT.join(add.fieldNames()), referenceField);

    } else if (add.position() instanceof TableChange.First) {
      pendingUpdate.moveFirst(DOT.join(add.fieldNames()));

    } else {

View on GitHub (pinned to 86d9c8fc54)