{"record":{"id":"dd1c87bb828c4e8a","repo":"apache/iceberg","slug":"cannot-add-column-s-since-setting-default-values-dd1c87","errorCode":null,"errorMessage":"Cannot add column %s since setting default values in Spark is currently unsupported","messagePattern":"Cannot add column (.+?) since setting default values in Spark is currently unsupported","errorType":"exception","errorClass":"java.lang.UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java","lineNumber":235,"sourceCode":"      TableChange.After after = (TableChange.After) update.position();\n      String referenceField = peerName(update.fieldNames(), after.column());\n      pendingUpdate.moveAfter(DOT.join(update.fieldNames()), referenceField);\n\n    } else if (update.position() instanceof TableChange.First) {\n      pendingUpdate.moveFirst(DOT.join(update.fieldNames()));\n\n    } else {\n      throw new IllegalArgumentException(\"Unknown position for reorder: \" + update.position());\n    }\n  }\n\n  private static void apply(UpdateSchema pendingUpdate, TableChange.AddColumn add) {\n    Preconditions.checkArgument(\n        add.isNullable(),\n        \"Incompatible change: cannot add required column: %s\",\n        leafName(add.fieldNames()));\n    if (add.defaultValue() != null) {\n      throw new UnsupportedOperationException(\n          String.format(\n              \"Cannot add column %s since setting default values in Spark is currently unsupported\",\n              leafName(add.fieldNames())));\n    }\n\n    Type type = SparkSchemaUtil.convert(add.dataType());\n    pendingUpdate.addColumn(\n        parentName(add.fieldNames()), leafName(add.fieldNames()), type, add.comment());\n\n    if (add.position() instanceof TableChange.After) {\n      TableChange.After after = (TableChange.After) add.position();\n      String referenceField = peerName(add.fieldNames(), after.column());\n      pendingUpdate.moveAfter(DOT.join(add.fieldNames()), referenceField);\n\n    } else if (add.position() instanceof TableChange.First) {\n      pendingUpdate.moveFirst(DOT.join(add.fieldNames()));\n\n    } else {","sourceCodeStart":217,"sourceCodeEnd":253,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java#L217-L253","documentation":"Iceberg supports column default values, but the Spark DDL integration in Spark3Util does not: when an AddColumn change carries a non-null default value it throws UnsupportedOperationException because Spark cannot yet express the default in a way this path supports.","triggerScenarios":"ALTER TABLE ... ADD COLUMN c INT DEFAULT 5 (or equivalent AddColumn with defaultValue() != null) processed through Spark3Util.applySchemaChanges.","commonSituations":"Spark 3.x DDL with DEFAULT clauses added in newer Spark releases; SQL generators that emit defaults on every column; migrating DDL from engines that support defaults.","solutions":["Remove the DEFAULT clause from the ADD COLUMN statement and set defaults afterwards via a write or update path","Issue the add-column without a default, then apply the default using an Iceberg API that supports it outside this Spark path","Upgrade Iceberg — newer versions may add default-value support in Spark DDL conversion"],"exampleFix":"// before\nALTER TABLE t ADD COLUMN c INT DEFAULT 5;\n// after\nALTER TABLE t ADD COLUMN c INT;\n-- then backfill/apply default separately, e.g. UPDATE or rewrite defaults via Iceberg API","handlingStrategy":"validation","validationCode":"TableChange.AddColumn add = ...;\nif (add.defaultValue() != null) {\n  throw new IllegalArgumentException(\"Strip DEFAULT before ADD COLUMN via Spark3Util: \" + add);\n}","typeGuard":null,"tryCatchPattern":"try {\n  Spark3Util.applySchemaChanges(table, changes);\n} catch (UnsupportedOperationException e) {\n  if (e.getMessage().contains(\"default values\")) {\n    // retry with defaultValue stripped / apply defaults separately\n  }\n}","preventionTips":["Never emit DEFAULT clauses in ADD COLUMN targeting Iceberg tables via Spark","Apply defaults with a separate write/backfill step","Check Spark/Iceberg release notes for default-value DDL support before using it"],"tags":["spark","default-value","unsupported-feature"],"backgroundTag":"operation-not-supported","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}