{"record":{"id":"640f57d8405aecf9","repo":"apache/iceberg","slug":"cannot-add-column-s-since-setting-default-values-640f57","errorCode":null,"errorMessage":"Cannot add column %s since setting default values in Spark is currently unsupported","messagePattern":"Cannot add column (.+?) since setting default values in Spark is currently unsupported","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java","lineNumber":242,"sourceCode":"      TableChange.After after = (TableChange.After) update.position();\n      String referenceField = peerName(update.fieldNames(), after.column());\n      pendingUpdate.moveAfter(DOT.join(update.fieldNames()), referenceField);\n\n    } else if (update.position() instanceof TableChange.First) {\n      pendingUpdate.moveFirst(DOT.join(update.fieldNames()));\n\n    } else {\n      throw new IllegalArgumentException(\"Unknown position for reorder: \" + update.position());\n    }\n  }\n\n  private static void apply(UpdateSchema pendingUpdate, TableChange.AddColumn add) {\n    Preconditions.checkArgument(\n        add.isNullable(),\n        \"Incompatible change: cannot add required column: %s\",\n        leafName(add.fieldNames()));\n    if (add.defaultValue() != null) {\n      throw new UnsupportedOperationException(\n          String.format(\n              \"Cannot add column %s since setting default values in Spark is currently unsupported\",\n              leafName(add.fieldNames())));\n    }\n\n    Type type = SparkSchemaUtil.convert(add.dataType());\n    pendingUpdate.addColumn(\n        parentName(add.fieldNames()), leafName(add.fieldNames()), type, add.comment());\n\n    if (add.position() instanceof TableChange.After) {\n      TableChange.After after = (TableChange.After) add.position();\n      String referenceField = peerName(add.fieldNames(), after.column());\n      pendingUpdate.moveAfter(DOT.join(add.fieldNames()), referenceField);\n\n    } else if (add.position() instanceof TableChange.First) {\n      pendingUpdate.moveFirst(DOT.join(add.fieldNames()));\n\n    } else {","sourceCodeStart":224,"sourceCodeEnd":260,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java#L224-L260","documentation":"Thrown when a Spark ADD COLUMN change carries a default value. Setting column default values through Spark DDL is not supported by Iceberg's Spark3Util schema-change path, so this UnsupportedOperationException is raised.","triggerScenarios":"ALTER TABLE ... ADD COLUMN c INT DEFAULT 5 (or any AddColumn with defaultValue() != null) against an Iceberg table.","commonSituations":"DDL scripts generated for engines supporting defaults (e.g. newer Spark, Delta); migration tooling preserving DEFAULT clauses.","solutions":["Remove the DEFAULT clause and backfill values after adding the column","Set default values directly through Iceberg's UpdateSchema API (setDefault) instead of Spark DDL","Upgrade Iceberg if a newer release adds Spark default-value support"],"exampleFix":"-- before\nALTER TABLE t ADD COLUMN c INT DEFAULT 5;\n-- after\nALTER TABLE t ADD COLUMN c INT;\nUPDATE t SET c = 5 WHERE c IS NULL;","handlingStrategy":"validation","validationCode":"if (add.defaultValue() != null) { throw new IllegalArgumentException(\"Remove DEFAULT before adding the column: \" + String.join(\".\", add.fieldNames())); }","typeGuard":"boolean hasNoDefault(TableChange.AddColumn a) { return a.defaultValue() == null; }","tryCatchPattern":"try { spark.sql(ddl); } catch (UnsupportedOperationException e) { if (e.getMessage().contains(\"setting default values in Spark is currently unsupported\")) { // strip DEFAULT clause and backfill via UPDATE spark.sql(stripDefault(ddl)); } else throw e; }","preventionTips":["Strip DEFAULT clauses when generating Iceberg DDL","Backfill defaults with UPDATE statements after the column exists","Use Iceberg UpdateSchema.setDefault directly when defaults are required"],"tags":["spark","schema","default-value"],"backgroundTag":"unsupported-operation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}