What causes "Invalid column: ${update.name()}" in apache/iceberg?

Ensure the table contains the column (recreate or ALTER TABLE ADD COLUMN).

apache/iceberg · error · IllegalArgumentException

Invalid column

Error message

Invalid column: ${update.name()}

What it means

SchemaUtils.updateTypes validates each requested UpdateType against the current Iceberg schema by looking up the column name. A null type means the column is absent, so an IllegalArgumentException 'Invalid column' is thrown before attempting the type update.

Solutions

Ensure the table contains the column (recreate or ALTER TABLE ADD COLUMN).
Align field naming between the topic schema and Iceberg schema (e.g. recreate table with auto-create from current schema).
Disable evolution for that run or pre-sync schemas before enabling auto-evolution.

Example fix

// before: topic has field 'ts' but table lacks it
ALTER TABLE db.target ADD COLUMN ts TIMESTAMP;
// after: updateTypes finds the column and proceeds

Defensive patterns

Strategy: validation

Validate before calling

UpdateType u = ...;
if (schema.findType(u.name()) == null) {
  throw new ConfigException("Column not in table: " + u.name());
}

Try / catch

try { SchemaUtils.updateTypes(schema, updates); }
catch (IllegalArgumentException e) { LOG.error("evolution failed: {}", e.getMessage()); throw e; }

Prevention

Keep table schema and topic schema in sync before enabling auto-evolution
Avoid renaming fields upstream without corresponding table changes
Log the schema diff when evolution errors occur

When it happens

Trigger: Schema evolution handler applying column type updates from the Kafka topic schema to the Iceberg table when the renamed/missing column is not present in the table schema.

Common situations: Auto-evolution enabled but table manually altered/recreated; field renamed upstream; case mismatch between Connect field names and Iceberg field names.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/00f5c6c31db6cef5. Report an issue: GitHub.

Appendix: source

Thrown at kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/SchemaUtils.java:143

    // apply the updates
    UpdateSchema updateSchema = table.updateSchema();
    addColumns.forEach(
        update -> updateSchema.addColumn(update.parentName(), update.name(), update.type()));
    updateTypes.forEach(update -> updateSchema.updateColumn(update.name(), update.type()));
    makeOptionals.forEach(update -> updateSchema.makeColumnOptional(update.name()));
    updateSchema.commit();
    LOG.info("Schema for table {} updated with new columns", table.name());
  }

  private static boolean columnExists(org.apache.iceberg.Schema schema, AddColumn update) {
    return schema.findType(update.key()) != null;
  }

  private static boolean typeMatches(org.apache.iceberg.Schema schema, UpdateType update) {
    Type type = schema.findType(update.name());
    if (type == null) {
      throw new IllegalArgumentException("Invalid column: " + update.name());
    }
    return type.typeId() == update.type().typeId();
  }

  private static boolean isOptional(org.apache.iceberg.Schema schema, MakeOptional update) {
    NestedField field = schema.findField(update.name());
    if (field == null) {
      throw new IllegalArgumentException("Invalid column: " + update.name());
    }
    return field.isOptional();
  }

  static PartitionSpec createPartitionSpec(
      org.apache.iceberg.Schema schema, List<String> partitionBy) {
    if (partitionBy.isEmpty()) {
      return PartitionSpec.unpartitioned();
    }

View on GitHub (pinned to 86d9c8fc54)