apache/iceberg · error · org.apache.flink.table.api.ValidationException

Invalid primary key ' '. Column ' ' is not a physical…

Error message

Invalid primary key '%s'. Column '%s' is not a physical column.

What it means

validatePrimaryKey requires every primary key column to be a physical column. If the resolved column is a computed/metadata column (column.isPhysical() false), it throws ValidationException. Iceberg identifiers must map to physical table columns.

Solutions

Remove the computed column from the PRIMARY KEY and key on physical columns only.
If a stable identifier is needed, materialize the computed value into a physical column and use that.
Validate the key columns are physical before building the schema programmatically.

Example fix

// before
id BIGINT, ts AS CURRENT_TIMESTAMP, PRIMARY KEY (id, ts) NOT ENFORCED
// after
id BIGINT, PRIMARY KEY (id) NOT ENFORCED

Defensive patterns

Strategy: validation

Validate before calling

for (String c : primaryKey.getColumns()) {
  Column col = schema.getColumn(c).orElse(null);
  if (col == null || !col.isPhysical()) {
    throw new IllegalArgumentException("PK must reference physical columns, got: " + c);
  }
}

Try / catch

try {
  ResolvedSchema rs = FlinkSchemaUtil.toResolvedSchema(schema);
} catch (ValidationException e) {
  // message names the non-physical column; remove it from the key
}

Prevention

Never include computed (AS ...) or metadata columns in primary keys.
Materialize needed computed values into physical columns if they must be keyed.
Document that Iceberg identifiers map to physical columns only.

When it happens

Trigger: toResolvedSchema with a primary key that includes a computed column (e.g. AS ... expression column) or a metadata column, as in PRIMARY KEY (id, computed_ts).

Common situations: Including computed/generated columns in keys when porting DDL from other engines; referencing watermark or metadata columns in the key.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/43b86d6b70995ce0. Report an issue: GitHub.

Appendix: source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java:365

    if (!duplicateColumns.isEmpty()) {
      throw new ValidationException(
          String.format(
              "Invalid primary key '%s'. A primary key must not contain duplicate columns. Found: %s",
              primaryKey.getName(), duplicateColumns));
    }

    for (String columnName : primaryKey.getColumns()) {
      Column column = columnsByNameLookup.get(columnName);
      if (column == null) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' does not exist.",
                primaryKey.getName(), columnName));
      }

      if (!column.isPhysical()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is not a physical column.",
                primaryKey.getName(), columnName));
      }

      final LogicalType columnType = column.getDataType().getLogicalType();
      if (columnType.isNullable()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is nullable.",
                primaryKey.getName(), columnName));
      }
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)