apache/iceberg · error · ValidationException

Invalid primary key ' '. Column ' ' is not a physical…

Error message

Invalid primary key '%s'. Column '%s' is not a physical column.

What it means

validatePrimaryKey throws this ValidationException when a primary-key column exists but is not a physical column (e.g. a computed or metadata column). Iceberg primary keys (identifier fields) must map to physical table columns, so non-physical columns are rejected during toResolvedSchema conversion.

Solutions

1. Move the key expression into a real physical column and use that column in PRIMARY KEY.
2. Drop the computed column from the key and pick an underlying physical column.
3. If the expression is needed, materialize it at write time via an INSERT INTO ... SELECT that computes the column.

Example fix

// before
CREATE TABLE t (
  id BIGINT,
  name AS upper(id || '') ,
  PRIMARY KEY (name) NOT ENFORCED
);
// after
CREATE TABLE t (
  id BIGINT,
  key_name STRING,
  PRIMARY KEY (id) NOT ENFORCED
);

for (String name : pkColumns) {
  Column c = resolvedSchema.getColumn(name).orElseThrow();
  Preconditions.checkArgument(c.isPhysical(), "PK column %s must be physical", name);
}

When it happens

Trigger: Declaring PRIMARY KEY over a computed column (`name AS expr`) or metadata column (`ts TIMESTAMP(3) WITH LOCAL TIME ZONE METADATA FROM ...`) in Flink DDL for an Iceberg table.

Common situations: Developers model derived keys as computed columns, or migrate DDL from other connectors that allow virtual key columns; Iceberg rejects them because identifiers must be stored fields.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Invalid primary key ' '. Column ' ' is not a physical…
• Hash distribute rows by equality fields, even though
• Hash distribute rows by equality fields, even though
• Hash distribute rows by equality fields, even though
• Hash distribute rows by equality fields, even though

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/ab02e1e5a7e6461e. Report an issue: GitHub.

Appendix: source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java:365

    if (!duplicateColumns.isEmpty()) {
      throw new ValidationException(
          String.format(
              "Invalid primary key '%s'. A primary key must not contain duplicate columns. Found: %s",
              primaryKey.getName(), duplicateColumns));
    }

    for (String columnName : primaryKey.getColumns()) {
      Column column = columnsByNameLookup.get(columnName);
      if (column == null) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' does not exist.",
                primaryKey.getName(), columnName));
      }

      if (!column.isPhysical()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is not a physical column.",
                primaryKey.getName(), columnName));
      }

      final LogicalType columnType = column.getDataType().getLogicalType();
      if (columnType.isNullable()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is nullable.",
                primaryKey.getName(), columnName));
      }
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)