apache/iceberg · error · org.apache.flink.table.api.ValidationException

Invalid primary key '%s'. Column '%s' is not a physical colu

Error message

Invalid primary key '%s'. Column '%s' is not a physical column.

What it means

A primary key column must be a physical column of the table; computed/metadata columns cannot participate in a key. validatePrimaryKey throws this ValidationException when the referenced column is a computed (non-physical) column.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java:365

    if (!duplicateColumns.isEmpty()) {
      throw new ValidationException(
          String.format(
              "Invalid primary key '%s'. A primary key must not contain duplicate columns. Found: %s",
              primaryKey.getName(), duplicateColumns));
    }

    for (String columnName : primaryKey.getColumns()) {
      Column column = columnsByNameLookup.get(columnName);
      if (column == null) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' does not exist.",
                primaryKey.getName(), columnName));
      }

      if (!column.isPhysical()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is not a physical column.",
                primaryKey.getName(), columnName));
      }

      final LogicalType columnType = column.getDataType().getLogicalType();
      if (columnType.isNullable()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is nullable.",
                primaryKey.getName(), columnName));
      }
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

Make the key column a regular physical column (materialize the value on write).
Create a physical column that stores the computed expression's value and key on that instead.
Remove the computed column from the primary key.

Example fix

// before
ts AS PROCTIME(), PRIMARY KEY (ts) NOT ENFORCED
// after
ts TIMESTAMP(3), PRIMARY KEY (ts) NOT ENFORCED

Defensive patterns

Strategy: validation

Validate before calling

for (String col : primaryKey.getColumns()) {
  Column c = schema.getColumn(col);
  if (c != null && !c.isPhysical()) {
    throw new IllegalArgumentException("PK column must be physical: " + col);
  }
}

Type guard

null

Try / catch

try {
  FlinkSchemaUtil.toResolvedSchema(schema, partitionKeys, primaryKey);
} catch (ValidationException e) {
  LOG.error("PK on non-physical column: {}", e.getMessage());
  throw e;
}

Prevention

Only reference physical columns in PRIMARY KEY clauses.
Materialize needed computed values into physical columns before keying on them.
Review DDL for AS/METADATA columns when defining keys.

When it happens

Trigger: PRIMARY KEY clause naming a computed column (AS expr) or a metadata column (column METADATA FROM ...), e.g. PRIMARY KEY (event_ts_computed) NOT ENFORCED where event_ts_computed is defined with AS.

Common situations: Users trying to key tables on derived/computed values; DDL copied from sources with virtual columns; Flink SQL with generated columns.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/52979042b1be0677. Report an issue: GitHub.