apache/iceberg · error · ValidationException

Invalid primary key ' '. Column ' ' does not exist.

Error message

Invalid primary key '%s'. Column '%s' does not exist.

What it means

FlinkSchemaUtil.validatePrimaryKey throws this ValidationException while converting a Flink ResolvedSchema into an Iceberg schema when a primary-key constraint references a column name that is not present in the schema's columns. The conversion cannot build an Iceberg identifier field for a nonexistent column, so it fails fast. It is raised from toResolvedSchema during table schema conversion.

Solutions

1. Fix the PRIMARY KEY clause to reference an existing column name exactly as declared.
2. Check the schema column names with `table.getResolvedSchema().getColumnNames()` or DESCRIBE before declaring the key.
3. If the key column is computed, materialize it as a physical column instead.

Example fix

// before
CREATE TABLE iceberg_table (
  id BIGINT,
  data STRING,
  PRIMARY KEY (uid) NOT ENFORCED
) WITH (...);
// after
CREATE TABLE iceberg_table (
  id BIGINT,
  data STRING,
  PRIMARY KEY (id) NOT ENFORCED
) WITH (...);

ResolvedSchema rs = table.getResolvedSchema();
Set<String> cols = rs.getColumnNames().stream().collect(Collectors.toSet());
if (!cols.containsAll(pkColumns)) {
  throw new IllegalArgumentException("PK columns missing from schema: " + pkColumns);
}

When it happens

Trigger: Calling FlinkSchemaUtil.toResolvedSchema (or creating an Iceberg-backed Flink table via TableEnvironment) where the DDL's PRIMARY KEY (col) clause names a column that does not exist in the column list — typically a typo or a column defined only in a computed/virtual expression key.

Common situations: Typos in PRIMARY KEY clauses after renaming a column; copying DDL between tables; defining a primary key over a metadata or computed column that exists in the Flink catalog but not as a physical schema column.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Creating table with computed columns is not supported yet.
• Creating table with watermark specs is not supported yet.
• Hash distribute rows by equality fields, even though
• Hash distribute rows by equality fields, even though
• Hash distribute rows by equality fields, even though

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/4e7e363c9106fe28. Report an issue: GitHub.

Appendix: source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java:358

    final Map<String, Column> columnsByNameLookup =
        columns.stream().collect(Collectors.toMap(Column::getName, Function.identity()));

    final Set<String> duplicateColumns =
        primaryKey.getColumns().stream()
            .filter(name -> Collections.frequency(primaryKey.getColumns(), name) > 1)
            .collect(Collectors.toSet());

    if (!duplicateColumns.isEmpty()) {
      throw new ValidationException(
          String.format(
              "Invalid primary key '%s'. A primary key must not contain duplicate columns. Found: %s",
              primaryKey.getName(), duplicateColumns));
    }

    for (String columnName : primaryKey.getColumns()) {
      Column column = columnsByNameLookup.get(columnName);
      if (column == null) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' does not exist.",
                primaryKey.getName(), columnName));
      }

      if (!column.isPhysical()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is not a physical column.",
                primaryKey.getName(), columnName));
      }

      final LogicalType columnType = column.getDataType().getLogicalType();
      if (columnType.isNullable()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is nullable.",
                primaryKey.getName(), columnName));

View on GitHub (pinned to 86d9c8fc54)