What causes "Invalid primary key '%s'. Column '%s' does not exist." in apache/iceberg?

Correct the primary key column name to exactly match an existing column.

apache/iceberg · error · org.apache.flink.table.api.ValidationException

Invalid primary key '%s'. Column '%s' does not exist.

Error message

Invalid primary key '%s'. Column '%s' does not exist.

What it means

validatePrimaryKey looks up each primary key column in the resolved column map; if a key column name has no matching column, it throws ValidationException. The key references a column that does not exist in the schema.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java:358

    final Map<String, Column> columnsByNameLookup =
        columns.stream().collect(Collectors.toMap(Column::getName, Function.identity()));

    final Set<String> duplicateColumns =
        primaryKey.getColumns().stream()
            .filter(name -> Collections.frequency(primaryKey.getColumns(), name) > 1)
            .collect(Collectors.toSet());

    if (!duplicateColumns.isEmpty()) {
      throw new ValidationException(
          String.format(
              "Invalid primary key '%s'. A primary key must not contain duplicate columns. Found: %s",
              primaryKey.getName(), duplicateColumns));
    }

    for (String columnName : primaryKey.getColumns()) {
      Column column = columnsByNameLookup.get(columnName);
      if (column == null) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' does not exist.",
                primaryKey.getName(), columnName));
      }

      if (!column.isPhysical()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is not a physical column.",
                primaryKey.getName(), columnName));
      }

      final LogicalType columnType = column.getDataType().getLogicalType();
      if (columnType.isNullable()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is nullable.",
                primaryKey.getName(), columnName));

View on GitHub (pinned to 86d9c8fc54)

Solutions

Correct the primary key column name to exactly match an existing column.
Print the schema (DESCRIBE table / schema.toString()) and align the key names.
If the column was removed, redefine the primary key over existing columns.

Example fix

// before
PRIMARY KEY (userid) NOT ENFORCED
// after
PRIMARY KEY (user_id) NOT ENFORCED

Defensive patterns

Strategy: validation

Validate before calling

java.util.Set<String> names = schema.getColumns().stream().map(Column::getName).collect(java.util.stream.Collectors.toSet());
primaryKey.getColumns().forEach(c -> {
  if (!names.contains(c)) throw new IllegalArgumentException("PK column missing: " + c);
});

Try / catch

try {
  ResolvedSchema rs = FlinkSchemaUtil.toResolvedSchema(schema);
} catch (ValidationException e) {
  // message names the missing column; fix the key definition
}

Prevention

Cross-check PRIMARY KEY names against DESCRIBE/schema output before submitting DDL.
Update key definitions whenever columns are renamed or dropped.
Use a shared constant for column names in programmatic schema construction.

When it happens

Trigger: FlinkSchemaUtil.toResolvedSchema called with a primary key whose column list contains a name not present in the schema's columns — e.g. PRIMARY KEY (userid) when the column is named 'user_id'.

Common situations: Typos or case mismatches in DDL key columns; renaming a column without updating PRIMARY KEY; generated schemas where key column names come from config.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/b41b9d1d2b08210c. Report an issue: GitHub.