What causes "Invalid primary key '%s'. Column '%s' is nullable." in apache/iceberg?

Declare key columns with NOT NULL in the DDL, e.g. PRIMARY KEY (id) NOT ENFORCED with id NOT NULL.

apache/iceberg · error · org.apache.flink.table.api.ValidationException

Invalid primary key '%s'. Column '%s' is nullable.

Error message

Invalid primary key '%s'. Column '%s' is nullable.

What it means

Primary key columns must be NOT NULL in Flink; validatePrimaryKey rejects any key column whose LogicalType is nullable. This matches Iceberg's identifier-field requirements, which demand non-null key columns.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java:373

    for (String columnName : primaryKey.getColumns()) {
      Column column = columnsByNameLookup.get(columnName);
      if (column == null) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' does not exist.",
                primaryKey.getName(), columnName));
      }

      if (!column.isPhysical()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is not a physical column.",
                primaryKey.getName(), columnName));
      }

      final LogicalType columnType = column.getDataType().getLogicalType();
      if (columnType.isNullable()) {
        throw new ValidationException(
            String.format(
                "Invalid primary key '%s'. Column '%s' is nullable.",
                primaryKey.getName(), columnName));
      }
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

Declare key columns with NOT NULL in the DDL, e.g. PRIMARY KEY (id) NOT ENFORCED with id NOT NULL.
When building schemas programmatically, use the non-nullable LogicalType (e.g. DataTypes.BIGINT().NOT_NULL()) for key columns.
Use ResolvedSchema.newTableSchema-style helpers that enforce key nullability.

Example fix

// before
id BIGINT, PRIMARY KEY (id) NOT ENFORCED
// after
id BIGINT NOT NULL, PRIMARY KEY (id) NOT ENFORCED

Defensive patterns

Strategy: validation

Validate before calling

for (String col : primaryKey.getColumns()) {
  if (schema.getColumn(col) != null &&
      schema.getColumn(col).getDataType().getLogicalType().isNullable()) {
    throw new IllegalArgumentException("PK column must be NOT NULL: " + col);
  }
}

Type guard

null

Try / catch

try {
  FlinkSchemaUtil.toResolvedSchema(schema, partitionKeys, primaryKey);
} catch (ValidationException e) {
  LOG.error("PK column nullable: {}", e.getMessage());
  throw e;
}

Prevention

Always declare key columns NOT NULL in DDL and DataTypes....NOT_NULL() in code.
Remember Flink auto-marks declared PK columns NOT NULL only in DDL; programmatic schemas need explicit types.
Add a schema lint step validating key nullability before job submission.

When it happens

Trigger: Defining PRIMARY KEY on a column declared without NOT NULL in Flink DDL (Flink makes declared key columns non-null automatically only when declared together in some paths; programmatic constraints can bypass this), or building a ResolvedSchema where key columns retain nullable types.

Common situations: Constructing UniqueConstraint + ResolvedSchema in code without marking key columns non-null; DDL where the key column was added without NOT NULL; schema conversions from systems allowing nullable keys.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/88816a6895657d4a. Report an issue: GitHub.