apache/iceberg · error · IllegalArgumentException

Cannot find projected field:

Error message

Cannot find projected field: 

What it means

In GenericPartitionFieldSummary's projection-matching constructor, each field of the projected schema must be found among the partition-field-summary fields to record fromProjectionPos; when a projected field has no matching summary field, the constructor throws IllegalArgumentException('Cannot find projected field: ' + field). It means the requested projection is not a subset of the fields a partition stats summary actually provides (contains_null, contains_nan, lower_bound, upper_bound).

Source

Thrown at core/src/main/java/org/apache/iceberg/GenericPartitionFieldSummary.java:67

  public GenericPartitionFieldSummary(Schema avroSchema) {
    this.avroSchema = avroSchema;

    List<Types.NestedField> fields =
        AvroSchemaUtil.convert(avroSchema).asNestedType().asStructType().fields();
    List<Types.NestedField> allFields = PartitionFieldSummary.getType().fields();

    this.fromProjectionPos = new int[fields.size()];
    for (int i = 0; i < fromProjectionPos.length; i += 1) {
      boolean found = false;
      for (int j = 0; j < allFields.size(); j += 1) {
        if (fields.get(i).fieldId() == allFields.get(j).fieldId()) {
          found = true;
          fromProjectionPos[i] = j;
        }
      }

      if (!found) {
        throw new IllegalArgumentException("Cannot find projected field: " + fields.get(i));
      }
    }
  }

  public GenericPartitionFieldSummary(
      boolean containsNull, boolean containsNaN, ByteBuffer lowerBound, ByteBuffer upperBound) {
    this.avroSchema = AVRO_SCHEMA;
    this.containsNull = containsNull;
    this.containsNaN = containsNaN;
    this.lowerBound = ByteBuffers.toByteArray(lowerBound);
    this.upperBound = ByteBuffers.toByteArray(upperBound);
    this.fromProjectionPos = null;
  }

  // for testing backward compatibility only
  @VisibleForTesting
  GenericPartitionFieldSummary(boolean containsNull, ByteBuffer lowerBound, ByteBuffer upperBound) {
    this.avroSchema = AVRO_SCHEMA;

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Rebuild the projection from the current partition stats type (Partitioning.partitionType / stats schema) instead of a cached schema.
2. Regenerate partition statistics files after partition spec evolution so summaries match the projected fields.
3. Check field names in the error message against the summary struct fields (contains_null, contains_nan, lower_bound, upper_bound) for typos or drift.

Example fix

// before: stale cached projection
Types.StructType projection = cachedProjection; // from old partition type
// after: derive from current summary type
Types.StructType projection = partitionStatsType();
GenericPartitionFieldSummary summary = new GenericPartitionFieldSummary(projection, ...);

Types.StructType summaryType = partitionStatsSummaryType();
for (Types.NestedField f : projection.asStructType().fields()) {
  Preconditions.checkArgument(summaryType.field(f.name()) != null,
      "Field not in partition stats summary: %s", f.name());
}

boolean isProjectable(String fieldName) {
  return Set.of("contains_null", "contains_nan", "lower_bound", "upper_bound").contains(fieldName);
}

try {
  GenericPartitionFieldSummary s = new GenericPartitionFieldSummary(projection, fields...);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Cannot find projected field")) {
    // rebuild the projection from the current partition stats type
  } else throw e;
}

When it happens

Trigger: Constructing GenericPartitionFieldSummary with a projection schema that contains a field name not present in the partition statistics summary struct — e.g. projecting partition stats with fields from a different/older partition type, or misspelled field names in custom projection code.

Common situations: Reading partition statistics files written for an older partition spec against a table whose spec evolved; custom metadata-table code building projections by hand; Spark/Flink readers reusing stale projections after schema evolution.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Cannot find field %s in %s
• Unknown field ordinal: ${basePos}
• Unknown field ordinal:
• Unknown field ordinal:
• Unsupported file content type:

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/64190ab6cdb62c26. Report an issue: GitHub.