apache/iceberg · error · IllegalArgumentException

Cannot find field %s in %s

Error message

Cannot find field %s in %s

What it means

StructProjection projects a source StructType down to a set of projected fields. During create(), each projected field is looked up in the source struct; if a required (or non-allowMissing) projected field has no matching source field, this IllegalArgumentException is thrown. It means the projection schema and the struct schema are out of sync.

Source

Thrown at api/src/main/java/org/apache/iceberg/util/StructProjection.java:173

              Preconditions.checkArgument(
                  elementProjectable,
                  "Cannot project a partial list element struct. Trying to project %s out of %s",
                  projectedField,
                  dataField);

              nestedProjections[pos] = null;
              break;
            default:
              nestedProjections[pos] = null;
          }
        }
      }

      if (!found && projectedField.isOptional() && allowMissing) {
        positionMap[pos] = -1;
        nestedProjections[pos] = null;
      } else if (!found) {
        throw new IllegalArgumentException(
            String.format("Cannot find field %s in %s", projectedField, structType));
      }
    }
  }

  public int projectedFields() {
    return (int) Ints.asList(positionMap).stream().filter(val -> val != -1).count();
  }

  public StructProjection wrap(StructLike newStruct) {
    this.struct = newStruct;
    return this;
  }

  public StructProjection copyFor(StructLike newStruct) {
    return new StructProjection(type, positionMap, nestedProjections).wrap(newStruct);
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Rebuild the StructProjection from the current source struct type instead of a stale/cached one
2. Check that every field name/id in the projected schema exists in the source schema (Types.StructType.field(name) != null) before creating the projection
3. If optional projected fields may legitimately be absent, use the create overload with allowMissing=true
4. Align case sensitivity: use the same case-sensitivity setting as the schema or case-insensitive lookup

Example fix

// before
StructProjection.create(oldTableSchema, projectedType);
// after
Types.StructType current = table.schema().asStruct();
Types.NestedField f = current.field(projectedFieldId);
if (f == null) { throw new IllegalStateException("schema changed; rebuild projection"); }
StructProjection.create(current, projectedType);

boolean projectable = projectedType.fields().stream()
    .allMatch(f -> f.isOptional() || sourceType.field(f.name()) != null);
if (!projectable) { throw new IllegalStateException("projected schema fields missing from source"); }

try {
  StructProjection p = StructProjection.create(sourceType, ids);
} catch (IllegalArgumentException e) {
  // rebuild projection from the current schema or fall back to full row
}

When it happens

Trigger: Calling StructProjection.create(sourceType, projectedType or ids) when the projected schema contains a field name/id that does not exist in the source struct type, and the field is required or allowMissing=false.

Common situations: Evolving a table (renaming/dropping a field) while cached projections or scan schemas still reference the old field; passing a projection built from one table's schema to rows of another schema; case-sensitivity mismatches in field names.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Unsorted order ID must be 0
• Field + field.name() + not found in source schema
• Unknown field ordinal: ${basePos}
• Invalid projection for field ${field.name()}: ${e.getMessage
• Invalid projection for field <field.name()>: <e.getMessage()

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/1d1af7020e4b6a15. Report an issue: GitHub.