apache/iceberg · error · IllegalArgumentException

Cannot find field in

Error message

Cannot find field %s in %s

What it means

StructProjection projects a source StructType down to a set of projected fields. During create(), each projected field is looked up in the source struct; if a required (or non-allowMissing) projected field has no matching source field, this IllegalArgumentException is thrown. It means the projection schema and the struct schema are out of sync.

Solutions

1. Rebuild the StructProjection from the current source struct type instead of a stale/cached one
2. Check that every field name/id in the projected schema exists in the source schema (Types.StructType.field(name) != null) before creating the projection
3. If optional projected fields may legitimately be absent, use the create overload with allowMissing=true
4. Align case sensitivity: use the same case-sensitivity setting as the schema or case-insensitive lookup

Example fix

// before
StructProjection.create(oldTableSchema, projectedType);
// after
Types.StructType current = table.schema().asStruct();
Types.NestedField f = current.field(projectedFieldId);
if (f == null) { throw new IllegalStateException("schema changed; rebuild projection"); }
StructProjection.create(current, projectedType);

boolean projectable = projectedType.fields().stream()
    .allMatch(f -> f.isOptional() || sourceType.field(f.name()) != null);
if (!projectable) { throw new IllegalStateException("projected schema fields missing from source"); }

try {
  StructProjection p = StructProjection.create(sourceType, ids);
} catch (IllegalArgumentException e) {
  // rebuild projection from the current schema or fall back to full row
}

When it happens

Trigger: Calling StructProjection.create(sourceType, projectedType or ids) when the projected schema contains a field name/id that does not exist in the source struct type, and the field is required or allowMissing=false.

Common situations: Evolving a table (renaming/dropping a field) while cached projections or scan schemas still reference the old field; passing a projection built from one table's schema to rows of another schema; case-sensitivity mismatches in field names.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Field + field.name() + not found in source schema
• Invalid projection for field
• Invalid projection for field <field.name()>…
• Unknown field ordinal
• Unsorted order ID must be 0

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/1d1af7020e4b6a15. Report an issue: GitHub.

Appendix: source

Thrown at api/src/main/java/org/apache/iceberg/util/StructProjection.java:173

              Preconditions.checkArgument(
                  elementProjectable,
                  "Cannot project a partial list element struct. Trying to project %s out of %s",
                  projectedField,
                  dataField);

              nestedProjections[pos] = null;
              break;
            default:
              nestedProjections[pos] = null;
          }
        }
      }

      if (!found && projectedField.isOptional() && allowMissing) {
        positionMap[pos] = -1;
        nestedProjections[pos] = null;
      } else if (!found) {
        throw new IllegalArgumentException(
            String.format("Cannot find field %s in %s", projectedField, structType));
      }
    }
  }

  public int projectedFields() {
    return (int) Ints.asList(positionMap).stream().filter(val -> val != -1).count();
  }

  public StructProjection wrap(StructLike newStruct) {
    this.struct = newStruct;
    return this;
  }

  public StructProjection copyFor(StructLike newStruct) {
    return new StructProjection(type, positionMap, nestedProjections).wrap(newStruct);
  }

View on GitHub (pinned to 86d9c8fc54)