{"record":{"id":"1fa6b288cd87ac69","repo":"apache/iceberg","slug":"cannot-find-projected-field-1fa6b2","errorCode":null,"errorMessage":"Cannot find projected field: ","messagePattern":"Cannot find projected field: ","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/avro/SupportsIndexProjection.java","lineNumber":53,"sourceCode":"  }\n\n  /** Base constructor for building the type mapping */\n  protected SupportsIndexProjection(Types.StructType baseType, Types.StructType projectionType) {\n    List<Types.NestedField> allFields = baseType.fields();\n    List<Types.NestedField> fields = projectionType.fields();\n\n    this.fromProjectionPos = new int[fields.size()];\n    for (int i = 0; i < fromProjectionPos.length; i += 1) {\n      boolean found = false;\n      for (int j = 0; j < allFields.size(); j += 1) {\n        if (fields.get(i).fieldId() == allFields.get(j).fieldId()) {\n          found = true;\n          fromProjectionPos[i] = j;\n        }\n      }\n\n      if (!found) {\n        throw new IllegalArgumentException(\"Cannot find projected field: \" + fields.get(i));\n      }\n    }\n  }\n\n  /** Constructor with a precomputed position mapping */\n  protected SupportsIndexProjection(int[] fromProjectionPos) {\n    this.fromProjectionPos = fromProjectionPos;\n  }\n\n  /** Copy constructor */\n  protected SupportsIndexProjection(SupportsIndexProjection toCopy) {\n    this.fromProjectionPos = toCopy.fromProjectionPos;\n  }\n\n  protected abstract <T> T internalGet(int pos, Class<T> javaClass);\n\n  protected abstract <T> void internalSet(int pos, T value);\n","sourceCodeStart":35,"sourceCodeEnd":71,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/avro/SupportsIndexProjection.java#L35-L71","documentation":"Thrown by SupportsIndexProjection when building an index-based projection mapping: a projected field from the requested schema cannot be located in the source Avro schema fields by field ID. The library cannot construct the position array used to project values, so record reading cannot proceed.","triggerScenarios":"Reading an Avro data or manifest file whose schema lacks a field that the table schema (or a projection of it) declares; typically a stale data file written by an older schema after the table added a required field.","commonSituations":"Schema evolution (field added as required) combined with old data files; custom Avro schemas missing field IDs; manually constructed read schemas where field-id properties were stripped by tools.","solutions":["Ensure all Avro schema fields carry the correct 'field-id' (iceberg-field-id) attribute matching the Iceberg schema","Verify the data file schema is in sync with the current table schema; rewrite/repair stale files","Make newly added fields optional so missing values can be filled with null instead of failing projection","Regenerate the read schema from the table schema via AvroSchemaUtil rather than hand-editing"],"exampleFix":"// before\nSchema readSchema = new Schema(Optional.of(\"t\"), required(5, \"notes\", StringType.get()));\n// after\nSchema readSchema = new Schema(Optional.of(\"t\"), optional(5, \"notes\", StringType.get()));","handlingStrategy":"validation","validationCode":"Set<Integer> ids = table.schema().columns().stream().map(Types.NestedField::fieldId).collect(Collectors.toSet());\nSchema fileSchema = AvroSchemaUtil.readSchema(file).orElseThrow();\nfor (Types.NestedField f : readSchema.columns()) {\n  Object id = fileSchema.asStruct().field(f.name()) != null ? f.fieldId() : null;\n  if (id == null || !ids.contains(f.fieldId())) throw new IllegalStateException(\"field-id missing for \" + f.name());\n}","typeGuard":"boolean hasField(Schema schema, Types.NestedField f) { return Arrays.stream(schema.getFields().toArray(new Field[0])).anyMatch(x -> x.getProp(AvroSchemaUtil.FIELD_ID_PROP) != null); }","tryCatchPattern":"try (CloseableIterable<Record> rows = Avro.read(file).project(table.schema()).build()) { rows.forEach(...); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith(\"Cannot find projected field\")) { /* resync schema / rewrite files */ } else throw e; }","preventionTips":["Always keep field-id properties in Avro schemas produced or transformed by external tools","Prefer optional columns for schema evolution","Rewrite old files after required-column additions","Use AvroSchemaUtil.convert from the table schema instead of hand-built schemas"],"tags":["avro","schema-evolution","projection"],"backgroundTag":"schema-validation-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}