{"record":{"id":"81b870601cf5865b","repo":"apache/iceberg","slug":"invalid-projection-for-field-field-name-e-ge","errorCode":null,"errorMessage":"Invalid projection for field <field.name()>: <e.getMessage()>","messagePattern":"Invalid projection for field <field\\.name\\(\\)>: <e\\.getMessage\\(\\)>","errorType":"exception","errorClass":"java.lang.IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/PruneColumnsWithoutReordering.java","lineNumber":138,"sourceCode":"      if (filterRefs.contains(field.fieldId())) {\n        return field.type();\n      }\n      return null;\n    }\n\n    int fieldIndex = requestedStruct.fieldIndex(field.name());\n    StructField requestedField = requestedStruct.fields()[fieldIndex];\n\n    Preconditions.checkArgument(\n        requestedField.nullable() || field.isRequired(),\n        \"Cannot project an optional field as non-null: %s\",\n        field.name());\n\n    this.current = requestedField.dataType();\n    try {\n      return fieldResult.get();\n    } catch (IllegalArgumentException e) {\n      throw new IllegalArgumentException(\n          \"Invalid projection for field \" + field.name() + \": \" + e.getMessage(), e);\n    } finally {\n      this.current = requestedStruct;\n    }\n  }\n\n  @Override\n  public Type list(Types.ListType list, Supplier<Type> elementResult) {\n    Preconditions.checkArgument(current instanceof ArrayType, \"Not an array: %s\", current);\n    ArrayType requestedArray = (ArrayType) current;\n\n    Preconditions.checkArgument(\n        requestedArray.containsNull() || !list.isElementOptional(),\n        \"Cannot project an array of optional elements as required elements: %s\",\n        requestedArray);\n\n    this.current = requestedArray.elementType();\n    try {","sourceCodeStart":120,"sourceCodeEnd":156,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/PruneColumnsWithoutReordering.java#L120-L156","documentation":"PruneColumnsWithoutReordering.field projects a requested field type onto the current struct type during column pruning. If computing the projection throws IllegalArgumentException, it is re-wrapped with the field name: 'Invalid projection for field <name>: <msg>', giving context about which field failed.","triggerScenarios":"Schema pruning encounters a requested type incompatible with the table's current type for a given field (e.g. mismatched struct/array/map shapes after schema evolution or incompatible requested schema).","commonSituations":"Evolving a column's type in the table after queries/plans were built against the old schema; requesting a projection schema that diverges from the table schema; partition-transform mismatches.","solutions":["Read the inner message for the actual type mismatch and align the requested schema with the table's current schema","Refresh cached tables/dataframes so plans use the latest table schema","Re-check schema evolution changes that altered this field's type","Report upstream if the mismatch comes from Iceberg's own pruning with identical schemas"],"exampleFix":"// before\nspark.table(\"t\").select(\"evolved_struct.newly_renamed_field\")\n// after\nspark.table(\"t\")            // refresh plan against current schema\n  .select(\"evolved_struct.newly_renamed_field\")","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  df = spark.read().format(\"iceberg\").load(path);\n} catch (IllegalArgumentException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Invalid projection for field\")) {\n    log.error(\"Schema drift on field; refresh schema. {}\", e.getMessage(), e);\n    spark.catalog().refreshTable(path);\n  } else throw e;\n}","preventionTips":["spark.catalog().refreshTable after schema evolution","Avoid long-lived cached DataFrames across schema changes","Test projections after type-evolution operations"],"tags":["spark","schema","projection"],"backgroundTag":"schema-validation-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}