{"record":{"id":"cfe76a8e834e7e38","repo":"apache/iceberg","slug":"missing-orc-reader-for-field-s-s","errorCode":null,"errorMessage":"Missing ORC reader for field %s (%s)","messagePattern":"Missing ORC reader for field (.+?) \\((.+?)\\)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"orc/src/main/java/org/apache/iceberg/orc/OrcValueReaders.java","lineNumber":223,"sourceCode":"        } else if (idToConstant.containsKey(field.fieldId())) {\n          this.isConstantOrMetadataField[pos] = true;\n          this.readers[pos] = constants(idToConstant.get(field.fieldId()));\n        } else if (field.equals(MetadataColumns.ROW_POSITION)) {\n          this.isConstantOrMetadataField[pos] = true;\n          this.readers[pos] = new RowPositionReader();\n        } else if (field.equals(MetadataColumns.IS_DELETED)) {\n          this.isConstantOrMetadataField[pos] = true;\n          this.readers[pos] = constants(false);\n        } else if (fileReader != null) {\n          this.isConstantOrMetadataField[pos] = false;\n          this.orcFieldIndex[pos] = fieldIdToOrcIndex.getOrDefault(field.fieldId(), -1);\n          this.readers[pos] = fileReader;\n        } else if (MetadataColumns.isMetadataColumn(field.name())\n            || field.type().typeId() == Type.TypeID.UNKNOWN) {\n          this.isConstantOrMetadataField[pos] = true;\n          this.readers[pos] = constants(null);\n        } else {\n          throw new IllegalArgumentException(\n              String.format(\"Missing ORC reader for field %s (%s)\", field.name(), field.fieldId()));\n        }\n      }\n    }\n\n    private Map<Integer, Integer> buildFieldIdToOrcIndex(TypeDescription orcType) {\n      List<TypeDescription> children = orcType.getChildren();\n      Map<Integer, Integer> mapping = Maps.newHashMap();\n      for (int i = 0; i < children.size(); i++) {\n        mapping.put(ORCSchemaUtil.fieldId(children.get(i)), i);\n      }\n\n      return mapping;\n    }\n\n    private Map<Integer, OrcValueReader<?>> readersByFieldId(\n        TypeDescription orcType, List<OrcValueReader<?>> readerList) {\n      List<TypeDescription> children = orcType.getChildren();","sourceCodeStart":205,"sourceCodeEnd":241,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/orc/src/main/java/org/apache/iceberg/orc/OrcValueReaders.java#L205-L241","documentation":"StructReader builds a per-position array of ORC value readers for each field of an Iceberg struct. When a projected field is neither a metadata column, an UNKNOWN-typed field, nor covered by a reader produced from the ORC file schema, it throws IllegalArgumentException. This indicates the ORC file schema cannot satisfy the requested read schema.","triggerScenarios":"Calling ORC reads (e.g. via OrcBatchReader/Spark/Flink ORC scans) with an expected schema containing a field whose fieldId has no corresponding ORC column and is not a metadata column or UNKNOWN type — typically a schema-evolution mismatch where the file predates the field.","commonSituations":"Reading ORC files written before a column was added to the table schema; a wrong/renamed write.schema vs read schema; using a custom ORC writer that did not embed Iceberg field IDs; case where nameMapping is absent so field-ID mapping fails.","solutions":["Ensure the ORC file was written by Iceberg with field IDs embedded in the ORC type description, or set table property schema.name-mapping.default so field IDs can be resolved by name.","Check that the read schema's field IDs exist in the file's ORC schema; re-read with the file-compatible schema or rewrite the files via Iceberg rewrite/compact.","Update to a recent Iceberg version if the file was written by a much older writer with a different ORC field-ID convention."],"exampleFix":"// before: reading old files with new schema, no name mapping\ntable.updateProperties().set(TableProperties.DEFAULT_NAME_MAPPING, \"{\\\"type\\\":\\\"struct\\\",\\\"fields\\\":[...id...\\\"]}\").commit();\n// after: name mapping present, field IDs resolvable, read succeeds","handlingStrategy":"validation","validationCode":"// Verify every read-schema field exists in the ORC file schema (or has a name mapping)\nNameMapping mapping = Optional.ofNullable(table.properties().get(TableProperties.DEFAULT_NAME_MAPPING))\n    .map(Mappings::fromJson).orElse(null);\nfor (Types.NestedField f : expectedSchema.columns()) {\n  if (MetadataColumns.metadataFieldIds().contains(f.fieldId())) continue;\n  if (MappingCreateVisitor.find(mapping, f.fieldId()) == null && orcSchema.findField(f.fieldId()) == null) {\n    throw new IllegalArgumentException(\"Field \" + f.fieldId() + \" missing from ORC file\");\n  }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always write ORC via Iceberg so field IDs are embedded in the ORC schema","Set schema.name-mapping.default when importing external ORC files","Check field IDs after schema evolution; rewrite old files before dropping name mappings"],"tags":["orc","schema-evolution","read-path"],"backgroundTag":"schema-validation-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}