apache/iceberg · error · IllegalArgumentException

Missing ORC reader for field

Error message

Missing ORC reader for field %s (%s)

What it means

StructReader builds a per-position array of ORC value readers for each field of an Iceberg struct. When a projected field is neither a metadata column, an UNKNOWN-typed field, nor covered by a reader produced from the ORC file schema, it throws IllegalArgumentException. This indicates the ORC file schema cannot satisfy the requested read schema.

Solutions

  1. Ensure the ORC file was written by Iceberg with field IDs embedded in the ORC type description, or set table property schema.name-mapping.default so field IDs can be resolved by name.
  2. Check that the read schema's field IDs exist in the file's ORC schema; re-read with the file-compatible schema or rewrite the files via Iceberg rewrite/compact.
  3. Update to a recent Iceberg version if the file was written by a much older writer with a different ORC field-ID convention.

Example fix

// before: reading old files with new schema, no name mapping
table.updateProperties().set(TableProperties.DEFAULT_NAME_MAPPING, "{\"type\":\"struct\",\"fields\":[...id...\"]}").commit();
// after: name mapping present, field IDs resolvable, read succeeds
Defensive patterns

Strategy: validation

Validate before calling

// Verify every read-schema field exists in the ORC file schema (or has a name mapping)
NameMapping mapping = Optional.ofNullable(table.properties().get(TableProperties.DEFAULT_NAME_MAPPING))
    .map(Mappings::fromJson).orElse(null);
for (Types.NestedField f : expectedSchema.columns()) {
  if (MetadataColumns.metadataFieldIds().contains(f.fieldId())) continue;
  if (MappingCreateVisitor.find(mapping, f.fieldId()) == null && orcSchema.findField(f.fieldId()) == null) {
    throw new IllegalArgumentException("Field " + f.fieldId() + " missing from ORC file");
  }
}

Prevention

When it happens

Trigger: Calling ORC reads (e.g. via OrcBatchReader/Spark/Flink ORC scans) with an expected schema containing a field whose fieldId has no corresponding ORC column and is not a metadata column or UNKNOWN type — typically a schema-evolution mismatch where the file predates the field.

Common situations: Reading ORC files written before a column was added to the table schema; a wrong/renamed write.schema vs read schema; using a custom ORC writer that did not embed Iceberg field IDs; case where nameMapping is absent so field-ID mapping fails.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/cfe76a8e834e7e38. Report an issue: GitHub.

Appendix: source

Thrown at orc/src/main/java/org/apache/iceberg/orc/OrcValueReaders.java:223

        } else if (idToConstant.containsKey(field.fieldId())) {
          this.isConstantOrMetadataField[pos] = true;
          this.readers[pos] = constants(idToConstant.get(field.fieldId()));
        } else if (field.equals(MetadataColumns.ROW_POSITION)) {
          this.isConstantOrMetadataField[pos] = true;
          this.readers[pos] = new RowPositionReader();
        } else if (field.equals(MetadataColumns.IS_DELETED)) {
          this.isConstantOrMetadataField[pos] = true;
          this.readers[pos] = constants(false);
        } else if (fileReader != null) {
          this.isConstantOrMetadataField[pos] = false;
          this.orcFieldIndex[pos] = fieldIdToOrcIndex.getOrDefault(field.fieldId(), -1);
          this.readers[pos] = fileReader;
        } else if (MetadataColumns.isMetadataColumn(field.name())
            || field.type().typeId() == Type.TypeID.UNKNOWN) {
          this.isConstantOrMetadataField[pos] = true;
          this.readers[pos] = constants(null);
        } else {
          throw new IllegalArgumentException(
              String.format("Missing ORC reader for field %s (%s)", field.name(), field.fieldId()));
        }
      }
    }

    private Map<Integer, Integer> buildFieldIdToOrcIndex(TypeDescription orcType) {
      List<TypeDescription> children = orcType.getChildren();
      Map<Integer, Integer> mapping = Maps.newHashMap();
      for (int i = 0; i < children.size(); i++) {
        mapping.put(ORCSchemaUtil.fieldId(children.get(i)), i);
      }

      return mapping;
    }

    private Map<Integer, OrcValueReader<?>> readersByFieldId(
        TypeDescription orcType, List<OrcValueReader<?>> readerList) {
      List<TypeDescription> children = orcType.getChildren();

View on GitHub (pinned to 86d9c8fc54)