apache/iceberg · error · IllegalArgumentException

ORC schema does not contain Iceberg IDs

Error message

ORC schema does not contain Iceberg IDs

What it means

When converting an existing ORC file schema back into an Iceberg Schema, Iceberg IDs are recovered from ORC column attributes (iceberg.id). If, after visiting the whole ORC schema, no field carries those attributes, the resulting field list is empty and IllegalArgumentException is thrown. This protects against silently producing an empty schema for a non-Iceberg-written ORC file.

Source

Thrown at orc/src/main/java/org/apache/iceberg/orc/ORCSchemaUtil.java:293

   * @return the Iceberg schema
   * @throws IllegalArgumentException if ORC schema has no columns with Iceberg ID attributes
   */
  public static Schema convert(TypeDescription orcSchema) {
    List<TypeDescription> children = orcSchema.getChildren();
    List<String> childrenNames = orcSchema.getFieldNames();
    Preconditions.checkState(
        children.size() == childrenNames.size(),
        "Error in ORC file, children fields and names do not match.");

    OrcToIcebergVisitor schemaConverter = new OrcToIcebergVisitor();
    List<Types.NestedField> fields =
        OrcToIcebergVisitor.visitSchema(orcSchema, schemaConverter).stream()
            .filter(Optional::isPresent)
            .map(Optional::get)
            .collect(Collectors.toList());

    if (fields.isEmpty()) {
      throw new IllegalArgumentException("ORC schema does not contain Iceberg IDs");
    }

    return new Schema(fields);
  }

  /**
   * Converts an Iceberg schema to a corresponding ORC schema within the context of an existing ORC
   * file schema. This method also handles schema evolution from the original ORC file schema to the
   * given Iceberg schema. It builds the desired reader schema with the schema evolution rules and
   * pass that down to the ORC reader, which would then use its schema evolution to map that to the
   * writer’s schema.
   *
   * <p>Example: <code>
   * Iceberg writer                                        ORC writer
   * struct&lt;a (1): int, b (2): string&gt;                     struct&lt;a: int, b: string&gt;
   * struct&lt;a (1): struct&lt;b (2): string, c (3): date&gt;&gt;     struct&lt;a: struct&lt;b:string, c:date&gt;&gt;
   * </code> Iceberg reader ORC reader <code>
   * struct&lt;a (2): string, c (3): date&gt;                    struct&lt;b: string, c: date&gt;

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Rewrite the data with Iceberg's ORC writer so iceberg.id attributes are stamped into the ORC schema.
2. If consuming external ORC files, provide an explicit Iceberg schema / name mapping (e.g. via name-mapping) instead of inferring one from the ORC schema.
3. Verify the file was produced by the same Iceberg version lineage that sets ORC column attributes; re-export if the attributes are missing.

Example fix

// before (inferring schema from external ORC file)
Schema schema = ORCSchemaUtil.convert(orcReader.getSchema()); // IllegalArgumentException

// after (supply explicit schema / name mapping instead)
Schema schema = new Schema(
    Types.NestedField.required(1, "id", Types.LongType.get()),
    Types.NestedField.optional(2, "data", Types.StringType.get()));

TypeDescription orcSchema = orcReader.getSchema();
boolean hasIds = orcSchema.getChildren().stream()
    .anyMatch(t -> t.getAttributeValue("iceberg.id") != null);
if (!hasIds) {
  throw new IllegalStateException("ORC file lacks Iceberg ID attributes; supply an explicit schema");
}

try {
  Schema s = ORCSchemaUtil.convert(orcSchema);
} catch (IllegalArgumentException e) {
  // fall back to externally provided schema
}

When it happens

Trigger: Calling ORCSchemaUtil.convert(orcSchema) (schema conversion from ORC to Iceberg) on an ORC file whose TypeDescription attributes lack the Iceberg ID attribute — i.e. the file was not written by Iceberg's ORC writer or the attributes were stripped.

Common situations: Pointing Iceberg at ORC files written by other tools (Hive/Spark raw ORC, Trino); files rewritten or copied with an ORC library that drops custom attributes; older Iceberg-written files predating the ID attribute convention.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Unhandled type
• Field + name + not found in source schema
• Cannot parse type string to primitive: + typeString
• Invalid default value for %s: %s (must be null)
• Entries must have explicit snapshot ids if inherited metadat

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/97bfa89d6c5fa85d. Report an issue: GitHub.