prestodb/presto · error · PrestoException

ICEBERG_INVALID_METADATA

ICEBERG_INVALID_METADATA

Error message

Unable to find sort field source column in the table schema:

What it means

During IcebergPageSink construction, each sort field from the table's Iceberg sort order is resolved against the sink's output schema via findField(sourceColumnId). If the schema has no field with that column ID, the table metadata is internally inconsistent (sort order references a column that no longer exists), so Presto throws ICEBERG_INVALID_METADATA rather than writing corrupt data.

Source

Thrown at presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergPageSink.java:194

                .map(IcebergColumnHandle::getType)
                .collect(toImmutableList());
        this.sortParameters = sortParameters;
        if (!sortOrder.isEmpty()) {
            ImmutableList.Builder<Integer> sortColumnIndexes = ImmutableList.builder();
            ImmutableList.Builder<SortOrder> sortOrders = ImmutableList.builder();
            for (SortField sortField : sortOrder) {
                if (sortField.getSourceColumnId() == Z_ORDER.getId()) {
                    sortColumnIndexes.add(outputSchema.columns().size());
                    sortOrders.add(sortField.getSortOrder());
                    types = ImmutableList.<Type>builder()
                            .addAll(types)
                            .add(VARBINARY)
                            .build();
                }
                else {
                    Types.NestedField column = outputSchema.findField(sortField.getSourceColumnId());
                    if (column == null) {
                        throw new PrestoException(ICEBERG_INVALID_METADATA, "Unable to find sort field source column in the table schema: " + sortField);
                    }
                    sortColumnIndexes.add(outputSchema.columns().indexOf(column));
                    sortOrders.add(sortField.getSortOrder());
                }
            }
            this.sortColumnIndexes = sortColumnIndexes.build();
            this.sortOrders = sortOrders.build();
        }
        else {
            this.sortColumnIndexes = ImmutableList.of();
            this.sortOrders = ImmutableList.of();
        }
        this.columnTypes = types;
    }

    @Override
    public long getCompletedBytes()
    {

View on GitHub (pinned to 55bb57d202)

Solutions

Fix the table metadata: remove or rewrite the sort order so it only references existing schema columns (e.g. via Iceberg API removeSortOrder()/replaceSortOrder or a repair tool).
Restore the dropped/renamed column, or re-add it under a new name and update the sort order to point at the current field IDs.
Roll back to a prior metadata.json (rollback_to_snapshot / register previous metadata version) where schema and sort order were consistent, then evolve the schema correctly.
Upgrade Presto/Iceberg connector versions — newer writers handle schema-evolved sort orders more gracefully.

Example fix

// before: sort order references dropped column
// schema: [1:id, 2:name], sort order: [field 3:ts DESC]  <- field 3 gone
// after (Iceberg API repair)
table.replaceSortOrder().asc("id").commit();  // sort only on existing fields

Defensive patterns

Strategy: validation

Validate before calling

// check sort order vs current schema before writing (Iceberg API)
Schema schema = table.schema();
boolean consistent = table.sortOrder().fields().stream()
    .allMatch(f -> schema.findField(f.getSourceColumnId()) != null);
if (!consistent) throw new IllegalStateException("Sort order references missing columns; repair metadata before INSERT");

Try / catch

try { table.newAppend().appendFile(dataFile).commit(); }
catch (PrestoException e) {
  if (e.getCode().equals(ICEBERG_INVALID_METADATA)) {
    table.replaceSortOrder().commit(); // clear stale sort order, retry
  } else throw e;
}

Prevention

When dropping/renaming columns, always update (or clear) the table sort order in the same evolution operation.
Run metadata consistency checks after cross-engine writes or catalog migrations.
Avoid hand-editing metadata.json files.
Pin a recent Presto/Iceberg writer version that tolerates evolved sort orders.

When it happens

Trigger: Writing (INSERT/CTAS/refresh) into an Iceberg table whose sort order metadata references a source column ID absent from the table's current schema — typically after schema evolution dropped/renamed the sort column while stale sort-order entries remained, or reading a table written by another engine with inconsistent metadata.

Common situations: ALTER TABLE DROP/RENAME of a column that was part of the table's sort order without updating sort order; old writer versions reading tables modified by newer tooling; corrupted metadata.json hand-edited or produced by a buggy catalog migration.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/37d83a8d12cd0fdc. Report an issue: GitHub.