apache/iceberg · error · PendingUpdateFailedException

PendingUpdateFailedException

Error message

PendingUpdateFailedException

What it means

PendingUpdateFailedException is thrown when a nested PendingUpdate inside a BaseTransaction fails to commit even after retrying, because of a conflicting metadata change. The transaction aborts rather than looping forever, wrapping the original CommitFailedException as the cause. It signals the whole transaction cannot be applied and the caller must start over from refreshed table metadata.

Source

Thrown at core/src/main/java/org/apache/iceberg/BaseTransaction.java:450

  }

  private void deleteUncommittedFiles(Iterable<String> paths) {
    CatalogUtil.deleteFiles(ops.io(), paths, "uncommitted");
  }

  private void applyUpdates(TableOperations underlyingOps) {
    if (base != underlyingOps.refresh()) {
      // use refreshed the metadata
      this.base = underlyingOps.current();
      this.current = underlyingOps.current();
      for (PendingUpdate update : updates) {
        // re-commit each update in the chain to apply it and update current
        try {
          update.commit();
        } catch (CommitFailedException e) {
          // Cannot pass even with retry due to conflicting metadata changes. So, break the
          // retry-loop.
          throw new PendingUpdateFailedException(e);
        }
      }
    }
  }

  // returns the manifest lists and manifests referenced by the given committed snapshots
  private static Set<String> committedFiles(FileIO io, Set<Snapshot> snapshots) {
    Set<String> committedFiles = Sets.newHashSet();

    for (Snapshot snap : snapshots) {
      committedFiles.add(snap.manifestListLocation());
      snap.allManifests(io).forEach(manifest -> committedFiles.add(manifest.path()));
    }

    return committedFiles;
  }

  public class TransactionTableOperations implements TableOperations {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Refresh the table and re-create/re-run the whole transaction from the latest metadata
2. Reduce transaction scope so it commits quickly, minimizing the conflict window
3. Serialize or partition concurrent writers so they don't touch the same table metadata simultaneously
4. Inspect the wrapped CommitFailedException cause to identify which update is conflicting

Example fix

// before
Transaction tx = table.newTransaction();
tx.updateSchema().addColumn("new_col", Types.LongType.get()).commit();
tx.commitTransaction(); // may throw PendingUpdateFailedException
// after
try {
  Transaction tx = table.newTransaction();
  tx.updateSchema().addColumn("new_col", Types.LongType.get()).commit();
  tx.commitTransaction();
} catch (PendingUpdateFailedException e) {
  table.refresh();
  // rebuild and retry the transaction against fresh metadata
}

catch (PendingUpdateFailedException e) { table.refresh(); /* retry the whole transaction from fresh state */ } — always retry the full transaction, never individual updates

When it happens

Trigger: Calling Transaction.commitTransaction() where an individual update's commit() repeatedly fails with CommitFailedException against concurrent metadata changes (e.g. conflicting schema updates, partition spec changes, or property writes from another committer).

Common situations: Concurrent writers to the same table (Spark jobs, Flink sinks, REST catalog commits) racing on table metadata; long-running transactions whose base metadata is stale by commit time; two transactions both altering schema or sort order.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Cannot commit %s: concurrent update detected
• Cannot commit file that conflicts with existing partition: %
• Cannot commit to table %s metadata location from %s to %s be
• View was updated concurrently: %s
• The view %s.%s has been modified concurrently

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/cd5d300ccbbc4549. Report an issue: GitHub.