apache/iceberg · error · CommitFailedException

Cannot commit changes based on stale table metadata

Error message

Cannot commit changes based on stale table metadata

What it means

commit() requires that the base TableMetadata passed in is exactly the same object (same version) the operations instance currently holds. If the caller's base is stale relative to the on-disk table, the commit would silently overwrite concurrent changes, so CommitStateUnknown/CommitFailedException is thrown. Callers must refresh and retry.

Source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java:134

        ver += 1;
        metadataFile = nextMetadataFile;
        nextMetadataFile = getMetadataFile(ver + 1);
      }

      updateVersionAndMetadata(ver, metadataFile.toString());

      this.shouldRefresh = false;
      return currentMetadata;
    } catch (IOException e) {
      throw new RuntimeIOException(e, "Failed to refresh the table");
    }
  }

  @Override
  public void commit(TableMetadata base, TableMetadata metadata) {
    Pair<Integer, TableMetadata> current = versionAndMetadata();
    if (base != current.second()) {
      throw new CommitFailedException("Cannot commit changes based on stale table metadata");
    }

    if (base == metadata) {
      LOG.info("Nothing to commit.");
      return;
    }

    Preconditions.checkArgument(
        base == null || base.location().equals(metadata.location()),
        "Hadoop path-based tables cannot be relocated");
    Preconditions.checkArgument(
        !metadata.properties().containsKey(TableProperties.WRITE_METADATA_LOCATION),
        "Hadoop path-based tables cannot relocate metadata");

    String codecName =
        metadata.property(
            TableProperties.METADATA_COMPRESSION, TableProperties.METADATA_COMPRESSION_DEFAULT);
    TableMetadataParser.Codec codec = TableMetadataParser.Codec.fromName(codecName);

View on GitHub (pinned to 86d9c8fc54)

Solutions

Call table.refresh() to get latest metadata, reapply your changes to the fresh base, then retry commit
Use a catalog with atomic commit support instead of raw HadoopTables for multi-writer workloads
Serialize writers externally (lock) when using HadoopTables with concurrent writers
Wrap commit in a retry loop catching CommitFailedException with bounded attempts

Example fix

// before
ops.commit(staleBase, updatedMetadata);
// after
boolean committed = false;
for (int i = 0; i < 3 && !committed; i++) {
  try {
    ops.commit(ops.current(), updatedFor(ops.current()));
    committed = true;
  } catch (CommitFailedException e) {
    ops.refresh();
  }
}

Defensive patterns

Strategy: retry

Validate before calling

if (ops.current() != base) { ops.refresh(); /* rebase changes */ }

Try / catch

try { ops.commit(base, metadata); } catch (CommitFailedException e) { ops.refresh(); /* rebase and retry */ }

Prevention

Always refresh() before commit when the table may have changed externally
Wrap commits in bounded retry loops against CommitFailedException
Avoid long-lived Table references in multi-writer environments
Prefer catalogs with atomic commits over HadoopTables for concurrent writers

When it happens

Trigger: Calling ops.commit(base, metadata) after another process committed a new snapshot so base != current(); using a Table reference whose metadata was loaded before an external commit; calling commit without refresh() after a prior commit from the same reference.

Common situations: Two Spark jobs writing to the same Hadoop-backed table concurrently; reusing a cached Table object across a long job while other writers commit; skipping refresh() between consecutive commits from one process.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/8fd5619a0cec3abf. Report an issue: GitHub.