apache/iceberg · error · CommitFailedException

Failed to commit

Error message

Failed to commit

What it means

BigQueryTableOperations.doCommit first attempts the BigQuery Metastore conditional update; on failure it re-checks the commit status. If the re-check resolves to FAILURE it throws CommitFailedException("Failed to commit"), meaning the update definitively lost the race or was rejected and no new commit landed. If UNKNOWN it throws CommitStateUnknownException instead.

Source

Thrown at bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryTableOperations.java:109

    try {
      if (base == null) {
        createTable(newMetadataLocation, metadata);
      } else {
        updateTable(newMetadataLocation, metadata);
      }
      commitStatus = BaseMetastoreOperations.CommitStatus.SUCCESS;
    } catch (CommitFailedException | CommitStateUnknownException e) {
      throw e;
    } catch (Throwable e) {
      LOG.error("Exception thrown on commit: ", e);
      if (e instanceof AlreadyExistsException) {
        throw e;
      }
      commitStatus =
          BaseMetastoreOperations.CommitStatus.valueOf(
              checkCommitStatus(newMetadataLocation, metadata).name());
      if (commitStatus == BaseMetastoreOperations.CommitStatus.FAILURE) {
        throw new CommitFailedException(e, "Failed to commit");
      }
      if (commitStatus == BaseMetastoreOperations.CommitStatus.UNKNOWN) {
        throw new CommitStateUnknownException(e);
      }
    } finally {
      try {
        if (commitStatus == BaseMetastoreOperations.CommitStatus.FAILURE) {
          LOG.warn("Failed to commit updates to table {}", tableName());
          io().deleteFile(newMetadataLocation);
        }
      } catch (RuntimeException e) {
        LOG.error(
            "Failed to cleanup metadata file at {} for table {}",
            newMetadataLocation,
            tableName(),
            e);
      }
    }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Retry the operation: refresh the table and re-apply the changes — a genuine lost race is expected to succeed on retry.
2. Reduce concurrent writers to the same table, or serialize commits through one job/coordinator.
3. Check what metadata the competing commit wrote to determine whether your changes were superseded.
4. If it recurs with no competing writer, inspect the BigQuery Metastore table's etag/parameters for manual edits conflicting with Iceberg.

// Before committing, detect obvious concurrent writers:
Table loaded = catalog.loadTable(tableId);
long loadedSnapshotId = loaded.currentSnapshot() == null ? -1 : loaded.currentSnapshot().snapshotId();
// if your staged base snapshot != loadedSnapshotId, refresh and re-apply before commit

try {
  table.refresh();
} catch (CommitFailedException e) {
  // lost race: re-apply changes on the refreshed table and retry
  retryCommitWithBackoff();
}

When it happens

Trigger: Concurrent commits to the same table: another writer updated the metadata location between refresh and commit, so the etag-conditional update is rejected and the follow-up check confirms another commit won.

Common situations: Two Spark/Flink jobs writing the same table concurrently; a retry of a failed commit after another job already committed; frequent small commits from streaming jobs colliding.

Related errors

• Cannot commit changes based on stale table metadata
• Cannot commit: Base metadata location '%s' is not same as th
• The table %s.%s has been modified concurrently
• Cannot commit %s due to unexpected exception
• Fail to acquire lock %s to commit new metadata at %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/3b4373bd1233b953. Report an issue: GitHub.