apache/iceberg · error · CommitFailedException

Cannot commit %s because base metadata location '%s' is not

Error message

Cannot commit %s because base metadata location '%s' is not same as the current Glue location '%s'

What it means

CommitFailedException thrown by GlueTableOperations.checkMetadataLocation when the metadata location recorded in the Glue table's parameters does not match the metadata file location of the base TableMetadata the client committed against. This is Iceberg's optimistic-concurrency check for the Glue catalog: another writer has committed a new snapshot since this operation started, so committing would silently overwrite their work. The commit must be retried against the refreshed table state.

Source

Thrown at aws/src/main/java/org/apache/iceberg/aws/glue/GlueTableOperations.java:270

          DeleteTableRequest.builder().databaseName(databaseName).name(tableName).build());
    }
  }

  private void lock(String newMetadataLocation) {
    if (lockManager != null && !lockManager.acquire(commitLockEntityId, newMetadataLocation)) {
      throw new IllegalStateException(
          String.format(
              "Fail to acquire lock %s to commit new metadata at %s",
              commitLockEntityId, newMetadataLocation));
    }
  }

  private void checkMetadataLocation(Table glueTable, TableMetadata base) {
    String glueMetadataLocation =
        glueTable != null ? glueTable.parameters().get(METADATA_LOCATION_PROP) : null;
    String baseMetadataLocation = base != null ? base.metadataFileLocation() : null;
    if (!Objects.equals(baseMetadataLocation, glueMetadataLocation)) {
      throw new CommitFailedException(
          "Cannot commit %s because base metadata location '%s' is not same as the current Glue location '%s'",
          tableName(), baseMetadataLocation, glueMetadataLocation);
    }
  }

  private Table getGlueTable() {
    try {
      GetTableResponse response =
          glue.getTable(
              GetTableRequest.builder()
                  .catalogId(awsProperties.glueCatalogId())
                  .databaseName(databaseName)
                  .name(tableName)
                  .build());
      return response.table();
    } catch (EntityNotFoundException e) {
      return null;
    }

View on GitHub (pinned to 86d9c8fc54)

Solutions

Retry the commit: catch CommitFailedException, refresh the table (table.refresh()) and re-apply the operation — Iceberg's retryable commit loop normally does this automatically.
Verify all writers point at the same table identifier/warehouse; a stale Table handle should be re-created or refreshed before committing.
Check for external tools (Athena, EMR, scripts) modifying the Glue table's metadata_location parameter directly and stop them from doing so.
Reduce contention by isolating workloads (partition-level appends, separate tables) or enabling row-level merge-on-read instead of frequent overwrite commits.

Example fix

// before
Table table = catalog.loadTable(identifier);
Thread.sleep(600_000); // long work while others commit
table.append(df); // CommitFailedException: base metadata location stale
// after
Table table = catalog.loadTable(identifier);
// do work, then refresh right before committing
table.refresh();
table.append(df); // retried automatically by Iceberg on CommitFailedException

Defensive patterns

Strategy: retry

Validate before calling

table.refresh();
if (!Objects.equals(table.operations().current().metadataFileLocation(), baseLocation)) {
  // state moved; reload before committing
}

Try / catch

try {
  table.append(df);
} catch (CommitFailedException e) {
  table.refresh();
  // retry the operation (bounded retries)
}

Prevention

Always call table.refresh() before committing after long-running work
Rely on Iceberg's built-in commit retry loop rather than caching Operations long-term
Avoid external tools writing to Glue table parameters directly
Isolate high-frequency writers per partition or table to reduce contention

When it happens

Trigger: Calling doCommit (via table refreshCommit/commit, fastAppend, overwrite, etc.) while the METADATA_LOCATION_PROP stored in Glue differs from base.metadataFileLocation() — i.e. a competing process committed to the same table between this operation's refresh and its commit.

Common situations: Two Spark/Flink jobs or a compaction job writing to the same Iceberg table concurrently; a long-running streaming job whose cached table metadata went stale while another writer advanced the table; manual edits to the Glue table parameters outside Iceberg; committing from a stale Table handle that was never refreshed.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/d72aefab82549638. Report an issue: GitHub.