apache/iceberg · error · CommitFailedException

Cannot commit: Base metadata location '%s' is not same as th

Error message

Cannot commit: Base metadata location '%s' is not same as the current table metadata location '%s' for %s.%s

What it means

HiveTableOperations.doCommit performs an optimistic-concurrency check before altering the Hive metastore table: the base metadata file location the commit was started from must equal the metadata_location parameter currently stored in the HMS table. If another process committed in between, the base is stale and this CommitFailedException is thrown so the caller can retry the whole operation.

Source

Thrown at hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java:307

        updateHiveTable = true;
        LOG.debug("Committing existing table: {}", fullName);
      } else {
        tbl =
            newHmsTable(
                tableMetadata.property(HiveCatalog.HMS_TABLE_OWNER, HiveHadoopUtil.currentUser()));
        LOG.debug("Committing new table: {}", fullName);
      }

      tbl.setSd(
          HiveOperationsBase.storageDescriptor(
              tableMetadata.schema(),
              tableMetadata.location(),
              hiveEngineEnabled)); // set to pickup any schema changes

      String metadataLocation = tbl.getParameters().get(METADATA_LOCATION_PROP);
      String baseMetadataLocation = base != null ? base.metadataFileLocation() : null;
      if (!Objects.equals(baseMetadataLocation, metadataLocation)) {
        throw new CommitFailedException(
            "Cannot commit: Base metadata location '%s' is not same as the current table metadata location '%s' for %s.%s",
            baseMetadataLocation, metadataLocation, database, tableName);
      }

      // get Iceberg props that have been removed
      Set<String> removedProps = Collections.emptySet();
      if (base != null) {
        removedProps =
            base.properties().keySet().stream()
                .filter(key -> !tableMetadata.properties().containsKey(key))
                .collect(Collectors.toSet());

        Preconditions.checkArgument(
            !removedProps.contains(TableProperties.ENCRYPTION_TABLE_KEY),
            "Cannot remove key ID from an encrypted table");

        Preconditions.checkArgument(
            Objects.equals(

View on GitHub (pinned to 86d9c8fc54)

Solutions

Retry the commit: re-read the table (table.refresh()) and re-apply the operation on fresh metadata — CommitFailedException is designed for client-level retry
Enable Hive lock acquisition (iceberg.engine.hive.enabled / lock settings) or use a catalog with atomic commit semantics (HiveCatalog with proper locking, REST, Nessie)
Serialize writers externally so only one job writes to the table at a time
Check the HMS table's metadata_location vs the base to confirm which writer won before retrying

Example fix

// before
Table table = catalog.loadTable(ident);
table.newAppend().appendFile(file).commit(); // stale base -> CommitFailedException
// after
Table table = catalog.loadTable(ident);
try {
  table.newAppend().appendFile(file).commit();
} catch (CommitFailedException e) {
  table.refresh();
  table.newAppend().appendFile(file).commit();
}

Defensive patterns

Strategy: retry

Validate before calling

String current = hmsClient.getTable(db, table).getParameters().get("metadata_location");
if (!Objects.equals(current, table.operations().current().metadataFileLocation())) {
  table.refresh(); // stale base, reload before writing
}

Try / catch

try {
  table.newAppend().appendFile(f).commit();
} catch (CommitFailedException e) {
  table.refresh();
  table.newAppend().appendFile(f).commit();
}

Prevention

Always wrap Iceberg commits in a CommitFailedException retry loop
Re-read (refresh) the table before each write attempt
Avoid multiple concurrent writers without locking or an atomic-commit catalog
Keep write transactions short to shrink the conflict window

When it happens

Trigger: Calling HiveTableOperations.doCommit (e.g. via Table.refresh/commit, append, overwrite) when another writer updated the table's metadata_location in HMS after the base TableMetadata snapshot was read; this test path runs with Hive locking disabled (hive.engine/hive lock disabled) so no lock serializes writers.

Common situations: Two Spark/Flink jobs or a Spark job and a maintenance tool (rewrite/expire) writing the same Iceberg table concurrently; long-running streams holding stale metadata while a compaction commits; HMS updated manually.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/cd479e2216ac226c. Report an issue: GitHub.