{"record":{"id":"d72aefab82549638","repo":"apache/iceberg","slug":"cannot-commit-s-because-base-metadata-location-d72aef","errorCode":null,"errorMessage":"Cannot commit %s because base metadata location '%s' is not same as the current Glue location '%s'","messagePattern":"Cannot commit (.+?) because base metadata location '(.+?)' is not same as the current Glue location '(.+?)'","errorType":"exception","errorClass":"CommitFailedException","httpStatus":null,"severity":"error","filePath":"aws/src/main/java/org/apache/iceberg/aws/glue/GlueTableOperations.java","lineNumber":270,"sourceCode":"          DeleteTableRequest.builder().databaseName(databaseName).name(tableName).build());\n    }\n  }\n\n  private void lock(String newMetadataLocation) {\n    if (lockManager != null && !lockManager.acquire(commitLockEntityId, newMetadataLocation)) {\n      throw new IllegalStateException(\n          String.format(\n              \"Fail to acquire lock %s to commit new metadata at %s\",\n              commitLockEntityId, newMetadataLocation));\n    }\n  }\n\n  private void checkMetadataLocation(Table glueTable, TableMetadata base) {\n    String glueMetadataLocation =\n        glueTable != null ? glueTable.parameters().get(METADATA_LOCATION_PROP) : null;\n    String baseMetadataLocation = base != null ? base.metadataFileLocation() : null;\n    if (!Objects.equals(baseMetadataLocation, glueMetadataLocation)) {\n      throw new CommitFailedException(\n          \"Cannot commit %s because base metadata location '%s' is not same as the current Glue location '%s'\",\n          tableName(), baseMetadataLocation, glueMetadataLocation);\n    }\n  }\n\n  private Table getGlueTable() {\n    try {\n      GetTableResponse response =\n          glue.getTable(\n              GetTableRequest.builder()\n                  .catalogId(awsProperties.glueCatalogId())\n                  .databaseName(databaseName)\n                  .name(tableName)\n                  .build());\n      return response.table();\n    } catch (EntityNotFoundException e) {\n      return null;\n    }","sourceCodeStart":252,"sourceCodeEnd":288,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/aws/src/main/java/org/apache/iceberg/aws/glue/GlueTableOperations.java#L252-L288","documentation":"CommitFailedException thrown by GlueTableOperations.checkMetadataLocation when the metadata location recorded in the Glue table's parameters does not match the metadata file location of the base TableMetadata the client committed against. This is Iceberg's optimistic-concurrency check for the Glue catalog: another writer has committed a new snapshot since this operation started, so committing would silently overwrite their work. The commit must be retried against the refreshed table state.","triggerScenarios":"Calling doCommit (via table refreshCommit/commit, fastAppend, overwrite, etc.) while the METADATA_LOCATION_PROP stored in Glue differs from base.metadataFileLocation() — i.e. a competing process committed to the same table between this operation's refresh and its commit.","commonSituations":"Two Spark/Flink jobs or a compaction job writing to the same Iceberg table concurrently; a long-running streaming job whose cached table metadata went stale while another writer advanced the table; manual edits to the Glue table parameters outside Iceberg; committing from a stale Table handle that was never refreshed.","solutions":["Retry the commit: catch CommitFailedException, refresh the table (table.refresh()) and re-apply the operation — Iceberg's retryable commit loop normally does this automatically.","Verify all writers point at the same table identifier/warehouse; a stale Table handle should be re-created or refreshed before committing.","Check for external tools (Athena, EMR, scripts) modifying the Glue table's metadata_location parameter directly and stop them from doing so.","Reduce contention by isolating workloads (partition-level appends, separate tables) or enabling row-level merge-on-read instead of frequent overwrite commits."],"exampleFix":"// before\nTable table = catalog.loadTable(identifier);\nThread.sleep(600_000); // long work while others commit\ntable.append(df); // CommitFailedException: base metadata location stale\n// after\nTable table = catalog.loadTable(identifier);\n// do work, then refresh right before committing\ntable.refresh();\ntable.append(df); // retried automatically by Iceberg on CommitFailedException","handlingStrategy":"retry","validationCode":"table.refresh();\nif (!Objects.equals(table.operations().current().metadataFileLocation(), baseLocation)) {\n  // state moved; reload before committing\n}","typeGuard":null,"tryCatchPattern":"try {\n  table.append(df);\n} catch (CommitFailedException e) {\n  table.refresh();\n  // retry the operation (bounded retries)\n}","preventionTips":["Always call table.refresh() before committing after long-running work","Rely on Iceberg's built-in commit retry loop rather than caching Operations long-term","Avoid external tools writing to Glue table parameters directly","Isolate high-frequency writers per partition or table to reduce contention"],"tags":["aws","glue","commit","concurrency"],"backgroundTag":"concurrent-commit-conflict","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}