{"record":{"id":"3b4373bd1233b953","repo":"apache/iceberg","slug":"failed-to-commit","errorCode":null,"errorMessage":"Failed to commit","messagePattern":"Failed to commit","errorType":"exception","errorClass":"CommitFailedException","httpStatus":null,"severity":"error","filePath":"bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryTableOperations.java","lineNumber":109,"sourceCode":"    try {\n      if (base == null) {\n        createTable(newMetadataLocation, metadata);\n      } else {\n        updateTable(newMetadataLocation, metadata);\n      }\n      commitStatus = BaseMetastoreOperations.CommitStatus.SUCCESS;\n    } catch (CommitFailedException | CommitStateUnknownException e) {\n      throw e;\n    } catch (Throwable e) {\n      LOG.error(\"Exception thrown on commit: \", e);\n      if (e instanceof AlreadyExistsException) {\n        throw e;\n      }\n      commitStatus =\n          BaseMetastoreOperations.CommitStatus.valueOf(\n              checkCommitStatus(newMetadataLocation, metadata).name());\n      if (commitStatus == BaseMetastoreOperations.CommitStatus.FAILURE) {\n        throw new CommitFailedException(e, \"Failed to commit\");\n      }\n      if (commitStatus == BaseMetastoreOperations.CommitStatus.UNKNOWN) {\n        throw new CommitStateUnknownException(e);\n      }\n    } finally {\n      try {\n        if (commitStatus == BaseMetastoreOperations.CommitStatus.FAILURE) {\n          LOG.warn(\"Failed to commit updates to table {}\", tableName());\n          io().deleteFile(newMetadataLocation);\n        }\n      } catch (RuntimeException e) {\n        LOG.error(\n            \"Failed to cleanup metadata file at {} for table {}\",\n            newMetadataLocation,\n            tableName(),\n            e);\n      }\n    }","sourceCodeStart":91,"sourceCodeEnd":127,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryTableOperations.java#L91-L127","documentation":"BigQueryTableOperations.doCommit first attempts the BigQuery Metastore conditional update; on failure it re-checks the commit status. If the re-check resolves to FAILURE it throws CommitFailedException(\"Failed to commit\"), meaning the update definitively lost the race or was rejected and no new commit landed. If UNKNOWN it throws CommitStateUnknownException instead.","triggerScenarios":"Concurrent commits to the same table: another writer updated the metadata location between refresh and commit, so the etag-conditional update is rejected and the follow-up check confirms another commit won.","commonSituations":"Two Spark/Flink jobs writing the same table concurrently; a retry of a failed commit after another job already committed; frequent small commits from streaming jobs colliding.","solutions":["Retry the operation: refresh the table and re-apply the changes — a genuine lost race is expected to succeed on retry.","Reduce concurrent writers to the same table, or serialize commits through one job/coordinator.","Check what metadata the competing commit wrote to determine whether your changes were superseded.","If it recurs with no competing writer, inspect the BigQuery Metastore table's etag/parameters for manual edits conflicting with Iceberg."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Before committing, detect obvious concurrent writers:\nTable loaded = catalog.loadTable(tableId);\nlong loadedSnapshotId = loaded.currentSnapshot() == null ? -1 : loaded.currentSnapshot().snapshotId();\n// if your staged base snapshot != loadedSnapshotId, refresh and re-apply before commit","typeGuard":null,"tryCatchPattern":"try {\n  table.refresh();\n} catch (CommitFailedException e) {\n  // lost race: re-apply changes on the refreshed table and retry\n  retryCommitWithBackoff();\n}","preventionTips":["Retry CommitFailedException with backoff — losing an optimistic race is expected, not a bug","Avoid multiple concurrent jobs committing to the same Iceberg table","Use frequent refresh() before long-running commit preparations","Keep commits small and short-lived to shrink the race window"],"tags":["concurrency","commit","bigquery","optimistic-locking"],"backgroundTag":"commit-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}