{"record":{"id":"f60524317c46607f","repo":"apache/iceberg","slug":"cannot-commit-s-concurrent-update-detected","errorCode":null,"errorMessage":"Cannot commit %s: concurrent update detected","messagePattern":"Cannot commit (.+?): concurrent update detected","errorType":"exception","errorClass":"CommitFailedException","httpStatus":409,"severity":"error","filePath":"aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbTableOperations.java","lineNumber":140,"sourceCode":"      commitStatus = CommitStatus.SUCCESS;\n    } catch (CommitFailedException e) {\n      // any explicit commit failures are passed up and out to the retry handler\n      throw e;\n    } catch (RuntimeException persistFailure) {\n      boolean conditionCheckFailed = persistFailure instanceof ConditionalCheckFailedException;\n\n      // If we got an exception we weren't expecting, or we got a ConditionalCheckFailedException\n      // but retries were performed, attempt to reconcile the actual commit status.\n      if (!conditionCheckFailed || retryDetector.retried()) {\n        LOG.warn(\n            \"Received unexpected failure when committing to {}, validating if commit ended up succeeding.\",\n            fullTableName,\n            persistFailure);\n        commitStatus = checkCommitStatus(newMetadataLocation, metadata);\n      }\n\n      if (commitStatus != CommitStatus.SUCCESS && conditionCheckFailed) {\n        throw new CommitFailedException(\n            persistFailure, \"Cannot commit %s: concurrent update detected\", tableName());\n      }\n\n      switch (commitStatus) {\n        case SUCCESS:\n          break;\n        case FAILURE:\n          throw new CommitFailedException(\n              persistFailure, \"Cannot commit %s due to unexpected exception\", tableName());\n        case UNKNOWN:\n          throw new CommitStateUnknownException(persistFailure);\n      }\n    } finally {\n      try {\n        if (commitStatus == CommitStatus.FAILURE) {\n          // if anything went wrong, clean up the uncommitted metadata file\n          io().deleteFile(newMetadataLocation);\n        }","sourceCodeStart":122,"sourceCodeEnd":158,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbTableOperations.java#L122-L158","documentation":"CommitFailedException thrown in DynamoDbTableOperations.doCommit when the conditional update failed (conditionCheckFailed) and the commit could not be confirmed as successful. Another writer updated the table's metadata location between refresh and commit, so the optimistic concurrency check failed.","triggerScenarios":"Two writers commit to the same table concurrently; the PutItem condition on COL_METADATA_LOCATION fails, then checkCommitStatus does not return SUCCESS.","commonSituations":"Multiple Spark jobs or services writing to the same table simultaneously; long-running plan followed by commit racing a shorter job; retries after lost leases.","solutions":["Refresh the table and retry the operation from the new base metadata","Serialize writers with DynamoDbLockManager or an external lock so only one commits at a time","Ensure the committing job's metadata snapshot is fresh (call refresh() right before commit)"],"exampleFix":"// before\nTable t = catalog.loadTable(id);\nt.append(df); // commit may fail under contention\n// after\nTable t = catalog.loadTable(id);\ntry {\n  t.refresh();\n  t.append(df);\n} catch (CommitFailedException e) {\n  t.refresh();\n  t.append(df); // retry from fresh base\n}","handlingStrategy":"retry","validationCode":"table.refresh(); // rebase on latest metadata right before writing","typeGuard":null,"tryCatchPattern":"try {\n  table.append(df);\n} catch (CommitFailedException e) {\n  table.refresh();\n  table.append(df); // retry once from fresh metadata\n}","preventionTips":["Refresh the table immediately before commit","Reduce commit window length (plan long, commit short)","Serialize heavy writers with DynamoDbLockManager"],"tags":["aws","dynamodb","concurrency","commit-conflict"],"backgroundTag":"commit-conflict","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}