{"record":{"id":"213e038eef4330c2","repo":"apache/iceberg","slug":"cannot-commit-s-due-to-unexpected-exception-213e03","errorCode":null,"errorMessage":"Cannot commit %s due to unexpected exception","messagePattern":"Cannot commit (.+?) due to unexpected exception","errorType":"exception","errorClass":"CommitFailedException","httpStatus":null,"severity":"error","filePath":"aws/src/main/java/org/apache/iceberg/aws/glue/GlueTableOperations.java","lineNumber":187,"sourceCode":"      if (!isAwsServiceException || retryDetector.retried()) {\n        LOG.warn(\n            \"Received unexpected failure when committing to {}, validating if commit ended up succeeding.\",\n            fullTableName,\n            persistFailure);\n        commitStatus = checkCommitStatus(newMetadataLocation, metadata);\n      }\n\n      // If we got an AWS exception we would usually handle, but find we\n      // succeeded on a retry that threw an exception, skip the exception.\n      if (commitStatus != CommitStatus.SUCCESS && isAwsServiceException) {\n        handleAWSExceptions((AwsServiceException) persistFailure);\n      }\n\n      switch (commitStatus) {\n        case SUCCESS:\n          break;\n        case FAILURE:\n          throw new CommitFailedException(\n              persistFailure, \"Cannot commit %s due to unexpected exception\", tableName());\n        case UNKNOWN:\n          throw new CommitStateUnknownException(persistFailure);\n      }\n    } finally {\n      cleanupMetadataAndUnlock(commitStatus, newMetadataLocation);\n      cleanupGlueTempTableIfNecessary(glueTempTableCreated, commitStatus);\n    }\n  }\n\n  /**\n   * Validate the Glue table is Iceberg table by checking its parameters. If the table properties\n   * check does not pass, for Iceberg it is equivalent to not having a table in the catalog. We\n   * throw a {@link NoSuchIcebergTableException} in that case.\n   *\n   * @param table glue table\n   * @param fullName full table name for logging\n   * @throws NoSuchIcebergTableException if the table is not an Iceberg table","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/aws/src/main/java/org/apache/iceberg/aws/glue/GlueTableOperations.java#L169-L205","documentation":"GlueTableOperations.doCommit throws CommitFailedException when the metadata persistence to Glue failed with a definitive (FAILURE) commit status. The underlying exception (persistFailure) is attached as the cause; callers should treat this like any Iceberg commit conflict/failure and retry the operation from a fresh table state.","triggerScenarios":"A Glue UpdateTable call during commit fails definitively — e.g. OptimisticLockException style conflicts when another writer committed concurrently, validation errors on the table update, throttling, or AWS service errors that are deterministically failed rather than unknown-outcome.","commonSituations":"Multiple concurrent writers committing to the same table (classic optimistic concurrency conflict); Glue API throttling under heavy write load; Lake Formation rejecting the update due to permissions; transient AWS outages.","solutions":["Retry the write: reload the table (catalog.loadTable) to get fresh state and re-apply the operation — Iceberg commits are designed for optimistic retry.","Inspect the persistFailure cause (printed with the exception) to distinguish a conflict (retry-able) from a validation/permission error (fix required).","Reduce concurrent writers on the table or serialize commits through a single job/lock.","Check Glue service quotas/throttling and add backoff between commits; verify Lake Formation permissions for updates."],"exampleFix":"// before\ncatalog.loadTable(ident).append(df); // throws CommitFailedException on conflict\n\n// after\nint attempts = 3;\nwhile (true) {\n  try {\n    catalog.loadTable(ident).append(df); // reload inside loop = fresh snapshot\n    break;\n  } catch (CommitFailedException e) {\n    if (--attempts == 0) throw e;\n  }\n}","handlingStrategy":"retry","validationCode":"// no pre-call validation possible; conflicts happen at commit time\n// ensure table exists and writer count is bounded before writing\nif (!catalog.tableExists(ident)) {\n  throw new IllegalStateException(\"Table missing before commit: \" + ident);\n}","typeGuard":null,"tryCatchPattern":"int maxAttempts = 3;\nfor (int i = 0; i < maxAttempts; i++) {\n  try {\n    catalog.loadTable(ident).append(df); // fresh load each attempt\n    break;\n  } catch (CommitFailedException e) {\n    if (i == maxAttempts - 1) throw e;\n    Uninterruptibles.sleepUninterruptibly(100L * (i + 1), TimeUnit.MILLISECONDS);\n  }\n}","preventionTips":["Always retry with a freshly loaded table — CommitFailedException means state changed under you","Limit concurrent writers per table or serialize commits","Inspect the attached persistFailure cause to separate conflicts from permission/validation errors","Watch Glue throttling quotas under heavy write load and add backoff"],"tags":["aws","glue","commit","concurrency","retry"],"backgroundTag":"commit-conflict","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}