Received unexpected failure when committing to

Error message

Received unexpected failure when committing to {}, validating if commit ended up succeeding.

What it means

A warning from DynamoDbTableOperations.doCommit: the persist call to DynamoDB failed, and since the failure was either unexpected (not a ConditionalCheckFailedException) or a conditional-check failure after retries were performed, the commit outcome is ambiguous. The code calls checkCommitStatus to read back the table entry and determine whether the commit actually succeeded before deciding what exception to throw.

Solutions

Inspect the logged persistFailure cause (throttling, access denied, etc.) and address the underlying DynamoDB issue
Enable retries/backoff and adequate table capacity to reduce transient failures
If commit-status reconciliation reports the commit succeeded, no action is needed; if it reports failure, retry the write idempotently
Check IAM policies allow the conditional PutItem/UpdateItem on the catalog table

Example fix

// no caller code change — fix the environment
// before: table with 1 RCU getting throttled
// after: switch to on-demand billing or raise capacity
dynamodb.updateTable(t -> t.tableName(tableName)
    .provisionedThroughput(p -> p.readCapacityUnits(50L).writeCapacityUnits(50L)));

Defensive patterns

Strategy: retry

Validate before calling

// pre-check IAM/capacity
// ensure the principal can conditionally write the catalog item
// and that table capacity/on-demand is enabled

Prevention

Configure SDK retry strategy with exponential backoff
Use on-demand DynamoDB billing for bursty commit traffic
Avoid many concurrent committers to the same table
On CommitStateUnknownException, verify the table's metadata location before re-committing

When it happens

Trigger: Any RuntimeException from the DynamoDB persist (network error, throttling, provisioned throughput exceeded, service outage), or a ConditionalCheckFailedException raised after the retry detector observed retries — meaning the commit may or may not have landed.

Common situations: DynamoDB throttling under load; transient network partitions between the job and DynamoDB; long-running commits where another writer already updated the record so retries exhaust; IAM permission issues causing unexpected errors.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/f72ea196699e98e0. Report an issue: GitHub.

Appendix: source

Thrown at aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbTableOperations.java:132

              GetItemRequest.builder()
                  .tableName(awsProperties.dynamoDbTableName())
                  .consistentRead(true)
                  .key(tableKey)
                  .build());
      checkMetadataLocation(table, base);
      Map<String, String> properties = prepareProperties(table, newMetadataLocation);
      persistTable(tableKey, table, properties, retryDetector);
      commitStatus = CommitStatus.SUCCESS;
    } catch (CommitFailedException e) {
      // any explicit commit failures are passed up and out to the retry handler
      throw e;
    } catch (RuntimeException persistFailure) {
      boolean conditionCheckFailed = persistFailure instanceof ConditionalCheckFailedException;

      // If we got an exception we weren't expecting, or we got a ConditionalCheckFailedException
      // but retries were performed, attempt to reconcile the actual commit status.
      if (!conditionCheckFailed || retryDetector.retried()) {
        LOG.warn(
            "Received unexpected failure when committing to {}, validating if commit ended up succeeding.",
            fullTableName,
            persistFailure);
        commitStatus = checkCommitStatus(newMetadataLocation, metadata);
      }

      if (commitStatus != CommitStatus.SUCCESS && conditionCheckFailed) {
        throw new CommitFailedException(
            persistFailure, "Cannot commit %s: concurrent update detected", tableName());
      }

      switch (commitStatus) {
        case SUCCESS:
          break;
        case FAILURE:
          throw new CommitFailedException(
              persistFailure, "Cannot commit %s due to unexpected exception", tableName());
        case UNKNOWN:

View on GitHub (pinned to 86d9c8fc54)