apache/iceberg · error · CommitFailedException

Cannot commit due to unexpected exception

Error message

Cannot commit %s due to unexpected exception

What it means

CommitFailedException (with persistFailure as cause) thrown in doCommit when checkCommitStatus returns CommitStatus.FAILURE, meaning the persist step failed with an unexpected exception and the commit definitely did not succeed. The original failure is chained.

Solutions

1. Inspect the chained cause (persistFailure) — fix the underlying persist error first
2. Retry the commit after refreshing the table
3. Check DynamoDB capacity/provisioned throughput and IAM write permissions on the warehouse location

Example fix

// before
// cause hidden in logs
// after
try {
  table.append(df);
} catch (CommitFailedException e) {
  LOG.error("persist failure cause", e.getCause()); // inspect real failure
  throw e;
}

// pre-check permissions and throughput
 dynamo.describeTable(b -> b.tableName(catalogTable)); // accessible?
 // verify warehouse location is writable via FileIO

try {
  table.append(df);
} catch (CommitFailedException e) {
  Throwable cause = e.getCause(); // real persist failure
  LOG.error("Persist failed", cause);
  if (isTransient(cause)) {
    table.refresh();
    table.append(df);
  } else {
    throw e;
  }
}

When it happens

Trigger: The conditional Dynamo update (or the metadata file write) threw persistFailure, and the subsequent commit-status check determined the commit definitively failed.

Common situations: DynamoDB throughput exceeded or throttling during putItem; transient network failure writing the metadata JSON file; IAM permission missing for the metadata location.

Related errors

• Cannot commit because base metadata location ' ' is not…
• Cannot commit : concurrent update detected
• Cannot create namespace
• Cannot delete non-empty namespace
• Cannot find default warehouse location: namespace

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/a085da0b12949188. Report an issue: GitHub.

Appendix: source

Thrown at aws/src/main/java/org/apache/iceberg/aws/dynamodb/DynamoDbTableOperations.java:148

      // but retries were performed, attempt to reconcile the actual commit status.
      if (!conditionCheckFailed || retryDetector.retried()) {
        LOG.warn(
            "Received unexpected failure when committing to {}, validating if commit ended up succeeding.",
            fullTableName,
            persistFailure);
        commitStatus = checkCommitStatus(newMetadataLocation, metadata);
      }

      if (commitStatus != CommitStatus.SUCCESS && conditionCheckFailed) {
        throw new CommitFailedException(
            persistFailure, "Cannot commit %s: concurrent update detected", tableName());
      }

      switch (commitStatus) {
        case SUCCESS:
          break;
        case FAILURE:
          throw new CommitFailedException(
              persistFailure, "Cannot commit %s due to unexpected exception", tableName());
        case UNKNOWN:
          throw new CommitStateUnknownException(persistFailure);
      }
    } finally {
      try {
        if (commitStatus == CommitStatus.FAILURE) {
          // if anything went wrong, clean up the uncommitted metadata file
          io().deleteFile(newMetadataLocation);
        }
      } catch (RuntimeException e) {
        LOG.error("Failed to cleanup metadata file at {}", newMetadataLocation, e);
      }
    }
  }

  private void checkMetadataLocation(GetItemResponse table, TableMetadata base) {
    String dynamoMetadataLocation = table.hasItem() ? getMetadataLocation(table) : null;

View on GitHub (pinned to 86d9c8fc54)