apache/iceberg · error · CommitFailedException

Cannot commit %s because Glue detected concurrent update

Error message

Cannot commit %s because Glue detected concurrent update

What it means

CommitFailedException thrown by GlueTableOperations.handleAWSExceptions when the AWS Glue UpdateTable call fails with ConcurrentModificationException. Glue itself detected that the table was modified between its GetTable and UpdateTable calls, so the conditional update was rejected to prevent lost updates. It is a retryable commit failure from the catalog side.

Source

Thrown at aws/src/main/java/org/apache/iceberg/aws/glue/GlueTableOperations.java:355

              .overrideConfiguration(c -> c.addMetricPublisher(retryDetector))
              .catalogId(awsProperties.glueCatalogId())
              .databaseName(databaseName)
              .tableInput(
                  TableInput.builder()
                      .applyMutation(
                          builder ->
                              IcebergToGlueConverter.setTableInputInformation(builder, metadata))
                      .name(tableName)
                      .tableType(GLUE_EXTERNAL_TABLE_TYPE)
                      .parameters(parameters)
                      .build())
              .build());
    }
  }

  private void handleAWSExceptions(AwsServiceException persistFailure) {
    if (persistFailure instanceof ConcurrentModificationException) {
      throw new CommitFailedException(
          persistFailure, "Cannot commit %s because Glue detected concurrent update", tableName());
    } else if (persistFailure
        instanceof software.amazon.awssdk.services.glue.model.AlreadyExistsException) {
      throw new AlreadyExistsException(
          persistFailure,
          "Cannot commit %s because its Glue table already exists when trying to create one",
          tableName());
    } else if (persistFailure instanceof EntityNotFoundException) {
      throw new NotFoundException(
          persistFailure,
          "Cannot commit %s because Glue cannot find the requested entity",
          tableName());
    } else if (persistFailure instanceof AccessDeniedException) {
      throw new ForbiddenException(
          persistFailure,
          "Cannot commit %s because Glue cannot access the requested resources",
          tableName());
    } else if (persistFailure

View on GitHub (pinned to 86d9c8fc54)

Solutions

Retry the operation — this is a transient conflict and Iceberg's commit retry will refresh metadata and re-commit.
Reduce commit contention: batch appends, increase checkpoint intervals in Flink/Spark streaming, or stagger scheduled maintenance jobs.
Check for non-Iceberg actors (Glue Crawlers, Terraform, manual console edits) modifying the table and exclude the table from them.

Example fix

// before
streamingQuery.trigger(ProcessingTime("1 second")); // very frequent commits -> Glue conflicts
// after
streamingQuery.trigger(ProcessingTime("1 minute")); // fewer, batched commits reduce concurrent update conflicts

Defensive patterns

Strategy: retry

Try / catch

try {
  table.commit();
} catch (CommitFailedException e) {
  // Glue ConcurrentModificationException: back off and retry
  Thread.sleep(backoffMs);
  table.refresh();
  table.commit();
}

Prevention

Increase streaming checkpoint/commit intervals to reduce commit frequency
Schedule compaction away from peak writer traffic
Exclude Iceberg-managed tables from Glue Crawlers and manual edits
Use Iceberg's automatic retry configuration instead of one-shot commits

When it happens

Trigger: doCommit → persistGlueTable → glue.updateTable raises software.amazon.awssdk.services.glue.model.ConcurrentModificationException because another client updated the Glue table concurrently.

Common situations: Multiple Iceberg writers committing to the same table at the same moment; external processes (Crawlers, Athena DDL) touching the Glue table while an Iceberg commit is in flight; high-frequency commits from a streaming job overlapping with compaction.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/c353c879f31e417a. Report an issue: GitHub.