apache/iceberg · error

[For table {} with {}[{}] at {}]: Exception closing commit s

Error message

[For table {} with {}[{}] at {}]: Exception closing commit service

What it means

DataFileRewriteCommitter.processWatermark() closes the CommitService, which performs the actual Iceberg snapshot commit of all offered rewrite file groups. If commitService.close() throws (commit failure), it is logged with the maintenance MESSAGE_PREFIX and sent to the TaskResultAggregator error stream; the commit service reference is cleared so the next trigger starts fresh.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java:145

      errorCounter.inc();
    }
  }

  @Override
  public void processWatermark(Watermark mark) throws Exception {
    try {
      if (commitService != null) {
        commitService.close();
      }

      LOG.info(
          DataFileRewritePlanner.MESSAGE_PREFIX + "Successfully completed data file compaction",
          tableName,
          taskName,
          taskIndex,
          mark.getTimestamp());
    } catch (Exception e) {
      LOG.warn(
          DataFileRewritePlanner.MESSAGE_PREFIX + "Exception closing commit service",
          tableName,
          taskName,
          taskIndex,
          mark.getTimestamp(),
          e);
      output.collect(TaskResultAggregator.ERROR_STREAM, new StreamRecord<>(e));
      errorCounter.inc();
    }

    // Cleanup
    this.commitService = null;

    super.processWatermark(mark);
  }

  @Override
  public void close() throws IOException {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Read the chained exception: for CommitStateUnknownException, check the table's snapshots to determine whether the commit landed before retrying
2. Re-run maintenance on the next trigger — the operator resets commitService and the rewrite is re-planned
3. Serialize table writers: ensure no other job (Spark rewrite procedure, other maintenance job) commits to the same branch during the window
4. Check catalog health/quota (Hive metastore, REST catalog) if commits fail repeatedly
5. Refresh stale table metadata so commit validations see the current branch state

Example fix

// before: assuming the commit failed and blindly re-running produces duplicate rewrites
boolean committed = false;

// after: resolve CommitStateUnknown by inspecting snapshots before retry
try {
  commitService.close();
} catch (Exception e) {
  if (e.getCause() instanceof CommitStateUnknownException) {
    // check table.snapshots() for the rewritten files to decide retry vs skip
    resolveCommitState(table, e);
  } else {
    throw e;
  }
}

// before scheduling, confirm only one writer commits to the target branch
if (!lockFactory.createLock().tryLock()) {
  LOG.warn("Another maintenance run is active; skipping");
} else {
  Table table = tableLoader.loadTable();
  Snapshot head = table.snapshot(branch); // record head to detect concurrent commits later
}

// handle CommitStateUnknown explicitly when consuming the error side-output
if (e.getCause() instanceof CommitStateUnknownException) {
  // inspect table.snapshots() to determine if the commit landed; only then retry
  resolveCommitStateAndAlert(table, e);
} else {
  alertOps("Commit failed at watermark: " + e.getMessage());
}

When it happens

Trigger: Closing the CommitService at the watermark when the underlying commitFileGroups fails: CommitStateUnknownException from the catalog (commit may or may not have applied), catalog write rejection, branch snapshot conflict because the starting snapshot changed, or storage errors writing metadata.

Common situations: Two jobs rewriting the same table concurrently causing commit conflicts; REST/Hive catalog returning 5xx or timeouts during commit leaving state unknown; schema/partition spec changed between plan and commit; branch fast-forwarded or reset by another writer.

Related errors

• [For table {} with {}[{}] at {}]: Exception processing {}
• Failed to create tableMaintenance
• Unexpected delete file content:
• Failed to create tableMaintenance
• Skipping commit for table {} task {}: a DV writer reported a

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/3a6a355015072fba. Report an issue: GitHub.