What causes "[For table {} with {}[{}] at {}]: Exception closing commit service" in apache/iceberg?

Inspect the attached exception for the underlying commit failure

[For table {} with {}[{}] at {}]: Exception closing commit s

Error message

[For table {} with {}[{}] at {}]: Exception closing commit service

What it means

DataFileRewriteCommitter.processWatermark catches Exceptions while closing the commit service at the watermark (final flush of pending rewrite groups) and logs this warning with table/task/timestamp context. Pending groups that could not be committed are lost for this window.

Source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java:145

      errorCounter.inc();
    }
  }

  @Override
  public void processWatermark(Watermark mark) throws Exception {
    try {
      if (commitService != null) {
        commitService.close();
      }

      LOG.info(
          DataFileRewritePlanner.MESSAGE_PREFIX + "Successfully completed data file compaction",
          tableName,
          taskName,
          taskIndex,
          mark.getTimestamp());
    } catch (Exception e) {
      LOG.warn(
          DataFileRewritePlanner.MESSAGE_PREFIX + "Exception closing commit service",
          tableName,
          taskName,
          taskIndex,
          mark.getTimestamp(),
          e);
      output.collect(TaskResultAggregator.ERROR_STREAM, new StreamRecord<>(e));
      errorCounter.inc();
    }

    // Cleanup
    this.commitService = null;

    super.processWatermark(mark);
  }

  @Override
  public void close() throws IOException {

View on GitHub (pinned to 86d9c8fc54)

Solutions

Inspect the attached exception for the underlying commit failure
Re-run data file compaction for the table (affected groups were not committed)
Check catalog availability and concurrent-writer conflicts at commit time
Verify FileIO/storage health before the next maintenance window

Defensive patterns

Strategy: retry

Validate before calling

// verify catalog availability before maintenance window
 catalog.table(name).refresh();

Try / catch

try {
  commitService.close();
} catch (Exception e) {
  LOG.warn(MESSAGE_PREFIX + "Exception closing commit service", ...);
  // re-run compaction for uncommitted groups
}

Prevention

Ensure catalog and storage are healthy across the maintenance window
Schedule maintenance outside peak commit periods to avoid conflicts

When it happens

Trigger: Exception thrown by commitService.close()/commit flush in processWatermark — e.g. commit failure against the table while finalizing compaction results on the watermark.

Common situations: Catalog unavailable or commit conflict at the end of a maintenance window; FileIO errors writing manifest/commit data; expired snapshot references.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/cce76a7d913bc56a. Report an issue: GitHub.