apache/iceberg · warning

[For table {} with {}[{}] at {}]: Exception closing commit s

Error message

[For table {} with {}[{}] at {}]: Exception closing commit service

What it means

This is a warning logged by DataFileRewriteCommitter.processWatermark in Iceberg's Flink maintenance (rewrite data files) flow when the rewrite commit service fails to close after a completed compaction. It is not a compaction failure itself; the commit already happened or was attempted, but the underlying service/executor (RewriteDataFilesCommitService) threw during close, typically meaning some commit resources or threads could not be shut down cleanly. Iceberg logs it with the table/task context instead of failing the Flink checkpoint.

Source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java:145

      errorCounter.inc();
    }
  }

  @Override
  public void processWatermark(Watermark mark) throws Exception {
    try {
      if (commitService != null) {
        commitService.close();
      }

      LOG.info(
          DataFileRewritePlanner.MESSAGE_PREFIX + "Successfully completed data file compaction",
          tableName,
          taskName,
          taskIndex,
          mark.getTimestamp());
    } catch (Exception e) {
      LOG.warn(
          DataFileRewritePlanner.MESSAGE_PREFIX + "Exception closing commit service",
          tableName,
          taskName,
          taskIndex,
          mark.getTimestamp(),
          e);
      output.collect(TaskResultAggregator.ERROR_STREAM, new StreamRecord<>(e));
      errorCounter.inc();
    }

    // Cleanup
    this.commitService = null;

    super.processWatermark(mark);
  }

  @Override
  public void close() throws IOException {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check the nested cause 'e' in the log for the real failure (often ValidationException from a conflicting commit) and re-run the rewrite when the table is quiescent
2. Ensure only one rewrite/maintenance job runs against the table at a time to avoid concurrent-commit conflicts
3. Retry the maintenance job; if transient thread interruption, verify TaskManager shutdown/checkpoint settings
4. If persistent, inspect FileIO close behavior for the configured object store and its timeouts

// Library-side; user monitors logs/alerts on MESSAGE_PREFIX + 'Exception closing commit service'
if (logMessage.contains("Exception closing commit service")) { alertMaintenanceFailure(tableName); scheduleRewriteRetry(); }

When it happens

Trigger: Triggered in DataFileRewriteCommitter.processWatermark when closeService() (closing the RewriteDataFilesCommitService after collecting rewrite results) throws any Exception — e.g. an underlying commit failure surfaced on close, thread interruption while shutting the executor, or IOException releasing FileIO resources.

Common situations: Table metadata changed concurrently between planning and commit (schema/spec update or another compaction job committing), causing commitInsideService failures that surface during close; TaskManager thread interruption at checkpoint boundaries; slow ObjectStore FileIO close timeouts.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• [For table {} with {}[{}] at {}]: Exception processing {}
• [For table {} with {}[{}] at {}]: Exception closing commit s
• Skipping commit for table {} task {}: a DV writer reported a
• Skipping commit for table {} task {}: a DV writer reported a
• [For table {} with {}[{}] at {}]: Exception processing {}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/44c782b19baa9e53. Report an issue: GitHub.