apache/iceberg · error

[For table {} with {}[{}] at {}]: Exception processing {}

Error message

[For table {} with {}[{}] at {}]: Exception processing {}

What it means

DataFileRewriteCommitter.processElement() feeds executed rewrite groups into the RewriteDataFilesCommitManager's CommitService. Any exception while refreshing the table, creating/starting the commit service, or offering the group is logged with the maintenance MESSAGE_PREFIX and routed to the TaskResultAggregator error stream instead of failing the operator. The message names the offending ExecutedGroup so the failed rewrite batch is identifiable.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java:118

  @Override
  public void processElement(StreamRecord<DataFileRewriteRunner.ExecutedGroup> streamRecord) {
    DataFileRewriteRunner.ExecutedGroup executedGroup = streamRecord.getValue();
    try {
      if (commitService == null) {
        // Refresh the table to get the latest snapshot for the committer
        table.refresh();

        FlinkRewriteDataFilesCommitManager commitManager =
            new FlinkRewriteDataFilesCommitManager(
                table, executedGroup.snapshotId(), streamRecord.getTimestamp(), branch);
        this.commitService = commitManager.service(executedGroup.groupsPerCommit());
        commitService.start();
      }

      commitService.offer(executedGroup.group());
    } catch (Exception e) {
      LOG.warn(
          DataFileRewritePlanner.MESSAGE_PREFIX + "Exception processing {}",
          tableName,
          taskName,
          taskIndex,
          streamRecord.getTimestamp(),
          executedGroup,
          e);
      output.collect(TaskResultAggregator.ERROR_STREAM, new StreamRecord<>(e));
      errorCounter.inc();
    }
  }

  @Override
  public void processWatermark(Watermark mark) throws Exception {
    try {
      if (commitService != null) {
        commitService.close();
      }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Inspect the chained cause in the log to find the root failure (missing files vs catalog vs metadata)
2. Verify the rewritten data files still exist on storage and retention policies are not deleting them mid-run
3. Re-run the maintenance job — the error stream does not fail the job, so a subsequent trigger retries planning and rewriting
4. Check that the configured branch still exists and the starting snapshot was not expired concurrently
5. Validate catalog connectivity/credentials from the Flink TaskManager

Example fix

// before: concurrent expireSnapshots removes snapshots the rewriter depends on
TableMaintenance.builder().add(DataFileRewriteTask.create(maintenanceConfig))
    .add(ExpireSnapshotsTask.create(expireConfig)) // aggressive retention
    .schedule(...);

// after: keep snapshot retention window larger than the rewrite execution window
ExpireSnapshots expire = ExpireSnapshots.builder()...
    .retainLast(1).olderThan(System.currentTimeMillis() - MIN_SNAPSHOT_RETENTION_MS)
    .apply();

// before the maintenance run, verify the table and branch are reachable
TableLoader tableLoader = TableLoader.fromCatalog(...);
tableLoader.open();
Table table = tableLoader.loadTable(); // throws early if catalog/table unavailable
if (table.snapshot(branch) == null && !table.currentSnapshot().isNullSnapshot()) {
  LOG.warn("Branch {} missing; rewrite committer may fail", branch);
}

// the operator already routes errors to the side-output; monitor and react
DataStream<Exception> errors = result.getSideOutput(TaskResultAggregator.ERROR_STREAM);
errors.process((ProcessFunction<Exception, Void>) (e, ctx) -> {
  alertOps("Rewrite committer failure: " + e.getMessage());
  return null;
});

When it happens

Trigger: table.refresh() fails (catalog/network errors loading new metadata); constructing FlinkRewriteDataFilesCommitManager or starting the CommitService fails; commitService.offer(group) throws because the rewrite output files are missing or unreadable, or the group's starting snapshotId no longer exists.

Common situations: Rewritten data files deleted by aggressive retention or lost because the writer task's output dir was cleaned; catalog (Hive/REST/Hadoop) temporarily unreachable during refresh; concurrent expireSnapshots removing the starting snapshot; wrong branch configured for the table.

Related errors

• [For table {} with {}[{}] at {}]: Exception closing commit s
• Failed to create tableMaintenance
• Unexpected delete file content:
• Failed to create tableMaintenance
• Skipping commit for table {} task {}: a DV writer reported a

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/ebf20ad2330871a2. Report an issue: GitHub.