{"record":{"id":"2fd81d3a925f7e19","repo":"apache/iceberg","slug":"for-table-with-at-exception-proces-2fd81d","errorCode":null,"errorMessage":"[For table {} with {}[{}] at {}]: Exception processing {}","messagePattern":"\\[For table (.+?) with (.+?)\\[(.+?)\\] at (.+?)\\]: Exception processing (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java","lineNumber":118,"sourceCode":"\n  @Override\n  public void processElement(StreamRecord<DataFileRewriteRunner.ExecutedGroup> streamRecord) {\n    DataFileRewriteRunner.ExecutedGroup executedGroup = streamRecord.getValue();\n    try {\n      if (commitService == null) {\n        // Refresh the table to get the latest snapshot for the committer\n        table.refresh();\n\n        FlinkRewriteDataFilesCommitManager commitManager =\n            new FlinkRewriteDataFilesCommitManager(\n                table, executedGroup.snapshotId(), streamRecord.getTimestamp(), branch);\n        this.commitService = commitManager.service(executedGroup.groupsPerCommit());\n        commitService.start();\n      }\n\n      commitService.offer(executedGroup.group());\n    } catch (Exception e) {\n      LOG.warn(\n          DataFileRewritePlanner.MESSAGE_PREFIX + \"Exception processing {}\",\n          tableName,\n          taskName,\n          taskIndex,\n          streamRecord.getTimestamp(),\n          executedGroup,\n          e);\n      output.collect(TaskResultAggregator.ERROR_STREAM, new StreamRecord<>(e));\n      errorCounter.inc();\n    }\n  }\n\n  @Override\n  public void processWatermark(Watermark mark) throws Exception {\n    try {\n      if (commitService != null) {\n        commitService.close();\n      }","sourceCodeStart":100,"sourceCodeEnd":136,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewriteCommitter.java#L100-L136","documentation":"DataFileRewriteCommitter.processElement offers each executed rewrite file group to a background commit service that commits rewritten data files to the table. Any Exception in that processing (offering, starting, or the commit path) is logged with this prefixed warning, enriched with the table name, task name/index, record timestamp, and the offending executed group, so the group can be identified in logs. The message follows DataFileRewritePlanner.MESSAGE_PREFIX for cross-operator correlation.","triggerScenarios":"processElement receives an ExecutedGroup result and either commitManager.service(...)/commitService.start() or commitService.offer(executedGroup.group()) throws — e.g. the rewritten group references files that no longer exist or the table changed concurrently.","commonSituations":"A table was modified (compacted/expired) between the rewrite planner and committer stages so rewritten files are stale; concurrent commits conflict; snapshot expiration deleted files the group still references.","solutions":["Find the full stack trace in taskmanager logs keyed by the message prefix and the table/task fields to identify the root cause.","Re-run the maintenance procedure after ensuring no concurrent table rewrites/expiry are running against the same table.","Check for a commit conflict (another writer committed concurrently) and retry the rewrite group.","Verify the rewritten data files still exist in storage (snapshot expiration may have removed them mid-run)."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  commitService.offer(executedGroup.group());\n} catch (Exception e) {\n  LOG.warn(DataFileRewritePlanner.MESSAGE_PREFIX + \"Exception processing {}\",\n      tableName, taskName, taskIndex, timestamp, executedGroup, e);\n  // re-run the maintenance procedure for the affected table/group\n}","preventionTips":["Avoid running table expiration/compaction concurrently with Flink rewrite maintenance.","Re-run failed maintenance procedures; they are idempotent per rewrite group.","Correlate failures via the MESSAGE_PREFIX and tableName/task fields in logs."],"tags":["flink","maintenance","rewrite","commit"],"backgroundTag":"rewrite-commit-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}