{"record":{"id":"fbe9563fe4597e0b","repo":"apache/iceberg","slug":"for-table-with-at-failed-to-plan-d","errorCode":null,"errorMessage":"[For table {} with {}[{}] at {}]: Failed to plan data file rewrite groups","messagePattern":"\\[For table (.+?) with (.+?)\\[(.+?)\\] at (.+?)\\]: Failed to plan data file rewrite groups","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewritePlanner.java","lineNumber":184,"sourceCode":"          taskName,\n          taskIndex,\n          ctx.timestamp(),\n          groups.size(),\n          groups);\n      plannedGroupsCounter.inc(groups.size());\n\n      for (RewriteFileGroup group : groups) {\n        LOG.info(\n            DataFileRewritePlanner.MESSAGE_PREFIX + \"Emitting {}\",\n            tableName,\n            taskName,\n            taskIndex,\n            ctx.timestamp(),\n            group);\n        out.collect(new PlannedGroup(table, groupsPerCommit, group, branch));\n      }\n    } catch (Exception e) {\n      LOG.warn(\n          DataFileRewritePlanner.MESSAGE_PREFIX + \"Failed to plan data file rewrite groups\",\n          tableName,\n          taskName,\n          taskIndex,\n          ctx.timestamp(),\n          e);\n      ctx.output(TaskResultAggregator.ERROR_STREAM, e);\n      errorCounter.inc();\n    }\n  }\n\n  @Override\n  public void close() throws Exception {\n    super.close();\n    tableLoader.close();\n  }\n\n  public static class PlannedGroup {","sourceCodeStart":166,"sourceCodeEnd":202,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/DataFileRewritePlanner.java#L166-L202","documentation":"DataFileRewritePlanner.processElement() builds a BinPackRewriteFilePlanner over the table's current snapshot and emits PlannedGroup records. Any exception while loading the table, resolving the branch snapshot, initializing the planner with rewriter options, executing the plan, or iterating groups is caught, logged with the maintenance MESSAGE_PREFIX, and forwarded to the TaskResultAggregator error stream. No PlannedGroups are emitted for that trigger cycle.","triggerScenarios":"tableLoader.loadTable() or SerializableTable.copyOf fails (catalog unavailable, bad table name); table.snapshot(branch) fails for a nonexistent branch; planner.init(rewriterOptions) rejects an invalid option value; filterSupplier.get() throws; planner.plan() fails reading manifests/metrics; partialProgressMaxCommits <= 0 breaking the divide into groupsPerCommit.","commonSituations":"Typo'd or contradictory rewriter options (min-file-size > max-file-size, non-numeric partial-progress.max-commits) rejected at init; branch renamed/deleted; catalog outage or expired credentials on the TaskManager; table moved or dropped between job submission and trigger.","solutions":["Check the logged cause — planner.init option errors state the offending property; fix RewriterConfig values (min-file-size < max-file-size, positive partial-progress.max-commits)","Verify the configured branch exists and the table/catalog is reachable from the Flink cluster","Re-run the maintenance job; the operator records the error via the error counter and the next trigger re-plans","Validate table location and catalog options in TableLoader (Hive/Hadoop/REST catalog config)","Confirm the filterSupplier returns a valid serializable Expression and does not throw"],"exampleFix":"// before: invalid rewriter options crash planning every trigger\nMap<String, String> opts = ImmutableMap.of(\n    \"rewrite-data-files.min-file-size\", \"1GB\",\n    \"rewrite-data-files.max-file-size\", \"512MB\"); // min > max\n\n// after: valid bin-pack options\nMap<String, String> opts = ImmutableMap.of(\n    \"rewrite-data-files.min-file-size\", \"128MB\",\n    \"rewrite-data-files.max-file-size\", \"1GB\",\n    \"partial-progress.max-commits\", \"10\");","handlingStrategy":"validation","validationCode":"// validate options and branch before building the maintenance job\nTable table = tableLoader.loadTable();\nPreconditions.checkArgument(table.snapshot(branch) != null || table.snapshots().isEmpty(),\n    \"Branch %s does not exist\", branch);\nint maxCommits = Integer.parseInt(opts.get(\"partial-progress.max-commits\"));\nPreconditions.checkArgument(maxCommits > 0, \"partial-progress.max-commits must be > 0\");\nPreconditions.checkArgument(minFileSizeBytes < maxFileSizeBytes, \"min-file-size must be < max-file-size\");","typeGuard":null,"tryCatchPattern":"// planner errors surface on the error side-output stream; gate retries on it\nDataStream<Exception> errors = result.getSideOutput(TaskResultAggregator.ERROR_STREAM);\nerrors.process((ProcessFunction<Exception, Void>) (e, ctx) -> {\n  LOG.warn(\"Rewrite planning failed, next trigger will re-plan\", e);\n  return null;\n});","preventionTips":["Validate rewriter options (min/max file sizes, partial-progress.max-commits) before job submission — planner.init rejects invalid values at trigger time","Verify the branch name exists after any table renames or branch management operations","Test table loadability (catalog URL, credentials) from the Flink runtime environment","Ensure the filterSupplier returns a valid serializable Expression and does not throw"],"tags":["flink","maintenance","planning","iceberg"],"backgroundTag":"invalid-config-value","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}