{"record":{"id":"54070190198c2340","repo":"apache/iceberg","slug":"exception-planning-scan-for-at","errorCode":null,"errorMessage":"Exception planning scan for {} at {}","messagePattern":"Exception planning scan for (.+?) at (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MetadataTablePlanner.java","lineNumber":101,"sourceCode":"        ThreadPools.newFixedThreadPool(table.name() + \"-table-planner\", workerPoolSize);\n    this.splitSerializer = new IcebergSourceSplitSerializer(scanContext.caseSensitive());\n    this.errorCounter =\n        TableMaintenanceMetrics.groupFor(\n                getRuntimeContext(), originalTable.name(), taskName, taskIndex)\n            .counter(TableMaintenanceMetrics.ERROR_COUNTER);\n  }\n\n  @Override\n  public void processElement(Trigger trigger, Context ctx, Collector<SplitInfo> out)\n      throws Exception {\n    try {\n      table.refresh();\n      for (IcebergSourceSplit split :\n          FlinkSplitPlanner.planIcebergSourceSplits(table, scanContext, workerPool)) {\n        out.collect(new SplitInfo(splitSerializer.getVersion(), splitSerializer.serialize(split)));\n      }\n    } catch (Exception e) {\n      LOG.warn(\"Exception planning scan for {} at {}\", table, ctx.timestamp(), e);\n      ctx.output(DeleteOrphanFiles.ERROR_STREAM, e);\n      errorCounter.inc();\n    }\n  }\n\n  @Override\n  public void close() throws Exception {\n    super.close();\n    tableLoader.close();\n    if (workerPool != null) {\n      workerPool.shutdown();\n    }\n  }\n\n  public static class SplitInfo {\n    private final int version;\n    private final byte[] split;\n","sourceCodeStart":83,"sourceCodeEnd":119,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MetadataTablePlanner.java#L83-L119","documentation":"A WARN log emitted when the MetadataTablePlanner operator fails to plan a scan over a metadata table (e.g. all_manifests, all_data_files) used by the orphan-file/rewrite task. After refreshing the table, split planning via FlinkSplitPlanner threw; the exception is sent to the DeleteOrphanFiles ERROR_STREAM and the errorCounter is incremented, so the task can be marked failed downstream.","triggerScenarios":"processElement(): table.refresh() or FlinkSplitPlanner.planIcebergSourceSplits(table, scanContext, workerPool) throws — schema/format incompatibility, corrupt metadata, planning-time I/O reading manifests, or worker-pool issues.","commonSituations":"Reading metadata of a table written by a newer Iceberg version (unsupported spec/manifest format); corrupt or truncated manifest/metadata files; S3/HDFS read errors while opening manifests; table concurrently expiring snapshots so metadata disappears mid-plan.","solutions":["Check the underlying exception in the side output for unsupported-format or corrupt-file clues; upgrade the Iceberg runtime if the table uses a newer format version.","Verify all manifests/metadata files are readable with the job's credentials; repair with orphan cleanup or restore metadata if corrupted.","Avoid running concurrent expire/orphan jobs that delete metadata while the planner reads it.","Rerun the maintenance task — planning is retryable once the transient I/O issue clears."],"exampleFix":"// before: plan directly against possibly-stale concurrent state\ntable.refresh();\nplanIcebergSourceSplits(table, scanContext, workerPool);\n// after: serialize maintenance runs / pin snapshot before planning\nTable stable = table;\nstable.refresh();\nscanContext = scanContext.useSnapshot(stable.currentSnapshot().snapshotId());","handlingStrategy":"retry","validationCode":"// refresh and confirm the table is readable before planning\ntable.refresh();\nif (table.currentSnapshot() == null && table.schema().columns().isEmpty()) {\n  throw new IllegalStateException(\"Table metadata unreadable, aborting scan planning\");\n}","typeGuard":"boolean plannable(Table table) {\n  try { table.refresh(); return table.operations().current() != null; }\n  catch (RuntimeException e) { return false; }\n}","tryCatchPattern":"try {\n  FlinkSplitPlanner.planIcebergSourceSplits(table, scanContext, workerPool);\n} catch (Exception e) {\n  ctx.output(DeleteOrphanFiles.ERROR_STREAM, e);\n  errorCounter.inc();\n}","preventionTips":["Keep the Iceberg runtime version >= the format version of tables being scanned.","Do not run expire/orphan jobs concurrently with metadata-table scans.","Ensure manifests are readable (permissions, object-store availability) before planning.","Consume the ERROR_STREAM and retry failed maintenance runs after transient I/O issues."],"tags":["flink","scan-planning","metadata-table","orphan-files"],"backgroundTag":"database-query-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}