{"record":{"id":"1dc4340307ae1e84","repo":"apache/iceberg","slug":"exception-planning-scan-for-at-1dc434","errorCode":null,"errorMessage":"Exception planning scan for {} at {}","messagePattern":"Exception planning scan for (.+?) at (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MetadataTablePlanner.java","lineNumber":101,"sourceCode":"        ThreadPools.newFixedThreadPool(table.name() + \"-table-planner\", workerPoolSize);\n    this.splitSerializer = new IcebergSourceSplitSerializer(scanContext.caseSensitive());\n    this.errorCounter =\n        TableMaintenanceMetrics.groupFor(\n                getRuntimeContext(), originalTable.name(), taskName, taskIndex)\n            .counter(TableMaintenanceMetrics.ERROR_COUNTER);\n  }\n\n  @Override\n  public void processElement(Trigger trigger, Context ctx, Collector<SplitInfo> out)\n      throws Exception {\n    try {\n      table.refresh();\n      for (IcebergSourceSplit split :\n          FlinkSplitPlanner.planIcebergSourceSplits(table, scanContext, workerPool)) {\n        out.collect(new SplitInfo(splitSerializer.getVersion(), splitSerializer.serialize(split)));\n      }\n    } catch (Exception e) {\n      LOG.warn(\"Exception planning scan for {} at {}\", table, ctx.timestamp(), e);\n      ctx.output(DeleteOrphanFiles.ERROR_STREAM, e);\n      errorCounter.inc();\n    }\n  }\n\n  @Override\n  public void close() throws Exception {\n    super.close();\n    tableLoader.close();\n    if (workerPool != null) {\n      workerPool.shutdown();\n    }\n  }\n\n  public static class SplitInfo {\n    private final int version;\n    private final byte[] split;\n","sourceCodeStart":83,"sourceCodeEnd":119,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MetadataTablePlanner.java#L83-L119","documentation":"MetadataTablePlanner.processElement plans scans over Iceberg metadata tables (e.g. all_manifests, all_data_files) for the orphan-files flow and emits serialized splits. If planning fails — table refresh, scan planning via FlinkSplitPlanner, or serialization — the exception is logged with the table and timestamp, sent to the DeleteOrphanFiles ERROR_STREAM, and counted; the maintenance cycle continues without splits for that trigger.","triggerScenarios":"Raised in processElement after table.refresh() or FlinkSplitPlanner.planIcebergSourceSplits(table, scanContext, workerPool) throws — invalid scan context, catalog refresh failure, corrupted metadata, or worker-pool errors during split planning.","commonSituations":"Catalog connectivity or auth failure on refresh; very large tables exhausting planning memory/time; snapshot expired concurrently and removed mid-plan; invalid DeleteOrphanFiles scan configuration (e.g. bad maxCurrentlyReferencedFiles-style options).","solutions":["Inspect the exception on the ERROR_STREAM side output to identify whether refresh or split planning failed","Verify catalog access and that the table metadata is readable from the TaskManager","Increase planning resources (worker pool size, TaskManager memory) for very large metadata tables","Retry the maintenance job — the next trigger re-plans with a refreshed table"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Same side-output pattern as other maintenance operators\nresult.getSideOutput(DeleteOrphanFiles.ERROR_STREAM)\n      .executeAndCollect().forEachRemaining(e -> log.error(\"plan failed\", e));","preventionTips":["Size TaskManager memory/worker pool for large metadata-table scans","Verify catalog access before scheduling maintenance","Don't expire snapshots concurrently with scan planning","Alert on errorCounter metrics to detect persistent planning failures"],"tags":["flink","scan-planning","metadata-table","orphan-files"],"backgroundTag":"scan-planning-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}