apache/iceberg · error · UncheckedIOException

Failed to process tasks iterable

Error message

Failed to process tasks iterable

What it means

FlinkSplitPlanner.planInputSplits (the legacy Flink source path) iterates a CloseableIterable of CombinedScanTasks to build FlinkInputSplits; IOExceptions during task iteration (which lazily perform file IO) are wrapped into an UncheckedIOException 'Failed to process tasks iterable'.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/source/FlinkSplitPlanner.java:68

      List<CombinedScanTask> tasks = Lists.newArrayList(tasksIterable);
      FlinkInputSplit[] splits = new FlinkInputSplit[tasks.size()];
      boolean exposeLocality = context.exposeLocality();

      Tasks.range(tasks.size())
          .stopOnFailure()
          .executeWith(exposeLocality ? workerPool : null)
          .run(
              index -> {
                CombinedScanTask task = tasks.get(index);
                String[] hostnames = null;
                if (exposeLocality) {
                  hostnames = Util.blockLocations(table.io(), task);
                }
                splits[index] = new FlinkInputSplit(index, task, hostnames);
              });
      return splits;
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to process tasks iterable", e);
    }
  }

  /** This returns splits for the FLIP-27 source */
  public static List<IcebergSourceSplit> planIcebergSourceSplits(
      Table table, ScanContext context, ExecutorService workerPool) {
    try (CloseableIterable<CombinedScanTask> tasksIterable =
        planTasks(table, context, workerPool)) {
      return Lists.newArrayList(
          CloseableIterable.transform(tasksIterable, IcebergSourceSplit::fromCombinedScanTask));
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to process task iterable: ", e);
    }
  }

  static CloseableIterable<CombinedScanTask> planTasks(
      Table table, ScanContext context, ExecutorService workerPool) {
    ScanMode scanMode = checkScanMode(context);

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check the wrapped IOException cause for the specific storage/IO failure
2. Avoid expiring/rewriting data files between job planning and execution
3. Verify table metadata and manifest files are readable with the configured FileIO credentials
4. Retry split planning once storage is reachable again

Table table = loader.loadTable();
table.refresh();
table.scan().planTasks(); // surfaces IO issues before split planning

try {
  FlinkInputSplit[] splits = FlinkSplitPlanner.planInputSplits(table, context, pool);
} catch (UncheckedIOException e) {
  // transient storage failure: retry with backoff
  retryWithBackoff(() -> FlinkSplitPlanner.planInputSplits(table, context, pool));
}

When it happens

Trigger: Calling FlinkSplitPlanner.planInputSplits(table, context, workerPool) when the lazy task iterable throws while being consumed — e.g. manifest listing/IO failure, table files deleted between planning and materialization, or block location lookup failures via Util.blockLocations.

Common situations: Files removed by concurrent compaction/expiry while the source plans splits; storage outages during manifest reading; misconfigured FileIO; permission errors on metadata files.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to list partitions of table %s
• Failed to plan files for main index
• Failed to plan files for main index
• Failed to plan files for main index
• Failed to list partitions of table %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/ec5243636b585cb6. Report an issue: GitHub.