apache/iceberg · error · UncheckedIOException

Failed to process tasks iterable

Error message

Failed to process tasks iterable

What it means

FlinkSplitPlanner.planInputSplits materializes CombinedScanTask tasks into FlinkInputSplits; IOException while consuming the tasks iterable (including block location lookups via table.io()) is wrapped in UncheckedIOException. It indicates IO failure while reading table metadata or computing hostnames during legacy FlinkSource split planning.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/source/FlinkSplitPlanner.java:68

      List<CombinedScanTask> tasks = Lists.newArrayList(tasksIterable);
      FlinkInputSplit[] splits = new FlinkInputSplit[tasks.size()];
      boolean exposeLocality = context.exposeLocality();

      Tasks.range(tasks.size())
          .stopOnFailure()
          .executeWith(exposeLocality ? workerPool : null)
          .run(
              index -> {
                CombinedScanTask task = tasks.get(index);
                String[] hostnames = null;
                if (exposeLocality) {
                  hostnames = Util.blockLocations(table.io(), task);
                }
                splits[index] = new FlinkInputSplit(index, task, hostnames);
              });
      return splits;
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to process tasks iterable", e);
    }
  }

  /** This returns splits for the FLIP-27 source */
  public static List<IcebergSourceSplit> planIcebergSourceSplits(
      Table table, ScanContext context, ExecutorService workerPool) {
    try (CloseableIterable<CombinedScanTask> tasksIterable =
        planTasks(table, context, workerPool)) {
      return Lists.newArrayList(
          CloseableIterable.transform(tasksIterable, IcebergSourceSplit::fromCombinedScanTask));
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to process task iterable: ", e);
    }
  }

  static CloseableIterable<CombinedScanTask> planTasks(
      Table table, ScanContext context, ExecutorService workerPool) {
    ScanMode scanMode = checkScanMode(context);

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check the chained IOException cause for the underlying storage error
2. Verify storage credentials/connectivity for the configured FileIO
3. Ensure snapshot expiration does not race with split planning; pin snapshot-id if needed
4. Retry planning after transient storage errors; for S3 throttling, reduce workerPool parallelism

Example fix

// before
FlinkInputSplit[] splits = FlinkSplitPlanner.planInputSplits(table, context, workerPool);
// after: pin snapshot and check IO first
table.refresh();
long snapId = table.currentSnapshot().snapshotId();
ScanContext pinned = context.copyWithSnapshotId(snapId);
FlinkInputSplit[] splits = FlinkSplitPlanner.planInputSplits(table, pinned, workerPool);

table.refresh();
Preconditions.checkState(table.currentSnapshot() != null, "No snapshot available for planning");

try {
  splits = FlinkSplitPlanner.planInputSplits(table, context, pool);
} catch (UncheckedIOException e) {
  if (isTransient(e.getCause())) { splits = FlinkSplitPlanner.planInputSplits(table, context, pool); }
  else throw e;
}

When it happens

Trigger: Calling planInputSplits when table.io() cannot read manifests/metadata files, Util.blockLocations fails on the filesystem, or the CloseableIterable of tasks throws during iteration due to expired snapshots or unreachable storage.

Common situations: HDFS NameNode unavailability, S3 throttling/credentials errors during manifest reads, snapshot expiry race with compaction jobs, misconfigured IO implementation.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to create iceberg input splits for table:
• Failed to process task iterable:
• Failed to process tasks iterable
• Failed to process task iterable:
• Failed to create snapshot list writer for path: %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/3841c3812effb140. Report an issue: GitHub.