apache/iceberg · error · UncheckedIOException

Failed to process task iterable:

Error message

Failed to process task iterable: 

What it means

planIcebergSourceSplits plans splits for the FLIP-27 IcebergSource by iterating a CloseableIterable of CombinedScanTask produced by planTasks. An IOException while iterating the task iterable (closing it or transforming entries) is wrapped into an UncheckedIOException with this message. It means the scan planning pipeline failed on an I/O problem while enumerating scan tasks.

Source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/source/FlinkSplitPlanner.java:80

                  hostnames = Util.blockLocations(table.io(), task);
                }
                splits[index] = new FlinkInputSplit(index, task, hostnames);
              });
      return splits;
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to process tasks iterable", e);
    }
  }

  /** This returns splits for the FLIP-27 source */
  public static List<IcebergSourceSplit> planIcebergSourceSplits(
      Table table, ScanContext context, ExecutorService workerPool) {
    try (CloseableIterable<CombinedScanTask> tasksIterable =
        planTasks(table, context, workerPool)) {
      return Lists.newArrayList(
          CloseableIterable.transform(tasksIterable, IcebergSourceSplit::fromCombinedScanTask));
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to process task iterable: ", e);
    }
  }

  static CloseableIterable<CombinedScanTask> planTasks(
      Table table, ScanContext context, ExecutorService workerPool) {
    ScanMode scanMode = checkScanMode(context);
    if (scanMode == ScanMode.INCREMENTAL_APPEND_SCAN) {
      IncrementalAppendScan scan = table.newIncrementalAppendScan();
      scan = refineScanWithBaseConfigs(scan, context, workerPool);

      if (context.startTag() != null) {
        Preconditions.checkArgument(
            table.snapshot(context.startTag()) != null,
            "Cannot find snapshot with tag %s",
            context.startTag());
        scan = scan.fromSnapshotExclusive(table.snapshot(context.startTag()).snapshotId());
      }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Retry planning; ensure the enumerator restart path re-plans and that storage failures were transient.
2. Stop concurrent expireSnapshots/removeOrphanFiles maintenance while jobs are planning against the table.
3. Check storage credentials and network stability; refresh tokens (Kerberos ticket, cloud credentials).
4. Read the wrapped IOException cause to identify the failing manifest/file and repair table metadata if corrupted.

Example fix

// before
List<IcebergSourceSplit> splits = FlinkSplitPlanner.planIcebergSourceSplits(table, context, pool);
// after
try {
  List<IcebergSourceSplit> splits = FlinkSplitPlanner.planIcebergSourceSplits(table, context, pool);
} catch (UncheckedIOException e) {
  // inspect e.getCause(); retry or fall back to a re-planned scan
  throw new JobRecoverableException("Recoverable planning failure", e);
}

// precheck: ensure current snapshot metadata files are readable
Snapshot snapshot = table.currentSnapshot();
if (snapshot != null) { table.io().newInputFile(snapshot.manifestListLocation()).getLength(); }

try {
  return FlinkSplitPlanner.planIcebergSourceSplits(table, context, workerPool);
} catch (UncheckedIOException e) {
  // treat as recoverable: enumerator restarts and re-plans
  throw new IOException("Recoverable planning failure", e.getCause());
}

When it happens

Trigger: Calling planIcebergSourceSplits (via IcebergSource.enumerator/ batch splits) when iterating planTasks' CloseableIterable throws IOException — e.g., manifest reads fail via FileIO, or the try-with-resources close of the iterable fails.

Common situations: Object storage timeouts or throttling during parallel manifest scanning; manifests deleted by concurrent table maintenance (expire_snapshots); Kerberos/STS token expiry mid-planning; large table scans hitting storage read limits.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to process tasks iterable
• Failed to process tasks iterable
• Failed to process task iterable:
• Failed to list partitions of table %s
• Failed to read manifest: <manifest.path()>

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/adf43db78a5eed7e. Report an issue: GitHub.