apache/iceberg · error · UncheckedIOException

Failed to process task iterable:

Error message

Failed to process task iterable: 

What it means

FlinkSplitPlanner.planIcebergSourceSplits converts the CloseableIterable of CombinedScanTasks into IcebergSourceSplits for the FLIP-27 source; IOException while consuming/closing the tasks iterable is wrapped in UncheckedIOException. It signals failure reading table metadata/manifests during scan planning.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/source/FlinkSplitPlanner.java:80

                  hostnames = Util.blockLocations(table.io(), task);
                }
                splits[index] = new FlinkInputSplit(index, task, hostnames);
              });
      return splits;
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to process tasks iterable", e);
    }
  }

  /** This returns splits for the FLIP-27 source */
  public static List<IcebergSourceSplit> planIcebergSourceSplits(
      Table table, ScanContext context, ExecutorService workerPool) {
    try (CloseableIterable<CombinedScanTask> tasksIterable =
        planTasks(table, context, workerPool)) {
      return Lists.newArrayList(
          CloseableIterable.transform(tasksIterable, IcebergSourceSplit::fromCombinedScanTask));
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to process task iterable: ", e);
    }
  }

  static CloseableIterable<CombinedScanTask> planTasks(
      Table table, ScanContext context, ExecutorService workerPool) {
    ScanMode scanMode = checkScanMode(context);
    if (scanMode == ScanMode.INCREMENTAL_APPEND_SCAN) {
      IncrementalAppendScan scan = table.newIncrementalAppendScan();
      scan = refineScanWithBaseConfigs(scan, context, workerPool);

      if (context.startTag() != null) {
        Preconditions.checkArgument(
            table.snapshot(context.startTag()) != null,
            "Cannot find snapshot with tag %s",
            context.startTag());
        scan = scan.fromSnapshotExclusive(table.snapshot(context.startTag()).snapshotId());
      }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Inspect the wrapped IOException cause for the actual storage error
2. Verify FileIO credentials and network access to the table location
3. Pin a snapshot-id to avoid expiry races, or pause expiring/compacting jobs during planning
4. Retry source startup; reduce workerPool threads if the storage is throttling

Example fix

// before
List<IcebergSourceSplit> splits = FlinkSplitPlanner.planIcebergSourceSplits(table, context, pool);
// after: refresh and retry on transient IO failure
table.refresh();
List<IcebergSourceSplit> splits;
try {
  splits = FlinkSplitPlanner.planIcebergSourceSplits(table, context, pool);
} catch (UncheckedIOException e) {
  splits = FlinkSplitPlanner.planIcebergSourceSplits(table, context, pool); // retry once
}

table.refresh();
ScanContext ctx = context.copyWithSnapshotId(table.currentSnapshot().snapshotId());

try {
  return FlinkSplitPlanner.planIcebergSourceSplits(table, context, workerPool);
} catch (UncheckedIOException e) {
  LOG.warn("Split planning IO failure, retrying", e);
  return FlinkSplitPlanner.planIcebergSourceSplits(table, context, workerPool);
}

When it happens

Trigger: Iterating planTasks results when FileIO fails to read manifests/metadata, snapshots expired concurrently, or storage is unreachable while closing the CloseableIterable.

Common situations: S3 credentials misconfiguration, HDFS outage during job startup, snapshot expiry racing with an IcebergSource job's initial planning, network partitions to object storage.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to process tasks iterable
• Failed to process tasks iterable
• Failed to process task iterable:
• Failed to list partitions of table %s
• Failed to read manifest: <manifest.path()>

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/9fbded614e0872e9. Report an issue: GitHub.