apache/iceberg · error · UncheckedIOException

Failed to process tasks iterable

Error message

Failed to process tasks iterable

What it means

FlinkSplitPlanner.planInputSplits plans batch input splits for the legacy FlinkSource by iterating a CloseableIterable of CombinedScanTask. Any IOException raised while traversing the task iterable (e.g., reading manifests from FileIO) is wrapped in an UncheckedIOException with this message. It signals that split planning failed at the I/O layer, not that the query is invalid.

Source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/source/FlinkSplitPlanner.java:68

      List<CombinedScanTask> tasks = Lists.newArrayList(tasksIterable);
      FlinkInputSplit[] splits = new FlinkInputSplit[tasks.size()];
      boolean exposeLocality = context.exposeLocality();

      Tasks.range(tasks.size())
          .stopOnFailure()
          .executeWith(exposeLocality ? workerPool : null)
          .run(
              index -> {
                CombinedScanTask task = tasks.get(index);
                String[] hostnames = null;
                if (exposeLocality) {
                  hostnames = Util.blockLocations(table.io(), task);
                }
                splits[index] = new FlinkInputSplit(index, task, hostnames);
              });
      return splits;
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to process tasks iterable", e);
    }
  }

  /** This returns splits for the FLIP-27 source */
  public static List<IcebergSourceSplit> planIcebergSourceSplits(
      Table table, ScanContext context, ExecutorService workerPool) {
    try (CloseableIterable<CombinedScanTask> tasksIterable =
        planTasks(table, context, workerPool)) {
      return Lists.newArrayList(
          CloseableIterable.transform(tasksIterable, IcebergSourceSplit::fromCombinedScanTask));
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to process task iterable: ", e);
    }
  }

  static CloseableIterable<CombinedScanTask> planTasks(
      Table table, ScanContext context, ExecutorService workerPool) {
    ScanMode scanMode = checkScanMode(context);

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Verify table storage is reachable and credentials are valid (retry the job; check S3/HDFS connectivity and tokens).
2. Check for corrupted manifests: run the Iceberg 'remove_orphan_files'/'rewrite manifests' maintenance or inspect metadata JSON to confirm files exist.
3. Increase planning retry/timeout settings for the storage layer and re-submit the Flink job.
4. Inspect the wrapped IOException cause to pinpoint which file/operation failed and fix that specific access problem.

Example fix

// before
FlinkInputSplit[] splits = FlinkSplitPlanner.planInputSplits(table, context); // throws UncheckedIOException
// after
try {
  FlinkInputSplit[] splits = FlinkSplitPlanner.planInputSplits(table, context);
} catch (UncheckedIOException e) {
  LOG.error("Split planning failed due to IO problem; retrying after storage check", e);
  // verify storage access / table health before retry
}

// verify table metadata is readable before planning
TableMetadata metadata = TableMetadataParser.read(table.io(), table.location() + "/metadata/version-hint.text");
Preconditions.checkNotNull(metadata, "table metadata unreadable");

try {
  FlinkInputSplit[] splits = FlinkSplitPlanner.planInputSplits(table, context);
} catch (UncheckedIOException e) {
  // e.getCause() is the IOException; decide retry vs fail based on storage health
  throw new RuntimeException("Split planning IO failure; check storage", e.getCause());
}

When it happens

Trigger: Calling FlinkSplitPlanner.planInputSplits (directly or via FlinkSource) when the underlying table metadata or manifest files cannot be read — e.g., FileIO throwing IOException while listing/reading manifests or computing block locations (Util.blockLocations) during task iteration.

Common situations: Corrupted or deleted manifest files on HDFS/S3; transient network/credential failures reaching object storage; HDFS NameNode unavailability during batch split planning; permission errors reading table metadata.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to process task iterable:
• Failed to process tasks iterable
• Failed to process task iterable:
• Failed to list partitions of table %s
• Failed to read manifest: <manifest.path()>

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/ec7bff9279fd4ce6. Report an issue: GitHub.