apache/iceberg · error · UncheckedIOException

Failed to close scan: + scan

Error message

Failed to close scan: + scan

What it means

SparkPartitioningAwareScan plans tasks inside a try-with-resources that closes the underlying Iceberg scan (CloseableIterable). If closing the scan throws an IOException after tasks were planned, the scan wraps it in this UncheckedIOException naming the scan. Task planning results are discarded; the Spark query fails even though the read itself may have succeeded.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPartitioningAwareScan.java:196

  protected synchronized List<T> tasks() {
    if (tasks == null) {
      try (CloseableIterable<? extends ScanTask> taskIterable = scan.planFiles()) {
        List<T> plannedTasks = Lists.newArrayList();

        for (ScanTask task : taskIterable) {
          ValidationException.check(
              taskJavaClass().isInstance(task),
              "Unsupported task type, expected a subtype of %s: %s",
              taskJavaClass().getName(),
              task.getClass().getName());

          plannedTasks.add(taskJavaClass().cast(task));
        }

        this.tasks = plannedTasks;
      } catch (IOException e) {
        throw new UncheckedIOException("Failed to close scan: " + scan, e);
      }
    }

    return tasks;
  }

  @Override
  protected synchronized List<ScanTaskGroup<T>> taskGroups() {
    if (taskGroups == null) {
      if (groupingKeyType().fields().isEmpty()) {
        CloseableIterable<ScanTaskGroup<T>> plannedTaskGroups =
            TableScanUtil.planTaskGroups(
                CloseableIterable.withNoopClose(tasks()),
                adjustSplitSize(tasks(), scan.targetSplitSize()),
                scan.splitLookback(),
                scan.splitOpenFileCost());
        this.taskGroups = Lists.newArrayList(plannedTaskGroups);

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Inspect the cause IOException for the underlying close failure (usually network or filesystem) and address connectivity/retry settings.
2. Retry the failed Spark task — planning is retriable since nothing was committed.
3. Tune filesystem client timeouts/retry policies (fs.hdfs.impl, S3 retry configuration) for transient close failures.
4. Upgrade Iceberg/Hadoop versions if close failures stem from known stream-close bugs.

// ensure cluster connectivity to table storage before query
FileSystem fs = new Path(table.location()).getFileSystem(conf);
fs.getFileStatus(new Path(table.location()));

try { df = spark.read...load(); } catch (UncheckedIOException e) {
  if (e.getMessage().startsWith("Failed to close scan")) { retrySparkTask(); }
  else throw e;
}

When it happens

Trigger: tasks() planning completes and scan.close() throws IOException — usually a failure in the underlying FileIO closing manifest resources (HDFS stream close failure, S3 connection abort during cleanup).

Common situations: Transient network drops to HDFS/S3 during close; Hadoop filesystem cache eviction issues; long-running streaming queries whose scans accumulate file handles and hit failures during teardown.

Related errors

• Failed to close scan: ${scan}
• Failed to close changelog scan: ${scan}
• Failed to close scan: ${scan}
• Failed to close changelog scan: + scan
• Failed to close task iterable

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/ab616cbee2b7391d. Report an issue: GitHub.