apache/iceberg · warning

Failed to close task iterable

Error message

Failed to close task iterable

What it means

A WARN log in SparkTable.canDeleteUsingMetadata when closing the CloseableIterable of task files throws an IOException. The method then returns false, causing the delete to fall back to a copy-on-write rewrite instead of metadata-only deletion.

Solutions

1. Retry the DELETE statement; the IOException is often transient
2. Check storage connectivity and FileIO configuration
3. Accept the fallback: the delete will proceed via copy-on-write rewrite
4. Inspect the logged stack trace for the specific storage error

// check storage reachability before large DELETEs
fileIO.newInputFile(manifestPath).exists();

When it happens

Trigger: Evaluating whether a DELETE can be satisfied by metadata (deleting whole files/partitions) requires opening task iterables; closing them fails with an IOException from the underlying FileIO (e.g. network error on the manifest access).

Common situations: Transient object-store connectivity problems during DELETE statements; flaky HDFS access in the Spark executor/driver.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to close task iterable
• Failed to close task iterable
• Delete failed for
• Failed reading offset from
• Failed reading offset from

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/5608519c2b971425. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java:258

    try (CloseableIterable<FileScanTask> tasks = scan.planFiles()) {
      Map<Integer, Evaluator> evaluators = Maps.newHashMap();
      StrictMetricsEvaluator metricsEvaluator = new StrictMetricsEvaluator(schema, deleteExpr);
      return Iterables.all(
          tasks,
          task -> {
            DataFile file = task.file();
            PartitionSpec spec = task.spec();
            Evaluator evaluator =
                evaluators.computeIfAbsent(
                    spec.specId(),
                    specId ->
                        new Evaluator(
                            spec.partitionType(), Projections.strict(spec).project(deleteExpr)));
            return evaluator.eval(file.partition()) || metricsEvaluator.eval(file);
          });

    } catch (IOException ioe) {
      LOG.warn("Failed to close task iterable", ioe);
      return false;
    }
  }

  @Override
  public void deleteWhere(Predicate[] predicates) {
    Expression deleteExpr = SparkV2Filters.convert(predicates);

    if (deleteExpr == Expressions.alwaysFalse()) {
      LOG.info("Skipping the delete operation as the condition is always false");
      return;
    }

    DeleteFiles deleteFiles =
        table()
            .newDelete()
            .set("spark.app.id", spark().sparkContext().applicationId())
            .deleteFromRowFilter(deleteExpr);

View on GitHub (pinned to 86d9c8fc54)