apache/iceberg · warning

Failed to close task iterable

Error message

Failed to close task iterable

What it means

SparkTable.canDeleteUsingMetadata decides whether a DELETE can be satisfied by metadata-only file rewrites by evaluating the delete expression against file partitions and metrics inside a CloseableIterable. This warning is logged if closing that iterable throws an IOException; the method then returns false, meaning the DELETE falls back to a full row-level (copy-on-write) operation instead of metadata-only deletion.

Solutions

1. Retry the DELETE statement — the fallback path still executes correctly, just slower.
2. Inspect the logged IOException stack trace for the underlying file/IO cause and fix (permissions, connectivity, credentials).
3. If object-store latency is the cause, check FileIO configuration (retries, timeouts, endpoint).
4. Accept the copy-on-write fallback if transient; consider compaction so fewer files are evaluated.

// Verify target files are readable before DELETE
CloseableIterable<FileScanTask> tasks = table.newScan().planFiles();
try (CloseableIterable<FileScanTask> it = tasks) {
  it.forEach(t -> Preconditions.checkNotNull(t.file(), "unreadable task"));
} catch (IOException e) {
  LOG.warn("IO issues present; DELETE will fall back to copy-on-write", e);
}

try {
  table.deleteWhere(predicates);
} catch (IOException | UncheckedIOException e) {
  LOG.warn("Metadata delete degraded to row-level rewrite", e);
  // retry with backoff
}

When it happens

Trigger: Row-level operations (DELETE FROM) on a Spark v2 table where iterating/closing the file-task iterable during canDeleteWhere hits an IOException from the underlying FileIO (network blip to object store, filesystem error) — logged in canDeleteUsingMetadata called by canDeleteWhere.

Common situations: Transient S3/HDFS read failures during DELETE planning; credentials expiring mid-planning; corrupted or inaccessible data files whose metadata cannot be read.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to close task iterable
• Failed to check if can be pushed down
• Failed to check if can be pushed down
• Failed to close task iterable
• Failed to close task iterable

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/566a9b808418a488. Report an issue: GitHub.

Appendix: source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java:402

      StrictMetricsEvaluator metricsEvaluator =
          new StrictMetricsEvaluator(SnapshotUtil.schemaFor(table(), scanBranch), deleteExpr);

      return Iterables.all(
          tasks,
          task -> {
            DataFile file = task.file();
            PartitionSpec spec = task.spec();
            Evaluator evaluator =
                evaluators.computeIfAbsent(
                    spec.specId(),
                    specId ->
                        new Evaluator(
                            spec.partitionType(), Projections.strict(spec).project(deleteExpr)));
            return evaluator.eval(file.partition()) || metricsEvaluator.eval(file);
          });

    } catch (IOException ioe) {
      LOG.warn("Failed to close task iterable", ioe);
      return false;
    }
  }

  @Override
  public void deleteWhere(Predicate[] predicates) {
    Expression deleteExpr = SparkV2Filters.convert(predicates);

    if (deleteExpr == Expressions.alwaysFalse()) {
      LOG.info("Skipping the delete operation as the condition is always false");
      return;
    }

    DeleteFiles deleteFiles =
        icebergTable
            .newDelete()
            .set("spark.app.id", sparkSession().sparkContext().applicationId())
            .deleteFromRowFilter(deleteExpr);

View on GitHub (pinned to 86d9c8fc54)