{"record":{"id":"5608519c2b971425","repo":"apache/iceberg","slug":"failed-to-close-task-iterable-560851","errorCode":null,"errorMessage":"Failed to close task iterable","messagePattern":"Failed to close task iterable","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java","lineNumber":258,"sourceCode":"    try (CloseableIterable<FileScanTask> tasks = scan.planFiles()) {\n      Map<Integer, Evaluator> evaluators = Maps.newHashMap();\n      StrictMetricsEvaluator metricsEvaluator = new StrictMetricsEvaluator(schema, deleteExpr);\n      return Iterables.all(\n          tasks,\n          task -> {\n            DataFile file = task.file();\n            PartitionSpec spec = task.spec();\n            Evaluator evaluator =\n                evaluators.computeIfAbsent(\n                    spec.specId(),\n                    specId ->\n                        new Evaluator(\n                            spec.partitionType(), Projections.strict(spec).project(deleteExpr)));\n            return evaluator.eval(file.partition()) || metricsEvaluator.eval(file);\n          });\n\n    } catch (IOException ioe) {\n      LOG.warn(\"Failed to close task iterable\", ioe);\n      return false;\n    }\n  }\n\n  @Override\n  public void deleteWhere(Predicate[] predicates) {\n    Expression deleteExpr = SparkV2Filters.convert(predicates);\n\n    if (deleteExpr == Expressions.alwaysFalse()) {\n      LOG.info(\"Skipping the delete operation as the condition is always false\");\n      return;\n    }\n\n    DeleteFiles deleteFiles =\n        table()\n            .newDelete()\n            .set(\"spark.app.id\", spark().sparkContext().applicationId())\n            .deleteFromRowFilter(deleteExpr);","sourceCodeStart":240,"sourceCodeEnd":276,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java#L240-L276","documentation":"A WARN log in SparkTable.canDeleteUsingMetadata when closing the CloseableIterable of task files throws an IOException. The method then returns false, causing the delete to fall back to a copy-on-write rewrite instead of metadata-only deletion.","triggerScenarios":"Evaluating whether a DELETE can be satisfied by metadata (deleting whole files/partitions) requires opening task iterables; closing them fails with an IOException from the underlying FileIO (e.g. network error on the manifest access).","commonSituations":"Transient object-store connectivity problems during DELETE statements; flaky HDFS access in the Spark executor/driver.","solutions":["Retry the DELETE statement; the IOException is often transient","Check storage connectivity and FileIO configuration","Accept the fallback: the delete will proceed via copy-on-write rewrite","Inspect the logged stack trace for the specific storage error"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// check storage reachability before large DELETEs\nfileIO.newInputFile(manifestPath).exists();","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Ensure stable storage connectivity from driver/executors","Retry DELETE statements on transient IO warnings","Understand this warning only degrades to copy-on-write, not correctness"],"tags":["spark","io","delete","fallback"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}