{"record":{"id":"566a9b808418a488","repo":"apache/iceberg","slug":"failed-to-close-task-iterable-566a9b","errorCode":null,"errorMessage":"Failed to close task iterable","messagePattern":"Failed to close task iterable","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java","lineNumber":402,"sourceCode":"      StrictMetricsEvaluator metricsEvaluator =\n          new StrictMetricsEvaluator(SnapshotUtil.schemaFor(table(), scanBranch), deleteExpr);\n\n      return Iterables.all(\n          tasks,\n          task -> {\n            DataFile file = task.file();\n            PartitionSpec spec = task.spec();\n            Evaluator evaluator =\n                evaluators.computeIfAbsent(\n                    spec.specId(),\n                    specId ->\n                        new Evaluator(\n                            spec.partitionType(), Projections.strict(spec).project(deleteExpr)));\n            return evaluator.eval(file.partition()) || metricsEvaluator.eval(file);\n          });\n\n    } catch (IOException ioe) {\n      LOG.warn(\"Failed to close task iterable\", ioe);\n      return false;\n    }\n  }\n\n  @Override\n  public void deleteWhere(Predicate[] predicates) {\n    Expression deleteExpr = SparkV2Filters.convert(predicates);\n\n    if (deleteExpr == Expressions.alwaysFalse()) {\n      LOG.info(\"Skipping the delete operation as the condition is always false\");\n      return;\n    }\n\n    DeleteFiles deleteFiles =\n        icebergTable\n            .newDelete()\n            .set(\"spark.app.id\", sparkSession().sparkContext().applicationId())\n            .deleteFromRowFilter(deleteExpr);","sourceCodeStart":384,"sourceCodeEnd":420,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java#L384-L420","documentation":"SparkTable.canDeleteUsingMetadata decides whether a DELETE can be satisfied by metadata-only file rewrites by evaluating the delete expression against file partitions and metrics inside a CloseableIterable. This warning is logged if closing that iterable throws an IOException; the method then returns false, meaning the DELETE falls back to a full row-level (copy-on-write) operation instead of metadata-only deletion.","triggerScenarios":"Row-level operations (DELETE FROM) on a Spark v2 table where iterating/closing the file-task iterable during canDeleteWhere hits an IOException from the underlying FileIO (network blip to object store, filesystem error) — logged in canDeleteUsingMetadata called by canDeleteWhere.","commonSituations":"Transient S3/HDFS read failures during DELETE planning; credentials expiring mid-planning; corrupted or inaccessible data files whose metadata cannot be read.","solutions":["Retry the DELETE statement — the fallback path still executes correctly, just slower.","Inspect the logged IOException stack trace for the underlying file/IO cause and fix (permissions, connectivity, credentials).","If object-store latency is the cause, check FileIO configuration (retries, timeouts, endpoint).","Accept the copy-on-write fallback if transient; consider compaction so fewer files are evaluated."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Verify target files are readable before DELETE\nCloseableIterable<FileScanTask> tasks = table.newScan().planFiles();\ntry (CloseableIterable<FileScanTask> it = tasks) {\n  it.forEach(t -> Preconditions.checkNotNull(t.file(), \"unreadable task\"));\n} catch (IOException e) {\n  LOG.warn(\"IO issues present; DELETE will fall back to copy-on-write\", e);\n}","typeGuard":null,"tryCatchPattern":"try {\n  table.deleteWhere(predicates);\n} catch (IOException | UncheckedIOException e) {\n  LOG.warn(\"Metadata delete degraded to row-level rewrite\", e);\n  // retry with backoff\n}","preventionTips":["Ensure stable connectivity/credentials for the FileIO backend","Retry transient IO failures with backoff","Keep files compacted so metadata-only deletes touch fewer files","Monitor storage error rates"],"tags":["spark","ioexception","delete","fallback"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}