{"record":{"id":"c08107eacd7d0749","repo":"apache/iceberg","slug":"failed-to-close-task-iterator","errorCode":null,"errorMessage":"Failed to close task iterator","messagePattern":"Failed to close task iterator","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"core/src/main/java/org/apache/iceberg/actions/BaseRewriteDataFilesAction.java","lineNumber":298,"sourceCode":"            .collect(Collectors.toList());\n    replaceDataFiles(currentDataFiles, addedDataFiles, startingSnapshotId);\n\n    return new RewriteDataFilesActionResult(currentDataFiles, addedDataFiles);\n  }\n\n  private Map<StructLikeWrapper, Collection<FileScanTask>> groupTasksByPartition(\n      CloseableIterator<FileScanTask> tasksIter) {\n    ListMultimap<StructLikeWrapper, FileScanTask> tasksGroupedByPartition =\n        Multimaps.newListMultimap(Maps.newHashMap(), Lists::newArrayList);\n    StructLikeWrapper partitionWrapper = StructLikeWrapper.forType(spec.partitionType());\n    try (CloseableIterator<FileScanTask> iterator = tasksIter) {\n      iterator.forEachRemaining(\n          task -> {\n            StructLikeWrapper structLike = partitionWrapper.copyFor(task.file().partition());\n            tasksGroupedByPartition.put(structLike, task);\n          });\n    } catch (IOException e) {\n      LOG.warn(\"Failed to close task iterator\", e);\n    }\n    return tasksGroupedByPartition.asMap();\n  }\n\n  private void replaceDataFiles(\n      Iterable<DataFile> deletedDataFiles,\n      Iterable<DataFile> addedDataFiles,\n      long startingSnapshotId) {\n    try {\n      doReplace(deletedDataFiles, addedDataFiles, startingSnapshotId);\n    } catch (CommitStateUnknownException e) {\n      LOG.warn(\"Commit state unknown, cannot clean up files that may have been committed\", e);\n      throw e;\n    } catch (Exception e) {\n      if (e instanceof CleanableFailure) {\n        LOG.warn(\"Failed to commit rewrite, cleaning up rewritten files\", e);\n        Tasks.foreach(Iterables.transform(addedDataFiles, ContentFile::location))\n            .noRetry()","sourceCodeStart":280,"sourceCodeEnd":316,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/actions/BaseRewriteDataFilesAction.java#L280-L316","documentation":"While grouping FileScanTasks by partition in BaseRewriteDataFilesAction.groupTasksByPartition(), an IOException raised while iterating (or implicitly closing) the task iterator is caught and logged as this warning. The returned map may then be incomplete, so some file groups could be silently omitted from the rewrite. Callers should treat repeated occurrences as a sign of degraded rewrite coverage.","triggerScenarios":"Iterator exhaustion/close of the FileScanTask CloseableIterator throws IOException during groupTasksByPartition() — e.g. an open manifest/data stream fails mid-iteration.","commonSituations":"Object-store read errors mid-iteration; truncated manifest lists; network drops to storage during planning; Hadoop FS misconfiguration.","solutions":["Inspect the logged IOException to find the failing stream/storage location.","Fix storage access issues (permissions, connectivity) and re-run the rewrite action.","Re-plan the scan (table.refresh() then execute) to get a complete task list.","Compare the grouped partition count against expected partitions to detect silently missing groups."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"int planned = (int) streamOf(table.newScan().planTasks()).count();\nif (groupedTasks.size() < expectedPartitionCount) {\n  throw new IllegalStateException(\"Incomplete task grouping; re-plan the scan before rewriting\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  tasksByPartition = groupTasksByPartition(taskIterable.iterator());\n} catch (RuntimeException e) {\n  LOG.warn(\"grouping possibly incomplete due to IO error; aborting rewrite\", e);\n}","preventionTips":["Validate grouped partition counts against table partition counts before rewriting","Fix storage/manifest read errors surfaced in the logged stack trace","Refresh table metadata before planning scans","Avoid rewriting on unstable network links to storage"],"tags":["io-error","iteration","rewrite","partitioning"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}