{"record":{"id":"ab616cbee2b7391d","repo":"apache/iceberg","slug":"failed-to-close-scan-scan-ab616c","errorCode":null,"errorMessage":"Failed to close scan: + scan","messagePattern":"Failed to close scan: \\+ scan","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPartitioningAwareScan.java","lineNumber":196,"sourceCode":"\n  protected synchronized List<T> tasks() {\n    if (tasks == null) {\n      try (CloseableIterable<? extends ScanTask> taskIterable = scan.planFiles()) {\n        List<T> plannedTasks = Lists.newArrayList();\n\n        for (ScanTask task : taskIterable) {\n          ValidationException.check(\n              taskJavaClass().isInstance(task),\n              \"Unsupported task type, expected a subtype of %s: %s\",\n              taskJavaClass().getName(),\n              task.getClass().getName());\n\n          plannedTasks.add(taskJavaClass().cast(task));\n        }\n\n        this.tasks = plannedTasks;\n      } catch (IOException e) {\n        throw new UncheckedIOException(\"Failed to close scan: \" + scan, e);\n      }\n    }\n\n    return tasks;\n  }\n\n  @Override\n  protected synchronized List<ScanTaskGroup<T>> taskGroups() {\n    if (taskGroups == null) {\n      if (groupingKeyType().fields().isEmpty()) {\n        CloseableIterable<ScanTaskGroup<T>> plannedTaskGroups =\n            TableScanUtil.planTaskGroups(\n                CloseableIterable.withNoopClose(tasks()),\n                adjustSplitSize(tasks(), scan.targetSplitSize()),\n                scan.splitLookback(),\n                scan.splitOpenFileCost());\n        this.taskGroups = Lists.newArrayList(plannedTaskGroups);\n","sourceCodeStart":178,"sourceCodeEnd":214,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPartitioningAwareScan.java#L178-L214","documentation":"SparkPartitioningAwareScan plans tasks inside a try-with-resources that closes the underlying Iceberg scan (CloseableIterable). If closing the scan throws an IOException after tasks were planned, the scan wraps it in this UncheckedIOException naming the scan. Task planning results are discarded; the Spark query fails even though the read itself may have succeeded.","triggerScenarios":"tasks() planning completes and scan.close() throws IOException — usually a failure in the underlying FileIO closing manifest resources (HDFS stream close failure, S3 connection abort during cleanup).","commonSituations":"Transient network drops to HDFS/S3 during close; Hadoop filesystem cache eviction issues; long-running streaming queries whose scans accumulate file handles and hit failures during teardown.","solutions":["Inspect the cause IOException for the underlying close failure (usually network or filesystem) and address connectivity/retry settings.","Retry the failed Spark task — planning is retriable since nothing was committed.","Tune filesystem client timeouts/retry policies (fs.hdfs.impl, S3 retry configuration) for transient close failures.","Upgrade Iceberg/Hadoop versions if close failures stem from known stream-close bugs."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// ensure cluster connectivity to table storage before query\nFileSystem fs = new Path(table.location()).getFileSystem(conf);\nfs.getFileStatus(new Path(table.location()));","typeGuard":null,"tryCatchPattern":"try { df = spark.read...load(); } catch (UncheckedIOException e) {\n  if (e.getMessage().startsWith(\"Failed to close scan\")) { retrySparkTask(); }\n  else throw e;\n}","preventionTips":["Tune Hadoop/S3 client timeout and retry settings for the cluster","Watch for file-handle exhaustion in long-running streaming queries","Keep Iceberg/Hadoop versions patched for known close() bugs"],"tags":["spark","io","scan-planning","resource-cleanup"],"backgroundTag":"resource-cleanup-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}