{"record":{"id":"eb38f69bf0e25a18","repo":"apache/iceberg","slug":"failed-to-close-manifest-reader-eb38f6","errorCode":null,"errorMessage":"Failed to close manifest reader","messagePattern":"Failed to close manifest reader","errorType":"exception","errorClass":"RuntimeIOException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/ManifestMergeManager.java","lineNumber":218,"sourceCode":"            manifest.snapshotId() != null && snapshotId() != manifest.snapshotId();\n        try (ManifestReader<F> reader = newManifestReader(manifest, isCommitted)) {\n          for (ManifestEntry<F> entry : reader.entries()) {\n            if (entry.status() == Status.DELETED) {\n              // suppress deletes from previous snapshots. only files deleted by this snapshot\n              // should be added to the new manifest\n              if (entry.snapshotId() == snapshotId()) {\n                writer.delete(entry);\n              }\n            } else if (entry.status() == Status.ADDED && entry.snapshotId() == snapshotId()) {\n              // adds from this snapshot are still adds, otherwise they should be existing\n              writer.add(entry);\n            } else {\n              // add all files from the old manifest as existing files\n              writer.existing(entry);\n            }\n          }\n        } catch (IOException e) {\n          throw new RuntimeIOException(e, \"Failed to close manifest reader\");\n        }\n      }\n      threw = false;\n\n    } finally {\n      Exceptions.close(writer, threw);\n    }\n\n    ManifestFile manifest = writer.toManifestFile();\n\n    // cache the merged manifest to reuse when retrying and track replaced manifests\n    mergedManifests.put(bin, manifest);\n    for (ManifestFile m : bin) {\n      // only count manifests from previous snapshots; in-memory manifests are not replaced\n      if (snapshotId() != m.snapshotId()) {\n        replacedManifestsCount.incrementAndGet();\n      }\n    }","sourceCodeStart":200,"sourceCodeEnd":236,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/ManifestMergeManager.java#L200-L236","documentation":"Raised when closing the Avro manifest reader during manifest merge fails with an IOException. ManifestMergeManager.createManifest merges entries from older manifests into a new one and closes each source reader; a close failure is wrapped as RuntimeIOException.","triggerScenarios":"During mergeGroup/createManifest, iterating the source ManifestReader throws IOException on close — typically an underlying stream read/flush error surfaced at close time (truncated file, network drop, object-store error while draining the Avro stream).","commonSituations":"Object store connection reset mid-read during large compaction/rewrite jobs in Spark executors; corrupted or truncated manifest files left by a failed previous write; ephemeral network partitions during long-running merge tasks.","solutions":["Inspect the wrapped IOException for the underlying read/connection error.","Retry the rewrite/merge job — transient network failures are the most common cause.","Validate suspect manifests by re-reading them; replace/copy corrupted files if storage supports it.","Ensure executor network stability / increase object store client timeouts."],"exampleFix":"// before: single-shot rewrite susceptible to transient IO errors\nSparkActions.get().rewriteDataFiles(table).execute();\n// after: enable retry/partial progress in the action\nSparkActions.get().rewriteDataFiles(table)\n    .option(\"max-concurrent-file-group-rewrites\", \"4\")\n    .option(\"partial-progress.enabled\", \"true\")\n    .execute();","handlingStrategy":"retry","validationCode":"// validate manifests readable before merge\nfor (ManifestFile m : manifestsToMerge) {\n  try (ManifestReader<?> r = ManifestFiles.read(m, io)) {\n    r.iterator().hasNext(); // force open\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  mergeOperation.commit();\n} catch (RuntimeIOException e) {\n  if (isTransientNetwork(e.getCause())) retryWithBackoff();\n  else throw new IllegalStateException(\"Corrupt manifest: \" + e.getCause());\n}","preventionTips":["Enable partial progress in rewrite/compaction jobs","Increase object-store client timeouts for large merges","Detect and quarantine truncated manifest files","Run merges with stable network connectivity"],"tags":["io","avro","merge","read"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}