{"record":{"id":"b5bc84f80c4a925e","repo":"apache/iceberg","slug":"failed-to-determine-live-files-in-manifest-ret","errorCode":null,"errorMessage":"Failed to determine live files in manifest {}. Retrying","messagePattern":"Failed to determine live files in manifest (.+?)\\. Retrying","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"core/src/main/java/org/apache/iceberg/ReachableFileCleanup.java","lineNumber":182,"sourceCode":"              }\n            });\n\n    return manifestFiles;\n  }\n\n  private Set<String> findFilesToDelete(\n      Set<ManifestFile> manifestFilesToDelete,\n      Set<ManifestFile> currentManifestFiles,\n      Map<Integer, PartitionSpec> specsById) {\n    Set<String> filesToDelete = ConcurrentHashMap.newKeySet();\n\n    Tasks.foreach(manifestFilesToDelete)\n        .retry(3)\n        .suppressFailureWhenFinished()\n        .executeWith(planExecutorService)\n        .onFailure(\n            (item, exc) ->\n                LOG.warn(\n                    \"Failed to determine live files in manifest {}. Retrying\", item.path(), exc))\n        .run(\n            manifest -> {\n              try (CloseableIterable<String> paths =\n                  ManifestFiles.readPaths(manifest, fileIO, specsById)) {\n                paths.forEach(filesToDelete::add);\n              } catch (IOException e) {\n                throw new RuntimeIOException(e, \"Failed to read manifest file: %s\", manifest);\n              }\n            });\n\n    if (filesToDelete.isEmpty()) {\n      return filesToDelete;\n    }\n\n    try {\n      Tasks.foreach(currentManifestFiles)\n          .retry(3)","sourceCodeStart":164,"sourceCodeEnd":200,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/ReachableFileCleanup.java#L164-L200","documentation":"ReachableFileCleanup.findFilesToDelete reads live file paths from each to-be-deleted manifest using ManifestFiles.readPaths inside Tasks with 3 retries and suppressed failure; the onFailure callback logs this warning per manifest. A manifest that cannot be read means some of its live data files are not identified, and cleanup will conservatively leave related files alone (or, per implementation, proceed with what it did collect).","triggerScenarios":"Running expireSnapshots (dataFilesToDelete -> findFilesToDelete) when a manifest file targeted for deletion cannot be opened/parsed — missing file, corrupt Avro, throttled object store, wrong schema/spec mapping.","commonSituations":"Corrupt or truncated manifests from interrupted writes, external deletion of metadata by lifecycle rules, storage throttling during parallel reads, and spec ID mismatches after schema/partition evolution on copied tables.","solutions":["Inspect the wrapped exception; restore or repair the unreadable manifest file if it should still exist.","Re-run expiration after fixing transient storage issues; the retry loop already attempted 3 times.","Verify no external lifecycle policy deletes Iceberg metadata files.","If manifests are unreadable leftovers from a broken copy, run RewriteTablePathUtil or recreate the table metadata."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"specsById.values().forEach(spec -> { /* ensure manifests reference valid spec ids */ });\nPreconditions.checkArgument(io.newInputFile(manifest.path()).exists(), manifest.path());","typeGuard":null,"tryCatchPattern":"try { expireSnapshots(); } catch (Exception e) { /* re-run after repairing manifest storage access */ }","preventionTips":["Keep compaction jobs from being killed mid-write to avoid corrupt manifests","Retry expiration with backoff on throttled object stores","Verify manifest files exist before expiration"],"tags":["expiration","cleanup","manifest","io"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}