{"record":{"id":"0105b2c91f68bb4f","repo":"apache/iceberg","slug":"failed-to-get-added-files-this-may-cause-orphaned","errorCode":null,"errorMessage":"Failed to get added files: this may cause orphaned data files","messagePattern":"Failed to get added files: this may cause orphaned data files","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"core/src/main/java/org/apache/iceberg/IncrementalFileCleanup.java","lineNumber":319,"sourceCode":"                  // deleted\n                  if (entry.status() == ManifestEntry.Status.DELETED\n                      && !validIds.contains(entry.snapshotId())) {\n                    // use toString to ensure the path will not change (Utf8 is reused)\n                    filesToDelete.add(entry.file().location());\n                  }\n                }\n              } catch (IOException e) {\n                throw new RuntimeIOException(e, \"Failed to read manifest file: %s\", manifest);\n              }\n            });\n\n    Tasks.foreach(manifestsToRevert)\n        .retry(3)\n        .suppressFailureWhenFinished()\n        .executeWith(planExecutorService)\n        .onFailure(\n            (item, exc) ->\n                LOG.warn(\"Failed to get added files: this may cause orphaned data files\", exc))\n        .run(\n            manifest -> {\n              // the manifest has deletes, scan it to find files to delete\n              try (ManifestReader<?> reader = ManifestFiles.open(manifest, fileIO, specsById)) {\n                for (ManifestEntry<?> entry : reader.entries()) {\n                  // delete any ADDED file from manifests that were reverted\n                  if (entry.status() == ManifestEntry.Status.ADDED) {\n                    // use toString to ensure the path will not change (Utf8 is reused)\n                    filesToDelete.add(entry.file().location());\n                  }\n                }\n              } catch (IOException e) {\n                throw new RuntimeIOException(e, \"Failed to read manifest file: %s\", manifest);\n              }\n            });\n\n    return filesToDelete;\n  }","sourceCodeStart":301,"sourceCodeEnd":337,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/IncrementalFileCleanup.java#L301-L337","documentation":"In findFilesToDelete, manifests that must be reverted (files added after the 'before' state) are scanned so their ADDED files can be deleted. If such a scan fails after retries, this WARN 'Failed to get added files: this may cause orphaned data files' is logged and the run continues. Files added by reverted snapshots may then never be deleted — orphaned data files.","triggerScenarios":"filesToDelete -> findFilesToDelete processes manifestsToRevert; ManifestFiles.open or entry iteration throws for a manifest (IO error, missing object, corruption) so the ADDED files cannot be enumerated.","commonSituations":"Overlapping/concurrent expiration deleting manifests mid-run; eventual consistency showing stale listings; object-store errors under parallel planExecutorService load.","solutions":["Inspect the logged exception for the manifest path and fix access/corruption issues.","Rerun expiration (cleanup is idempotent and incremental) after access issues are resolved.","Avoid concurrent table expiration/maintenance to prevent manifest races.","Use full cleanup or remove-orphan-files to reclaim files skipped by this run."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// ensure all manifests to be reverted exist and are openable\nmanifestsToRevert.forEach(m -> Preconditions.checkArgument(io.newInputFile(m.path()).exists()));","typeGuard":null,"tryCatchPattern":"try {\n  table.expireSnapshots().execute();\n} catch (RuntimeException e) {\n  LOG.warn(\"Revert-scan failures may orphan added files; run orphan cleanup\", e);\n}","preventionTips":["Run a single expiration job per table at a time.","Use consistent storage to prevent stale manifest listings.","After failures, run remove-orphan-files with an appropriate olderThan window.","Fix underlying IO errors surfaced in the WARN stack trace before rerunning."],"tags":["cleanup","manifests","io","orphaned-files"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}