apache/iceberg · warning

Failed on snapshot {} while reading manifest list: {}

Error message

Failed on snapshot {} while reading manifest list: {}

What it means

IncrementalFileCleanup.cleanFiles plans deletion by reading each snapshot's manifest list in parallel with Tasks (retry 3, failures suppressed). If reading a snapshot's manifest list fails, this WARN names the snapshotId and manifestListLocation and continues. Files referenced by that snapshot may be missed by this cleanup run, potentially leaving them to be handled by a full (non-incremental) cleanup.

Source

Thrown at core/src/main/java/org/apache/iceberg/IncrementalFileCleanup.java:124

        // protect any snapshot that was cherry-picked into the current table state
        pickedAncestorSnapshotIds.add(Long.parseLong(sourceSnapshotId));
      }
    }

    // find manifests to clean up that are still referenced by a valid snapshot, but written by an
    // expired snapshot
    Set<String> validManifests = ConcurrentHashMap.newKeySet();
    Set<ManifestFile> manifestsToScan = ConcurrentHashMap.newKeySet();

    // Reads and deletes are done using Tasks.foreach(...).suppressFailureWhenFinished to complete
    // as much of the delete work as possible and avoid orphaned data or manifest files.
    Tasks.foreach(snapshots)
        .retry(3)
        .suppressFailureWhenFinished()
        .executeWith(planExecutorService)
        .onFailure(
            (snapshot, exc) ->
                LOG.warn(
                    "Failed on snapshot {} while reading manifest list: {}",
                    snapshot.snapshotId(),
                    snapshot.manifestListLocation(),
                    exc))
        .run(
            snapshot -> {
              try (CloseableIterable<ManifestFile> manifests = readManifests(snapshot)) {
                for (ManifestFile manifest : manifests) {
                  validManifests.add(manifest.path());

                  long snapshotId = manifest.snapshotId();
                  // whether the manifest was created by a valid snapshot (true) or an expired
                  // snapshot (false)
                  boolean fromValidSnapshots = validIds.contains(snapshotId);
                  // whether the snapshot that created the manifest was an ancestor of the table
                  // state
                  boolean isFromAncestor = ancestorIds.contains(snapshotId);
                  // whether the changes in this snapshot have been picked into the current table

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check the WARN's cause for the manifest list location — verify the object exists and is readable with the configured FileIO credentials.
2. Avoid running concurrent expirations on the same table; a concurrent run may delete the manifest lists being read.
3. Re-run expireSnapshots; incremental cleanup is resumable and suppressed failures only skip the affected snapshots.
4. If failures persist, perform a full cleanup (rewrite/remove_orphan mode) which does not rely on those manifest lists.

// sanity-check manifest list readability before expiration
for (Snapshot s : table.snapshots()) {
  if (s.manifestListLocation() != null) { io.newInputFile(s.manifestListLocation()); }
}

try {
  table.expireSnapshots().cleanExpiredMetadata(true).execute();
} catch (RuntimeException e) {
  LOG.warn("Incremental cleanup skipped some snapshots; re-run later", e);
}

When it happens

Trigger: expireSnapshots with incremental cleanup enabled runs against a table whose manifest list files are unreadable — deleted/corrupted objects, missing permissions, or FileIO IO errors — for snapshots being planned for file deletion.

Common situations: Manifest lists already removed by an earlier overlapping expiration (eventual consistency/stale metadata); network partition to the object store; corrupted or truncated manifest-list objects.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to get deleted files: this may cause orphaned data fi
• Failed to get added files: this may cause orphaned data file
• Failed to close manifest list: %s
• Failed to read manifest file: %s
• An error occurred while aborting the stream

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/c01e7a5e2b43cb4d. Report an issue: GitHub.