apache/iceberg · warning

Failed to determine manifests for snapshot {}

Error message

Failed to determine manifests for snapshot {}

What it means

ReachableFileCleanup.pruneReferencedManifests plans per-snapshot manifest reachability in parallel with Tasks (3 retries); when a snapshot fails all retries, the onFailure callback logs this warning. The snapshot's manifests stay in the candidate set, so they may be skipped from deletion rather than deleted — a safety-conservative outcome. Caused by an IOException reading the snapshot's manifest list.

Source

Thrown at core/src/main/java/org/apache/iceberg/ReachableFileCleanup.java:122

      LOG.debug("Deleting {} statistics files", expiredStatisticsFilesLocations.size());
      deleteFiles(expiredStatisticsFilesLocations, "statistics files");
    }
  }

  private Set<ManifestFile> pruneReferencedManifests(
      Set<Snapshot> snapshots,
      Set<ManifestFile> deletionCandidates,
      Consumer<ManifestFile> currentManifestCallback) {
    Set<ManifestFile> candidateSet = ConcurrentHashMap.newKeySet();
    candidateSet.addAll(deletionCandidates);
    Tasks.foreach(snapshots)
        .retry(3)
        .stopOnFailure()
        .throwFailureWhenFinished()
        .executeWith(planExecutorService)
        .onFailure(
            (snapshot, exc) ->
                LOG.warn(
                    "Failed to determine manifests for snapshot {}", snapshot.snapshotId(), exc))
        .run(
            snapshot -> {
              try (CloseableIterable<ManifestFile> manifestFiles = readManifests(snapshot)) {
                for (ManifestFile manifestFile : manifestFiles) {
                  candidateSet.remove(manifestFile);
                  if (candidateSet.isEmpty()) {
                    return;
                  }

                  currentManifestCallback.accept(manifestFile.copy());
                }
              } catch (IOException e) {
                throw new RuntimeIOException(
                    e, "Failed to close manifest list: %s", snapshot.manifestListLocation());
              }
            });

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Inspect the wrapped exception for the failing snapshot's manifest-list path and fix storage access (credentials/permissions/network).
2. Restore or repair the missing manifest list file, then re-run expireSnapshots.
3. Re-run expiration after transient object-store throttling subsides (add retry/backoff at the job level).
4. If the table location changed, run RewriteTablePathUtil or recreate metadata so paths resolve.

// pre-check manifest list readability per snapshot
for (Snapshot s : table.snapshots()) {
  Preconditions.checkArgument(table.io().newInputFile(s.manifestListLocation()).exists());
}

try { expireSnapshots(); } catch (RuntimeIOException e) { /* retry after restoring storage access */ }

When it happens

Trigger: Running expireSnapshots (or the ExpireSnapshotsProcedure) while one snapshot's manifest list file cannot be read — deleted from storage, permission denied, throttled by object store, or dangling after a table was copied without path rewriting.

Common situations: Manifest lists removed by overly aggressive external lifecycle rules, concurrent writers/compaction making metadata transiently unreadable, S3 503 throttling under load, and broken path references after renaming a table location.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to close manifest list: %s
• Failed to determine live files in manifest {}. Retrying
• Failed to read manifest file: %s
• An error occurred while aborting the stream
• Bulk deletion failed

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/62402fafc71c1dc6. Report an issue: GitHub.