apache/iceberg · error · ValidationException

Deleted manifest %s could not be found in the latest snapsho

Error message

Deleted manifest %s could not be found in the latest snapshot %d

What it means

RewriteManifests may delete manifests directly (when all entries are rewritten) and expects those deleted manifests to still be present in the snapshot that apply() is based on. validateDeletedManifests throws this ValidationException if any manifest slated for deletion is no longer in the latest snapshot, meaning the table changed concurrently and the rewrite is stale.

Source

Thrown at core/src/main/java/org/apache/iceberg/BaseRewriteManifests.java:294

  }

  private boolean containsDeletes(ManifestFile manifest) {
    return manifest.content() == ManifestContent.DELETES;
  }

  private boolean matchesPredicate(ManifestFile manifest) {
    return predicate == null || predicate.test(manifest);
  }

  private void validateDeletedManifests(
      Set<ManifestFile> currentManifests, long currentSnapshotID) {
    // directly deleted manifests must be still present in the current snapshot
    deletedManifests.stream()
        .filter(manifest -> !currentManifests.contains(manifest))
        .findAny()
        .ifPresent(
            manifest -> {
              throw new ValidationException(
                  "Deleted manifest %s could not be found in the latest snapshot %d",
                  manifest.path(), currentSnapshotID);
            });
  }

  private void validateFilesCounts() {
    Iterable<ManifestFile> createdManifests =
        Iterables.concat(newManifests, addedManifests, rewrittenAddedManifests);
    int createdManifestsFilesCount = activeFilesCount(createdManifests);

    Iterable<ManifestFile> replacedManifests =
        Iterables.concat(rewrittenManifests, deletedManifests);
    int replacedManifestsFilesCount = activeFilesCount(replacedManifests);

    if (createdManifestsFilesCount != replacedManifestsFilesCount) {
      throw new ValidationException(
          "Replaced and created manifests must have the same number of active files: %d (new), %d (old)",
          createdManifestsFilesCount, replacedManifestsFilesCount);

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Refresh the table and re-run rewriteManifests() from the latest snapshot.
2. Wrap the commit in a retry that rebuilds the operation after CommitFailedException/ValidationException.
3. Pause concurrent snapshot expiration during manifest rewrites, or sequence maintenance jobs.
4. Keep rewrite jobs short-lived to narrow the race window.

Example fix

// before
table.rewriteManifests().clusterBy(f -> f.partition()).commit(); // may throw once
// after
Tasks.foreach(new Object[] {null})
    .retry(3)
    .run(ignored ->
        table.refresh().rewriteManifests().clusterBy(f -> f.partition()).commit());

// before commit, verify planned manifests still exist in the latest snapshot
Snapshot current = table.currentSnapshot();
Set<String> present = current.allManifests(table.io()).stream()
    .map(m -> m.path().toString()).collect(Collectors.toSet());
boolean stale = plannedDeleted.stream().anyMatch(m -> !present.contains(m.path().toString()));
if (stale) { table.refresh(); /* rebuild the rewrite */ }

try {
  table.rewriteManifests().clusterBy(f -> f.partition()).commit();
} catch (ValidationException e) {
  if (e.getMessage().contains("could not be found in the latest snapshot")) {
    table.refresh(); // rebuild rewrite from latest snapshot and retry
  } else throw e;
}

When it happens

Trigger: Calling apply()/commit() on a RewritePartitions-style rewrite whose base snapshot is no longer current: another writer expired snapshots or committed new manifests between when the rewrite was planned and committed.

Common situations: A long-running rewriteManifests job racing with concurrent ingestion (new snapshots appended); snapshots expired by a maintenance job while the rewrite was in flight.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Replaced and created manifests must have the same number of
• Cannot cherrypick snapshot %s: already an ancestor
• Found conflicting files that can contain records matching pa
• Invalid distribution mode: %s
• this.getClass().getName() + " doesn't implement removedDelet

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/2dbf389ce3d96dc4. Report an issue: GitHub.