apache/iceberg · error · UncheckedIOException

Failed to validate replaced partitions

Error message

Failed to validate replaced partitions

What it means

During cherry-pick validation, the manifests of the snapshot being picked are scanned to ensure no replaced partitions had their partition spec/data changed (a full overwrite must be picked into a matching partition layout). An IOException while reading those manifest files is wrapped in this UncheckedIOException.

Source

Thrown at core/src/main/java/org/apache/iceberg/CherryPickOperation.java:248

        Iterable<CloseableIterable<DataFile>> addedFileTasks =
            Iterables.concat(
                Iterables.transform(
                    snapshots,
                    snap ->
                        Iterables.transform(
                            manifestsCreatedBy(snap, io),
                            manifest -> addedDataFiles(manifest, io, meta.specsById()))));

        try (CloseableIterable<DataFile> newFiles =
            new ParallelIterable<>(addedFileTasks, ThreadPools.getWorkerPool())) {
          for (DataFile newFile : newFiles) {
            ValidationException.check(
                !replacedPartitions.contains(newFile.specId(), newFile.partition()),
                "Cannot cherry-pick replace partitions with changed partition: %s",
                newFile.partition());
          }
        } catch (IOException e) {
          throw new UncheckedIOException("Failed to validate replaced partitions", e);
        }
      }
    }
  }

  private static Iterable<ManifestFile> manifestsCreatedBy(Snapshot snapshot, FileIO io) {
    return Iterables.filter(
        snapshot.dataManifests(io), m -> Objects.equals(m.snapshotId(), snapshot.snapshotId()));
  }

  private static CloseableIterable<DataFile> addedDataFiles(
      ManifestFile manifest, FileIO io, Map<Integer, PartitionSpec> specsById) {
    CloseableIterable<ManifestEntry<DataFile>> entries =
        ManifestFiles.read(manifest, io, specsById).entries();
    CloseableIterable<ManifestEntry<DataFile>> added =
        CloseableIterable.filter(entries, e -> e.status() == ManifestEntry.Status.ADDED);
    return CloseableIterable.transform(added, e -> e.file().copy());
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check the cause IOException for the real read failure (file-not-found, auth, checksum).
2. Verify manifests exist and are readable with the configured FileIO and credentials.
3. Ensure snapshot expiry has not deleted manifests still referenced by the snapshot being picked.
4. Retry after fixing storage access; the operation is read-only until validation passes.

Example fix

// inspect root cause
try {
  table.cherryPick(snapshotId);
} catch (UncheckedIOException e) {
  LOG.error("manifest read failed", e.getCause());
  throw e;
}

// pre-check manifest readability
Snapshot snap = table.snapshot(snapshotId);
snap.allManifests(table.io()).forEach(m ->
    Preconditions.check(table.io().newInputFile(m.path()).exists(), "missing manifest " + m.path()));

try { table.cherryPick(id); } catch (UncheckedIOException e) {
  LOG.error("manifest validation failed", e.getCause());
  throw e;
}

When it happens

Trigger: table.cherryPick(snapshotId) where the snapshot contains overwritten partitions and the manifest files listed in the snapshot cannot be read (missing, corrupt, or unreadable via the configured FileIO).

Common situations: Deleted or expired manifests referenced by a retained snapshot; wrong FileIO/credentials so manifest reads fail; corrupted object storage files after partial writes.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to read manifest file: %s
• Failed to write manifest
• Failed to read manifest file: %s
• Failed to read manifest file: %s
• Failed to close manifest reader

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/33ac2c84cc30559e. Report an issue: GitHub.