apache/iceberg · error · RuntimeIOException

Failed to read manifest file: %s

Error message

Failed to read manifest file: %s

What it means

findFilesToDelete reads each candidate manifest file and, if closing/reading the manifest reader raises IOException, wraps it as RuntimeIOException with the manifest path. This failure aborts the file-deletion computation because it cannot be determined which data files are still referenced, and deleting files on an incomplete answer would be unsafe.

Source

Thrown at core/src/main/java/org/apache/iceberg/IncrementalFileCleanup.java:309

        .executeWith(planExecutorService)
        .onFailure(
            (item, exc) ->
                LOG.warn("Failed to get deleted files: this may cause orphaned data files", exc))
        .run(
            manifest -> {
              // the manifest has deletes, scan it to find files to delete
              try (ManifestReader<?> reader = ManifestFiles.open(manifest, fileIO, specsById)) {
                for (ManifestEntry<?> entry : reader.entries()) {
                  // if the snapshot ID of the DELETE entry is no longer valid, the data can be
                  // deleted
                  if (entry.status() == ManifestEntry.Status.DELETED
                      && !validIds.contains(entry.snapshotId())) {
                    // use toString to ensure the path will not change (Utf8 is reused)
                    filesToDelete.add(entry.file().location());
                  }
                }
              } catch (IOException e) {
                throw new RuntimeIOException(e, "Failed to read manifest file: %s", manifest);
              }
            });

    Tasks.foreach(manifestsToRevert)
        .retry(3)
        .suppressFailureWhenFinished()
        .executeWith(planExecutorService)
        .onFailure(
            (item, exc) ->
                LOG.warn("Failed to get added files: this may cause orphaned data files", exc))
        .run(
            manifest -> {
              // the manifest has deletes, scan it to find files to delete
              try (ManifestReader<?> reader = ManifestFiles.open(manifest, fileIO, specsById)) {
                for (ManifestEntry<?> entry : reader.entries()) {
                  // delete any ADDED file from manifests that were reverted
                  if (entry.status() == ManifestEntry.Status.ADDED) {
                    // use toString to ensure the path will not change (Utf8 is reused)

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Restore or re-read the missing/corrupted manifest; check whether the manifest at the path in the message exists.
2. Retry the expiry job if the cause was a transient storage error.
3. Verify FileIO credentials/permissions for the manifest path.
4. Never manually delete manifests; if metadata is corrupted, restore table metadata from a backup or use a rollback to a good metadata version.

Example fix

// before
table.expireSnapshots().olderThan(ts).execute(); // fails: manifest unreadable
// after
ManifestFile missing = findMissingManifest(table);
if (missing != null) {
  restoreFromMetadataBackup(table); // or roll back metadata before expiring
}
table.expireSnapshots().olderThan(ts).execute();

// before expiring, ensure all referenced manifests are readable:
for (ManifestFile m : table.currentSnapshot().allManifests(table.io())) {
  if (!table.io().exists(m.path())) throw new IllegalStateException("missing manifest: " + m.path());
}

try {
  table.expireSnapshots().olderThan(ts).execute();
} catch (RuntimeIOException e) {
  // manifest unreadable: check existence/corruption before retrying
  investigateManifest(e.getMessage());
}

When it happens

Trigger: Running ExpireSnapshots (filesToDelete path) where a manifest file referenced by a remaining snapshot cannot be read or closed - missing/corrupted manifest, storage I/O error, or revoked access.

Common situations: Manifests deleted externally or by a bad earlier job; object-store outages during expiry; permission changes on the table location mid-job.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to read manifest:
• Failed to determine live files in manifest {}. Retrying
• Failed to read manifest file: %s
• Failed to validate replaced partitions
• Failed to write manifest

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/5f0cc817c4bbfb63. Report an issue: GitHub.