apache/iceberg · warning

Failed to read manifest list

Error message

Failed to read manifest list {}

What it means

RewriteTablePathUtil.manifestFilesInSnapshot calls snapshot.allManifests(io) to enumerate a snapshot's manifests while rewriting path references; on RuntimeIOException it logs this warning and returns an empty list. Because the manifest list cannot be read, its manifests are skipped during the rewrite, so the rewritten table may keep dangling path references for that snapshot.

Solutions

1. Restore access to the source manifest list (permissions/credentials/network) and re-run the rewrite.
2. Verify all snapshots' manifest lists exist before rewriting: check each snapshot.manifestListLocation() is readable.
3. Re-run RewriteTablePathUtil for the affected snapshots after the source is available.
4. Copy the complete metadata tree before modifying/deleting the source location.

Example fix

// before: rewrite proceeds, skipping unreadable snapshots
RewriteTablePathUtil.rewriteTablePath(...);
// after: pre-flight check of every manifest list
for (Snapshot s : table.snapshots()) {
  Preconditions.checkState(table.io().newInputFile(s.manifestListLocation()).exists(),
      "Missing manifest list: " + s.manifestListLocation());
}
RewriteTablePathUtil.rewriteTablePath(...);

for (Snapshot s : table.snapshots()) {
  Preconditions.checkState(table.io().newInputFile(s.manifestListLocation()).exists(),
      "Unreadable manifest list: " + s.manifestListLocation());
}

try { RewriteTablePathUtil.rewriteTablePath(...); } catch (RuntimeIOException e) { /* restore source access and re-run */ }

When it happens

Trigger: Running RewriteTablePathUtil (table rename/relocation/copy between storage systems) when a snapshot's manifest list file cannot be read at its original location — file deleted, permission denied, or source bucket unreachable.

Common situations: Migrating tables after the source was partially cleaned up (expired metadata), missing read permissions on the source bucket, and interrupted migrations where some metadata files were already moved.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to close manifest list
• Failed to determine manifests for snapshot
• An error occurred while aborting the stream
• An error occurred while closing the stream
• Backup table name cannot be specified

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/6823f8671ce253ee. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/RewriteTablePathUtil.java:354

        if (rewrittenManifestLengths.containsKey(file.path())) {
          result.toRewrite().add(file);
          result
              .copyPlan()
              .add(Pair.of(stagingPath(file.path(), sourcePrefix, stagingDir), newFile.path()));
        }
      }
      return result;
    } catch (IOException e) {
      throw new UncheckedIOException(
          "Failed to rewrite the manifest list file " + snapshot.manifestListLocation(), e);
    }
  }

  private static List<ManifestFile> manifestFilesInSnapshot(FileIO io, Snapshot snapshot) {
    try {
      return snapshot.allManifests(io);
    } catch (RuntimeIOException e) {
      LOG.warn("Failed to read manifest list {}", snapshot.manifestListLocation(), e);
      return ImmutableList.of();
    }
  }

  /**
   * Rewrite a data manifest, replacing path references.
   *
   * @param manifestFile source manifest file to rewrite
   * @param snapshotIds snapshot ids for filtering returned data manifest entries
   * @param outputFile output file to rewrite manifest file to
   * @param io file io
   * @param format format of the manifest file
   * @param specsById map of partition specs by id
   * @param sourcePrefix source prefix that will be replaced
   * @param targetPrefix target prefix that will replace it
   * @return a copy plan of content files in the manifest that was rewritten, recording the
   *     rewritten manifest's byte length
   */

View on GitHub (pinned to 86d9c8fc54)