What causes "Failed to read manifest file: %s" in apache/iceberg?

Verify the manifest file exists and is readable via the table's FileIO (check credentials and permissions)

apache/iceberg · error · RuntimeIOException

Failed to read manifest file: %s

Error message

Failed to read manifest file: %s

What it means

During expiration of table data (dropTableData / deleteRemovedMetadataFiles), Iceberg reads each manifest to determine which files are deletable; if an IOException occurs while reading, it is wrapped in RuntimeIOException with the manifest path. Delete operations abort rather than silently skipping, since skipping could orphan or wrongly retain/delete data files.

Source

Thrown at core/src/main/java/org/apache/iceberg/CatalogUtil.java:196

                LOG.warn("Failed to get deleted files: this may cause orphaned data files", exc))
        .run(
            manifest -> {
              try (ManifestReader<?> reader = ManifestFiles.open(manifest, io, specsById)) {
                List<String> pathsToDelete = Lists.newArrayList();
                for (ManifestEntry<?> entry : reader.entries()) {
                  // intern the file path because the weak key map uses identity (==) instead of
                  // equals
                  String path = entry.file().location().intern();
                  Boolean alreadyDeleted = deletedFiles.putIfAbsent(path, true);
                  if (alreadyDeleted == null || !alreadyDeleted) {
                    pathsToDelete.add(path);
                  }
                }

                String type = reader.isDeleteManifestReader() ? "delete" : "data";
                deleteFiles(io, pathsToDelete, type, false);
              } catch (IOException e) {
                throw new RuntimeIOException(
                    e, "Failed to read manifest file: %s", manifest.path());
              }
            });
  }

  /**
   * Helper to delete files. Bulk deletion is used if possible, otherwise deletions are done
   * concurrently for non-bulk FileIO.
   *
   * @param io FileIO for deletes
   * @param files files to delete
   * @param type type of files being deleted
   */
  public static void deleteFiles(FileIO io, Iterable<String> files, String type) {
    deleteFiles(io, files, type, true);
  }

  /**

View on GitHub (pinned to 86d9c8fc54)

Solutions

Verify the manifest file exists and is readable via the table's FileIO (check credentials and permissions)
Re-run the operation after fixing transient storage/credential issues
Ensure no external process deletes metadata files concurrently
Restore the corrupted manifest from backup if available

Example fix

// before
table.expireSnapshots().expireOlderThan(ts).commit(); // may throw on unreadable manifest
// after
try {
  table.expireSnapshots().expireOlderThan(ts).commit();
} catch (RuntimeIOException e) {
  logger.error("Manifest unreadable during expiration: {}", e.getMessage(), e);
  // fix storage access, then retry
}

Defensive patterns

Strategy: try-catch

Try / catch

catch (RuntimeIOException e) { if (e.getCause() instanceof IOException) { /* check credentials/connectivity for the manifest path */ } throw e; }

Prevention

Never manually delete files under the table's metadata directory
Ensure FileIO credentials are valid and long-lived enough for the whole expiration
Avoid concurrent metadata cleanup by multiple jobs
Test expiration against a copy of production metadata before running at scale

When it happens

Trigger: Calling Table.expireSnapshots()...expireOlderThan(ts).commit() or catalog.dropTableData(ident) when a manifest listed in a snapshot cannot be read — file deleted/corrupted in storage, credentials/permission problems, transient storage errors.

Common situations: Manifests removed manually or by an external lifecycle policy; expired S3/GCS/ADLS credentials mid-operation; network partitions to object storage.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/67dfe41048c4c8c7. Report an issue: GitHub.