apache/iceberg · error · RuntimeIOException

Failed to create snapshot list writer for path: %s

Error message

Failed to create snapshot list writer for path: %s

What it means

Thrown as a RuntimeIOException when the Avro file appender for a manifest list (snapshot list) cannot be created at the given output file location. The library wraps the underlying IOException so callers get a single unchecked exception type for I/O failures during snapshot commit. It almost always indicates a problem with the underlying FileIO/filesystem, not with Iceberg logic.

Source

Thrown at core/src/main/java/org/apache/iceberg/ManifestListWriter.java:156

        // leave space for existing and added rows, in case any of the existing data files do not
        // have an assigned first-row-id (this is the case with manifests from pre-v3 snapshots)
        this.nextRowId += manifest.existingRowsCount() + manifest.addedRowsCount();
        return wrapper;
      }
    }

    @Override
    protected FileAppender<ManifestFile> newAppender(OutputFile file, Map<String, String> meta) {
      try {
        return InternalData.write(FileFormat.AVRO, file)
            .schema(V4Metadata.MANIFEST_LIST_SCHEMA)
            .named("manifest_file")
            .meta(meta)
            .overwrite()
            .build();

      } catch (IOException e) {
        throw new RuntimeIOException(
            e, "Failed to create snapshot list writer for path: %s", file.location());
      }
    }

    @Override
    public Long nextRowId() {
      return nextRowId;
    }
  }

  static class V3Writer extends ManifestListWriter {
    private final V3Metadata.ManifestFileWrapper wrapper;
    private Long nextRowId;

    V3Writer(
        OutputFile snapshotFile,
        EncryptionManager encryptionManager,
        long snapshotId,

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check the wrapped IOException cause for the real filesystem error (credentials, permissions, missing bucket).
2. Verify the table's FileIO configuration (io-impl and provider properties) and that credentials/config reach executors.
3. Confirm the output location/bucket exists and is writable; test writing a file to the same path manually.
4. Retry the commit; transient network failures against object stores are common.
5. Check network/proxy connectivity from the machine performing the commit.

Example fix

// before: FileIO created with no provider config
Table table = catalog.loadTable(identifier); // default HadoopFileIO, no S3 creds
// after: configure a concrete FileIO with credentials
catalogBuilder.withProperties(Map.of(
    CatalogProperties.FILE_IO_IMPL, "org.apache.iceberg.aws.s3.S3FileIO",
    "s3.access-key-id", ak,
    "s3.secret-access-key", sk));

// pre-commit storage sanity check
try (var out = table.io().newOutputFile(metadataPath).createOrOverwrite()) {
  // probe write succeeded
} catch (IOException e) {
  throw new IllegalStateException("Storage not writable: " + metadataPath, e);
}

try {
  table.refresh();
  Transaction tx = table.newTransaction();
  tx.commit();
} catch (RuntimeIOException e) {
  logger.error("Manifest list write failed; cause: {}", e.getCause(), e);
  // fix FileIO config/credentials, then retry
}

When it happens

Trigger: ManifestListWriter.newAppender calls fileIO.newOutputFile(location) and builds an Avro encoder; IOException from opening or preparing the output stream is wrapped. Specifically triggered by ManifestLists.write when writing v2/v3 snapshot files.

Common situations: Object store credentials missing or expired (S3/GCS/ADLS); bucket/container does not exist; network outage; permission denied on the output path; Hadoop misconfiguration (missing core-site.xml) in a Spark/Flink executor; disk full on local filesystems.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Cannot read manifest list file: %s
• Failed to create manifest writer for path: %s
• Cannot read manifest list file: %s
• Failed to close manifest reader
• Failed to write manifest list file

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/1b7ef9594f700058. Report an issue: GitHub.