apache/iceberg · error · RuntimeIOException

Failed to create snapshot list writer for path

Error message

Failed to create snapshot list writer for path: %s

What it means

Thrown as a RuntimeIOException when the Avro file appender for a manifest list (snapshot list) cannot be created at the given output file location. The library wraps the underlying IOException so callers get a single unchecked exception type for I/O failures during snapshot commit. It almost always indicates a problem with the underlying FileIO/filesystem, not with Iceberg logic.

Solutions

1. Check the wrapped IOException cause for the real filesystem error (credentials, permissions, missing bucket).
2. Verify the table's FileIO configuration (io-impl and provider properties) and that credentials/config reach executors.
3. Confirm the output location/bucket exists and is writable; test writing a file to the same path manually.
4. Retry the commit; transient network failures against object stores are common.
5. Check network/proxy connectivity from the machine performing the commit.

Example fix

// before: FileIO created with no provider config
Table table = catalog.loadTable(identifier); // default HadoopFileIO, no S3 creds
// after: configure a concrete FileIO with credentials
catalogBuilder.withProperties(Map.of(
    CatalogProperties.FILE_IO_IMPL, "org.apache.iceberg.aws.s3.S3FileIO",
    "s3.access-key-id", ak,
    "s3.secret-access-key", sk));

// pre-commit storage sanity check
try (var out = table.io().newOutputFile(metadataPath).createOrOverwrite()) {
  // probe write succeeded
} catch (IOException e) {
  throw new IllegalStateException("Storage not writable: " + metadataPath, e);
}

try {
  table.refresh();
  Transaction tx = table.newTransaction();
  tx.commit();
} catch (RuntimeIOException e) {
  logger.error("Manifest list write failed; cause: {}", e.getCause(), e);
  // fix FileIO config/credentials, then retry
}

When it happens

Trigger: ManifestListWriter.newAppender calls fileIO.newOutputFile(location) and builds an Avro encoder; IOException from opening or preparing the output stream is wrapped. Specifically triggered by ManifestLists.write when writing v2/v3 snapshot files.

Common situations: Object store credentials missing or expired (S3/GCS/ADLS); bucket/container does not exist; network outage; permission denied on the output path; Hadoop misconfiguration (missing core-site.xml) in a Spark/Flink executor; disk full on local filesystems.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Cannot read manifest list file
• Failed to create manifest writer for path
• Can't create file
• Cannot read manifest list file
• Decoding datum failed

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/1b7ef9594f700058. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/ManifestListWriter.java:156

        // leave space for existing and added rows, in case any of the existing data files do not
        // have an assigned first-row-id (this is the case with manifests from pre-v3 snapshots)
        this.nextRowId += manifest.existingRowsCount() + manifest.addedRowsCount();
        return wrapper;
      }
    }

    @Override
    protected FileAppender<ManifestFile> newAppender(OutputFile file, Map<String, String> meta) {
      try {
        return InternalData.write(FileFormat.AVRO, file)
            .schema(V4Metadata.MANIFEST_LIST_SCHEMA)
            .named("manifest_file")
            .meta(meta)
            .overwrite()
            .build();

      } catch (IOException e) {
        throw new RuntimeIOException(
            e, "Failed to create snapshot list writer for path: %s", file.location());
      }
    }

    @Override
    public Long nextRowId() {
      return nextRowId;
    }
  }

  static class V3Writer extends ManifestListWriter {
    private final V3Metadata.ManifestFileWrapper wrapper;
    private Long nextRowId;

    V3Writer(
        OutputFile snapshotFile,
        EncryptionManager encryptionManager,
        long snapshotId,

View on GitHub (pinned to 86d9c8fc54)