apache/iceberg · critical · RuntimeIOException

Failed to write manifest

Error message

Failed to write manifest

What it means

FastAppend.apply wraps its call to writeNewManifests() in a try/catch that converts any IOException from the underlying manifest writer into RuntimeIOException('Failed to write manifest'). It means Iceberg could not write a new Avro manifest file to the table's FileIO while preparing an append snapshot, so the append cannot be committed.

Source

Thrown at core/src/main/java/org/apache/iceberg/FastAppend.java:156

        manifest.partitionSpecId(),
        toCopy,
        current.specsById(),
        newManifestFile,
        snapshotId(),
        summaryBuilder);
  }

  @Override
  public List<ManifestFile> apply(TableMetadata base, Snapshot snapshot) {
    List<ManifestFile> manifests = Lists.newArrayList();

    try {
      List<ManifestFile> newWrittenManifests = writeNewManifests();
      if (newWrittenManifests != null) {
        manifests.addAll(newWrittenManifests);
      }
    } catch (IOException e) {
      throw new RuntimeIOException(e, "Failed to write manifest");
    }

    Iterable<ManifestFile> appendManifestsWithMetadata =
        Iterables.transform(
            Iterables.concat(appendManifests, rewrittenAppendManifests),
            manifest -> GenericManifestFile.copyOf(manifest).withSnapshotId(snapshotId()).build());
    Iterables.addAll(manifests, appendManifestsWithMetadata);

    if (snapshot != null) {
      manifests.addAll(snapshot.allManifests(ops().io()));
    }

    summaryBuilder.merge(buildManifestCountSummary(manifests, 0));

    return manifests;
  }

  @Override

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check the wrapped cause (`e.getCause()` / RuntimeIOException.getCause()) for the real storage error and fix storage access (credentials, permissions, connectivity).
2. Verify the table location and write.target-file-size / write locations are writable by the job's identity.
3. Test FileIO directly: write a scratch file to the same location with the same FileIO config before retrying the append.
4. Retry the append after transient network/storage failures; FastAppend.apply() has no side effects on commit failure.

Example fix

// before: swallowing the cause
catch (IOException e) {
  throw new RuntimeIOException(e, "Failed to write manifest");
}
// after (user side): surface the root cause
try {
  append.commit();
} catch (RuntimeIOException e) {
  LOG.error("Manifest write failed; root cause: {}", e.getCause(), e);
  throw e;
}

// pre-flight: ensure the table location is writable with the current FileIO
fileIO.newOutputFile(tableLocation + "/.write-check").create();

try {
  table.newFastAppend().appendFile(df).commit();
} catch (RuntimeIOException e) {
  LOG.error("Manifest write failed; root cause", e.getCause());
  if (isTransient(e.getCause())) retryWithBackoff();
  else throw e;
}

When it happens

Trigger: Calling table.newFastAppend().appendFile(...).apply() (or commit) when the manifest writer fails: unwritable target location, expired/invalid cloud credentials, missing bucket, HDFS NameNode unreachable, disk full, or FileIO misconfiguration.

Common situations: S3/GCS credentials rotated or missing IAM permissions on the table location; HDFS in safe mode; network partition to storage; wrong warehouse path configured (read-only or nonexistent filesystem); local FS used in container with read-only rootfs.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed to read manifest file: %s
• Failed to read manifest:
• RuntimeIOException
• Failed to validate replaced partitions
• Failed to read manifest file: %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/cd66f991edd6b048. Report an issue: GitHub.