apache/iceberg · error · UncheckedIOException

Failed to close equality delta writer

Error message

Failed to close equality delta writer

What it means

PartitionedDeltaWriter.close() closes the parent writer and then closes every per-partition RowDataDeltaWriter via Tasks.foreach; if any of those closes throw IOException it wraps it in UncheckedIOException 'Failed to close equality delta writer'. This indicates one or more data/delete files opened by a partition writer could not be finalized (flush/close failed), so written files may be incomplete or leaked.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/PartitionedDeltaWriter.java:97

      writer = new RowDataDeltaWriter(copiedKey, dvFileWriter());
      writers.put(copiedKey, writer);
    }

    return writer;
  }

  @Override
  public void close() {
    try {
      super.close();
      Tasks.foreach(writers.values())
          .throwFailureWhenFinished()
          .noRetry()
          .run(RowDataDeltaWriter::close, IOException.class);

      writers.clear();
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to close equality delta writer", e);
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Read the chained IOException cause to identify which file/filesystem failed
2. Fix the storage issue (free disk/quota, verify credentials, retry after object-store throttling) and let Flink restart from the last checkpoint — the sink is failure-tolerant on restart
3. Reduce partition cardinality per checkpoint (or enable write parallelism tuning) so fewer writers are open simultaneously
4. Check the target location's write permissions for the job's identity

Example fix

// before
// job fails: UncheckedIOException: Failed to close equality delta writer ... S3Exception: Slow Down
// after
// add retry/resilience on the filesystem side (s3.retry-throttling) and ensure checkpoints restart cleanly
conf.set("fs.s3a.retry.limit", "20");
conf.set("fs.s3a.assumed.role.session.duration", "8h"); // avoid mid-checkpoint credential expiry

// pre-flight: ensure target warehouse is writable and has headroom
io.doAs(() -> { try (OutputFile f = io.newOutputFile(targetPath + "/.write-probe")) {
  f.createOrOverwrite().write(new byte[1]); } return true; });

try {
  partitionedWriter.close();
} catch (UncheckedIOException e) {
  LOG.error("delta writer close failed: {} — files may be leaked; will retry from checkpoint", e.getCause(), e);
  throw e; // let Flink checkpoint/restart semantics recover
}

When it happens

Trigger: Closing a PartitionedDeltaWriter (equality-delete / upsert path) during checkpoint snapshot or writer dispose when an underlying file writer's close fails: disk full, filesystem outage, object-store throttling, or permission problems writing to the data directory.

Common situations: S3/HDFS transient errors during checkpoint, warehouse directory quota or disk space exceeded, credential rotation mid-job, or very high partition cardinality creating many concurrent open writers whose close fails.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed to close equality delta writer
• Failed to close equality delta writer
• Failed to close equality delta writer
• Failed to close current writer
• Failed to close dvFileWriter

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/922972e7b8641c25. Report an issue: GitHub.