{"record":{"id":"922972e7b8641c25","repo":"apache/iceberg","slug":"failed-to-close-equality-delta-writer","errorCode":null,"errorMessage":"Failed to close equality delta writer","messagePattern":"Failed to close equality delta writer","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/PartitionedDeltaWriter.java","lineNumber":97,"sourceCode":"      writer = new RowDataDeltaWriter(copiedKey, dvFileWriter());\n      writers.put(copiedKey, writer);\n    }\n\n    return writer;\n  }\n\n  @Override\n  public void close() {\n    try {\n      super.close();\n      Tasks.foreach(writers.values())\n          .throwFailureWhenFinished()\n          .noRetry()\n          .run(RowDataDeltaWriter::close, IOException.class);\n\n      writers.clear();\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Failed to close equality delta writer\", e);\n    }\n  }\n}\n","sourceCodeStart":79,"sourceCodeEnd":101,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/PartitionedDeltaWriter.java#L79-L101","documentation":"PartitionedDeltaWriter.close() closes the parent writer and then closes every per-partition RowDataDeltaWriter via Tasks.foreach; if any of those closes throw IOException it wraps it in UncheckedIOException 'Failed to close equality delta writer'. This indicates one or more data/delete files opened by a partition writer could not be finalized (flush/close failed), so written files may be incomplete or leaked.","triggerScenarios":"Closing a PartitionedDeltaWriter (equality-delete / upsert path) during checkpoint snapshot or writer dispose when an underlying file writer's close fails: disk full, filesystem outage, object-store throttling, or permission problems writing to the data directory.","commonSituations":"S3/HDFS transient errors during checkpoint, warehouse directory quota or disk space exceeded, credential rotation mid-job, or very high partition cardinality creating many concurrent open writers whose close fails.","solutions":["Read the chained IOException cause to identify which file/filesystem failed","Fix the storage issue (free disk/quota, verify credentials, retry after object-store throttling) and let Flink restart from the last checkpoint — the sink is failure-tolerant on restart","Reduce partition cardinality per checkpoint (or enable write parallelism tuning) so fewer writers are open simultaneously","Check the target location's write permissions for the job's identity"],"exampleFix":"// before\n// job fails: UncheckedIOException: Failed to close equality delta writer ... S3Exception: Slow Down\n// after\n// add retry/resilience on the filesystem side (s3.retry-throttling) and ensure checkpoints restart cleanly\nconf.set(\"fs.s3a.retry.limit\", \"20\");\nconf.set(\"fs.s3a.assumed.role.session.duration\", \"8h\"); // avoid mid-checkpoint credential expiry","handlingStrategy":"retry","validationCode":"// pre-flight: ensure target warehouse is writable and has headroom\nio.doAs(() -> { try (OutputFile f = io.newOutputFile(targetPath + \"/.write-probe\")) {\n  f.createOrOverwrite().write(new byte[1]); } return true; });","typeGuard":null,"tryCatchPattern":"try {\n  partitionedWriter.close();\n} catch (UncheckedIOException e) {\n  LOG.error(\"delta writer close failed: {} — files may be leaked; will retry from checkpoint\", e.getCause(), e);\n  throw e; // let Flink checkpoint/restart semantics recover\n}","preventionTips":["Monitor disk/quota and object-store error rates on writer taskmanagers","Avoid credential rotation mid-job; use long-lived sessions or role assumption","Limit per-task partition cardinality or increase write parallelism to reduce open writers","Rely on checkpoint restart rather than swallowing close failures — Iceberg files are transactional"],"tags":["flink","io","checkpoint","writer"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}