What causes "Unclosed output stream created by: {}" in apache/iceberg?

Close output streams in try-with-resources or finally blocks.

Unclosed output stream created by

Error message

Unclosed output stream created by:
	{}

What it means

HadoopStreams wraps Hadoop FSDataOutputStream as Iceberg PositionOutputStream. If the stream is finalized without being closed, finalize() closes it and logs a WARN with the creation-time stack trace, signaling a leaked output handle (possibly also incomplete/abandoned writes).

Solutions

Close output streams in try-with-resources or finally blocks.
Use Iceberg writers (FileAppender/DataWriter) which manage close semantics and call close() on completion or in finally.
Follow the creation stack trace in the WARN to locate the leaking open call.
On failed writes, delete the orphan output file after closing.

Example fix

// before
PositionOutputStream out = output.create();
out.write(data);
// missing close on exception path
// after
try (PositionOutputStream out = output.create()) {
  out.write(data);
}

Defensive patterns

Strategy: try-catch

Try / catch

try (PositionOutputStream out = outputFile.create()) {
  out.write(bytes);
} // closed even on exception; orphan cleanup on failure if needed

Prevention

Use try-with-resources for every output stream/writer.
Close Iceberg writers (DataWriter/FileAppender) in finally or via try-with-resources.
Delete orphan output files when a write fails after creation.
Trace any 'Unclosed output stream' WARN via its creation stack.

When it happens

Trigger: Code obtains table.io().newOutputFile(...).create(...) and never calls close() — e.g. writer objects not closed on exception paths, or output streams stored and abandoned after task failure.

Common situations: Custom file-writing code without try-with-resources; exceptions between create and close; task cancellation leaving writers open; leaked metadata/data writers on commit failure paths.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/049238a504d112d9. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopStreams.java:206

      // Need to check the interrupted flag to detect failed object upload
      // and propagate the error up.
      if (Thread.interrupted()
          && "org.apache.hadoop.fs.s3a.S3ABlockOutputStream"
              .equals(stream.getWrappedStream().getClass().getName())) {
        throw new IOException(
            "S3ABlockOutputStream failed to upload object after stream was closed");
      }
    }

    @SuppressWarnings({"checkstyle:NoFinalizer", "Finalize", "deprecation"})
    @Override
    protected void finalize() throws Throwable {
      super.finalize();
      if (!closed) {
        close(); // releasing resources is more important than printing the warning
        String trace =
            Joiner.on("\n\t").join(Arrays.copyOfRange(createStack, 1, createStack.length));
        LOG.warn("Unclosed output stream created by:\n\t{}", trace);
      }
    }
  }

  private static class WrappedSeekableInputStream extends FSInputStream
      implements DelegatingInputStream {
    private final SeekableInputStream inputStream;

    private WrappedSeekableInputStream(SeekableInputStream inputStream) {
      this.inputStream = inputStream;
    }

    @Override
    public void seek(long pos) throws IOException {
      inputStream.seek(pos);
    }

    @Override

View on GitHub (pinned to 86d9c8fc54)