What causes "Unclosed input stream created by: {}" in apache/iceberg?

Wrap stream-producing constructs in try-with-resources and close CloseableIterables/Readers in finally.

Unclosed input stream created by

Error message

Unclosed input stream created by:
	{}

What it means

HadoopStreams wraps Hadoop FSDataInputStream as Iceberg SeekableInputStream. When such a stream is garbage-collected without close() being called, finalize() closes it and logs a WARN containing the stack trace captured at creation, indicating the caller leaked a file handle. Unclosed streams can exhaust filesystem/socket resources.

Solutions

Wrap stream-producing constructs in try-with-resources and close CloseableIterables/Readers in finally.
Ensure callers close SeekableInputStream explicitly; do not rely on GC/finalize.
Check for early returns/exceptions between open and close in custom reader code; add close in finally.
Heap-dump or use the logged creation stack trace to find the leaking call site.

Example fix

// before
SeekableInputStream in = file.newStream();
AvroIterable<T> it = Avro.read(in).build();
return it.iterator();
// after
try (AvroIterable<T> it = Avro.read(file.newStream()).build()) {
  return ImmutableList.copyOf(it);
}

Defensive patterns

Strategy: try-catch

Try / catch

try (SeekableInputStream in = file.newStream()) {
  // read
} // auto-closed even on exception

Prevention

Always use try-with-resources for streams and iterables.
Close CloseableIterable/CloseableIterator in finally blocks.
Never rely on GC to release file handles.
Investigate any 'Unclosed input stream' WARN immediately using its stack trace.

When it happens

Trigger: A reader (e.g. Avro/Parquet reader or scan task) opens table.io().newInputFile(...).newStream() and is not closed — e.g. a CloseableIterable not fully consumed/closed, an exception path skipping close, or user code holding the stream without try-with-resources.

Common situations: Custom scan/reader code that breaks out of iteration early without closing; exceptions during read that skip finally blocks; streaming results to a consumer that never closes the reader.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/f8d283f00eaefc73. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopStreams.java:133

    @Override
    public int read() throws IOException {
      return stream.read();
    }

    @Override
    public int read(byte[] b, int off, int len) throws IOException {
      return stream.read(b, off, len);
    }

    @SuppressWarnings({"checkstyle:NoFinalizer", "Finalize", "deprecation"})
    @Override
    protected void finalize() throws Throwable {
      super.finalize();
      if (!closed) {
        close(); // releasing resources is more important than printing the warning
        String trace =
            Joiner.on("\n\t").join(Arrays.copyOfRange(createStack, 1, createStack.length));
        LOG.warn("Unclosed input stream created by:\n\t{}", trace);
      }
    }
  }

  /** PositionOutputStream implementation for FSDataOutputStream. */
  private static class HadoopPositionOutputStream extends PositionOutputStream
      implements DelegatingOutputStream {
    private final FSDataOutputStream stream;
    private final StackTraceElement[] createStack;
    private boolean closed;

    HadoopPositionOutputStream(FSDataOutputStream stream) {
      this.stream = stream;
      this.createStack = Thread.currentThread().getStackTrace();
      this.closed = false;
    }

    @Override

View on GitHub (pinned to 86d9c8fc54)