What causes "Stream closed" in apache/hadoop?

Use try-with-resources so the stream lifetime is lexical and single-owner

apache/hadoop · error · IOException

Stream closed

Error message

Stream closed

What it means

CryptoInputStream.checkStream() runs at the top of every read/seek/positioned operation; once close() has set closed = true (after freeing buffers and the codec), any subsequent operation throws IOException("Stream closed"). The guard fails fast instead of touching freed native buffers/decryptors.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CryptoInputStream.java:786

  @Override
  public FileDescriptor getFileDescriptor() throws IOException {
    if (in instanceof HasFileDescriptor) {
      return ((HasFileDescriptor) in).getFileDescriptor();
    } else if (in instanceof FileInputStream) {
      return ((FileInputStream) in).getFD();
    } else {
      return null;
    }
  }
  
  @Override
  public int read() throws IOException {
    return (read(oneByteBuf, 0, 1) == -1) ? -1 : (oneByteBuf[0] & 0xff);
  }
  
  private void checkStream() throws IOException {
    if (closed) {
      throw new IOException("Stream closed");
    }
  }
  
  /** Get direct buffer from pool */
  private ByteBuffer getBuffer() {
    ByteBuffer buffer = bufferPool.poll();
    if (buffer == null) {
      buffer = ByteBuffer.allocateDirect(bufferSize);
    }
    
    return buffer;
  }
  
  /** Return direct buffer to pool */
  private void returnBuffer(ByteBuffer buf) {
    if (buf != null) {
      buf.clear();
      bufferPool.add(buf);

View on GitHub (pinned to 2add963021)

Solutions

Use try-with-resources so the stream lifetime is lexical and single-owner
Null out the reference on close so later use fails at the owner with a clear NPE instead of deep in the stream
Coordinate close with an in-use latch or read lock so closes wait for outstanding reads
For shared access, wrap in a reference-counted handle that closes only when the last user finishes

Example fix

// before
FSDataInputStream in = fs.open(path);
process(in);
in.close();
readMore(in); // IOException: Stream closed

// after
try (FSDataInputStream in = fs.open(path)) {
  process(in);
  readMore(in);
} // closed exactly once, no use after close

Defensive patterns

Strategy: validation

Validate before calling

// No public isOpen()/isClosed() on CryptoInputStream; enforce ownership instead.
// Owner-tracked guard:
private FSDataInputStream in; // single owner, guarded by this
synchronized boolean isOpen() { return in != null; }
synchronized void closeQuietly() { IOUtils.closeStream(in); in = null; }

Try / catch

try {
  n = in.read(buf, off, len);
} catch (IOException e) {
  if ("Stream closed".equals(e.getMessage())) {
    throw new IllegalStateException("read after close on " + path, e);
  }
  throw e;
}

Prevention

Open streams with try-with-resources so scope equals lifetime
Never share a stream across threads without an external close/read protocol
Null out references after close so misuse surfaces at the owner
Use reference counting for shared file handles; close only on final release

When it happens

Trigger: read()/seek()/readFully() called after close(); concurrent close() from one thread while another thread is reading (checkStream passes, then close frees buffers mid-decrypt — a use-after-close race); wrapper layers forwarding operations after an inner close.

Common situations: Reader objects outliving their stream in file-handle caches or connection pools; an error/timeout handler closing the stream while a worker still reads; frameworks that close sinks on failure and then retry reads on the same handle.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/11c0ffa2d5e977f8. Report an issue: GitHub.