apache/hadoop · error · IOException

Stream is closed!

Error message

Stream is closed!

What it means

Thrown by CosNInputStream.read() (single-byte variant) when the stream's closed flag is already set. Like all CosN read-path methods, it refuses any I/O after close() and surfaces the shared Hadoop message FSExceptionMessages.STREAM_IS_CLOSED. It indicates the caller kept or reused a stream reference after closing it (or after a framework helper like IOUtils.closeQuietly closed it).

Source

Thrown at hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNInputStream.java:275

      this.reopen(pos);
    }
  }

  @Override
  public long getPos() {
    return this.position;
  }

  @Override
  public boolean seekToNewSource(long targetPos) {
    // Currently does not support to seek the offset of a new source
    return false;
  }

  @Override
  public int read() throws IOException {
    if (this.closed) {
      throw new IOException(FSExceptionMessages.STREAM_IS_CLOSED);
    }

    if (this.partRemaining <= 0 && this.position < this.fileSize) {
      this.reopen(this.position);
    }

    int byteRead = -1;
    if (this.partRemaining != 0) {
      byteRead = this.buffer[
          (int) (this.buffer.length - this.partRemaining)] & 0xff;
    }
    if (byteRead >= 0) {
      this.position++;
      this.partRemaining--;
      if (null != this.statistics) {
        this.statistics.incrementBytesRead(byteRead);
      }
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Restructure ownership: exactly one component owns close(), and no read is issued after that component's lifecycle ends.
2. Set the reference to null immediately after close() so a second use fails fast with NPE at the correct call site instead of this message.
3. If multiple consumers share one stream, wrap it in a ref-counted stream (e.g., a FilterInputStream with an acquire/release count) and close only when the count hits zero.
4. In concurrent code, synchronize or use an AtomicBoolean closed flag checked before every read; interrupt/await the reader before closing.

Example fix

// before
try {
  return in.read();
} finally {
  in.close();   // next call to read() throws 'Stream is closed!'
}

// after
try {
  return in.read();
} finally {
  in.close();
  in = null;    // fail fast with NPE if read is attempted later
}

if (in == null) throw new IllegalStateException("stream already closed");
// own flag when ownership is shared:
// if (!open.get()) throw new IllegalStateException(...);
int b = in.read();

catch (IOException e) {
  if (FSExceptionMessages.STREAM_IS_CLOSED.equals(e.getMessage())) {
    // lifecycle bug: reopen and replay the range from a known offset
    in = fs.open(path); in.seek(lastGoodPos);
  } else { throw e; }
}

When it happens

Trigger: Calling read() on a CosNInputStream after close() ran; typical shapes: close() in a finally block that also falls through to more reads, two code paths closing the same FSDataInputStream (double close is harmless, but read-after-second-close is not), or a reader thread racing a closer thread.

Common situations: Utility wrappers (e.g., custom RecordReader) that close the inner stream in nextKeyValue() but keep reading; retry loops that close the stream on first error then unconditionally read in a catch/finally block; caching layers that evict (and close) entries while another consumer still reads.

Related errors

• key + ": Stream is closed!"
• Cannot seek to a negative offset
• Attempted to seek or read past the end of the file
• Null IO stream
• Credentials requested after provider list was closed

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9a19daa91ffdc141. Report an issue: GitHub.