What causes "key + ": Stream is closed!"" in apache/hadoop?

Narrow try-with-resources (or finally-close) scope to exactly the code that reads

apache/hadoop · error · IOException

key + ": Stream is closed!"

Error message

key + ": Stream is closed!"

What it means

Every public operation on BosInputStream starts with checkNotClosed(); once the volatile closed flag is set — by close() or by internal failure paths such as closeStream('native store retrieve failed') — any further read/seek throws IOException '<key>: Stream is closed'. The message includes the object key so you can identify which stream was used after close.

Source

Thrown at hadoop-cloud-storage-project/hadoop-bos/src/main/java/org/apache/hadoop/fs/bos/BosInputStream.java:340

  public synchronized void setReadahead(Long readahead) {
    if (readahead == null) {
      this.readahead = DEFAULT_READAHEAD_LEN;
    } else {
      this.readahead = Math.max(
          readahead, DEFAULT_READAHEAD_LEN);
    }
  }

  /**
   * Verify that the input stream is open. Non blocking; this
   * gives the last state of the volatile {@link #closed}
   * field.
   *
   * @throws IOException if the connection is closed.
   */
  private void checkNotClosed() throws IOException {
    if (closed) {
      throw new IOException(
          key + ": "
              + FSExceptionMessages.STREAM_IS_CLOSED);
    }
  }

  /**
   * Perform lazy seek and adjust stream to correct position
   * for reading.
   *
   * @param targetPos position from where data should be read
   * @param len length of the content that needs to be read
   * @throws IOException if an I/O error occurs
   */
  private void lazySeek(long targetPos, long len)
      throws IOException {
    // For lazy seek
    try {
      seekInStream(targetPos, len);

View on GitHub (pinned to 2add963021)

Solutions

Narrow try-with-resources (or finally-close) scope to exactly the code that reads
After any IOException from a read, discard the stream and open a new one — internal failures mark it closed
Give the stream a single owner for its full lifecycle; do not share across threads

Example fix

// before
try (FSDataInputStream in = fs.open(p)) {
  return parse(in); // parse also called later with closed stream
}
parse(cachedIn); // reused after close

// after: open per consumer
try (FSDataInputStream in = fs.open(p)) {
  return parse(in);
}

Defensive patterns

Strategy: validation

Validate before calling

// track ownership instead of probing the stream
boolean readDone = false;
try (FSDataInputStream in = fs.open(p)) {
  readDone = consume(in);
} // nothing touches in after this block

Type guard

static boolean isStreamClosedMessage(IOException e) {
  return e.getMessage() != null && e.getMessage().endsWith("Stream is closed");
}

Try / catch

catch (IOException e) {
  if (isStreamClosedMessage(e)) {
    in = fs.open(path); // stream died earlier: reopen, do not reuse
    in.seek(lastGoodPos);
  } else { throw e; }
}

Prevention

Scope try-with-resources to exactly the reading code
Treat any prior read IOException as fatal for that stream instance
One owner thread per stream for its whole lifecycle

When it happens

Trigger: Calling read/seek/available after close(); continuing to use a stream whose earlier read already failed (internal recovery closes it); two threads sharing a stream where one closes it while the other reads.

Common situations: try-with-resources scope wrapped too wide, so a helper still holds the stream when the block exits; error paths that half-consume the stream and then continue processing; frameworks that pool or hand off streams across close boundaries.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9a76caefe7a9a9ea. Report an issue: GitHub.