apache/hadoop · error · IOException

{}: Stream is closed!

Error message

{}: Stream is closed!

What it means

OBSInputStream.checkNotClosed() guards every positional operation (seek, read, readFully, getPos when invoked via those paths): once the volatile closed flag is set by close(), any further operation throws IOException(uri + ': ' + STREAM_IS_CLOSED). The uri prefix distinguishes this from the block-buffer variant. Closing is synchronized and idempotent ('all later/blocked calls are no-ops'), so the error always means the caller used the stream after close — never a close-side race.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSInputStream.java:752

    long endTime = System.currentTimeMillis();
    LOG.debug(
        "Read-3args uri:{}, contentLength:{}, destLen:{}, readLen:{}, "
            + "position:{}, thread:{}, timeUsedMilliSec:{}",
        uri, contentLength, len, bytesRead,
        bytesRead >= 0 ? nextReadPos - bytesRead : nextReadPos, threadId,
        endTime - startTime);
    return bytesRead;
  }

  /**
   * Verify that the input stream is open. Non blocking; this gives the last
   * state of the volatile {@link #closed} field.
   *
   * @throws IOException if the connection is closed.
   */
  private void checkNotClosed() throws IOException {
    if (closed) {
      throw new IOException(
          uri + ": " + FSExceptionMessages.STREAM_IS_CLOSED);
    }
  }

  /**
   * Close the stream. This triggers publishing of the stream statistics back to
   * the filesystem statistics. This operation is synchronized, so that only one
   * thread can attempt to close the connection; all later/blocked calls are
   * no-ops.
   *
   * @throws IOException on any problem
   */
  @Override
  public synchronized void close() throws IOException {
    if (!closed) {
      closed = true;
      // close or abort the stream
      closeStream("close() operation", this.contentRangeFinish);

View on GitHub (pinned to 2add963021)

Solutions

1. Keep all reads strictly inside the try-with-resources (or try/finally) block that owns the stream
2. Centralize ownership: exactly one component closes the stream; readers check a shared closed/lifecycle flag before I/O
3. On cancellation paths, signal readers to stop before closing their streams (CountDownLatch/AtomicBoolean), then close
4. Wrap OBS reads in a guard: if (in instanceof FSDataInputStream) use available()/getPos() carefully — but the reliable fix is lifecycle discipline, not probing state

Example fix

// before
FSDataInputStream in = fs.open(path);
... in.close();
return in.read(); // IOException: s3a://...: Stream is closed!

// after
try (FSDataInputStream in = fs.open(path)) {
  return readAll(in);
}

try {
  return in.read(buf, off, len);
} catch (IOException e) {
  if (String.valueOf(e.getMessage()).endsWith(FSExceptionMessages.STREAM_IS_CLOSED)) {
    // lifecycle bug: reading after close — stop, never retry with the same handle
    throw new IllegalStateException("stream used after close", e);
  }
  throw e;
}

When it happens

Trigger: Calling read()/seek()/readFully() on an FSDataInputStream wrapping OBSInputStream after close(); try-with-resources scope ended but a cached reference is still used; another thread closed the stream (timeout handler, cancellation hook) while a reader continues; finally-block ordering that closes the stream before a last read.

Common situations: Wrapping the stream in a cached reader (BufferedReader, DataInputStream) and reading past the owning try block; async pipelines where a supervisor closes streams on error but workers keep polling; double-managed lifecycle — both framework and user code closing, then user code re-reading.

Related errors

• Stream is closed!
• Filesystem %s closed
• Cannot seek to a negative offset
• Attempted to seek or read past the end of the file
• Requested more bytes than destination buffer size: request l

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4a142a7196c46893. Report an issue: GitHub.