What causes "%s: Stream is closed!" in apache/hadoop?

Scope every read inside the try-with-resources block that owns the stream.

apache/hadoop · error · IOException

%s: Stream is closed!

Error message

%s: Stream is closed!

What it means

GoogleHadoopFSInputStream.checkNotClosed throws IOException("<path>: Stream is closed!") when any read/seek/available call happens after close() set the volatile closed flag. It is a client-side lifecycle guard - the stream object still exists but refuses further I/O.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleHadoopFSInputStream.java:184

  }

  @Override
  public int available() throws IOException {
    if (!channel.isOpen()) {
      throw new ClosedChannelException();
    }
    return super.available();
  }

  /**
   * Verify that the input stream is open. Non-blocking; this gives the last state of the volatile
   * {@link #closed} field.
   *
   * @throws IOException if the connection is closed.
   */
  private void checkNotClosed() throws IOException {
    if (closed) {
      throw new IOException(gcsPath + ": " + FSExceptionMessages.STREAM_IS_CLOSED);
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

Scope every read inside the try-with-resources block that owns the stream.
If a consumer needs the data later, copy the bytes out (or re-open the stream) instead of reusing a closed one.
Audit wrapper streams so only the outermost owner closes, and no reads occur after that point.

Example fix

// before
FSDataInputStream in = fs.open(path);
in.close();
in.read(buffer); // IOException: <path>: Stream is closed!

// after
try (FSDataInputStream in = fs.open(path)) {
  in.read(buffer); // all reads inside the owning scope
}

Defensive patterns

Strategy: validation

Validate before calling

try (FSDataInputStream in = fs.open(path)) {
  process(in); // every read, seek, available inside this scope
}
// never let `in` escape this block

Try / catch

catch IOException where message endsWith(": Stream is closed!") - re-open the stream (fs.open) rather than retrying on the closed instance.

Prevention

Always open GCS input streams with try-with-resources; keep all reads in scope.
Do not return streams from methods that also close them.
With lazy consumers, copy the data or pass an opener (supplier) instead of the stream itself.

When it happens

Trigger: Calling in.read(...), in.seek(...), or in.available() after in.close(); returning an FSDataInputStream from a method whose try-with-resources already closed it; a lazy consumer (iterator, Spark partition reader) outliving the producer's scope.

Common situations: Streams handed to frameworks that consume them lazily after the producer closed them; double-close followed by read in error-handling paths; wrapped streams where the outer wrapper is read after the inner one was closed.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/db1c635318d84285. Report an issue: GitHub.