apache/hadoop · error · IOException

Stream closed

Error message

Stream closed

What it means

The single-byte read() checks the wrapper's closed flag, which close() sets before releasing resources; any read after close throws IOException("Stream closed"). The check is purely on the wrapper — the underlying FTP data stream is already gone at that point.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FTPInputStream.java:73

  public long getPos() throws IOException {
    return pos;
  }

  // We don't support seek.
  @Override
  public void seek(long pos) throws IOException {
    throw new IOException("Seek not supported");
  }

  @Override
  public boolean seekToNewSource(long targetPos) throws IOException {
    throw new IOException("Seek not supported");
  }

  @Override
  public synchronized int read() throws IOException {
    if (closed) {
      throw new IOException("Stream closed");
    }

    int byteRead = wrappedStream.read();
    if (byteRead >= 0) {
      pos++;
    }
    if (stats != null && byteRead >= 0) {
      stats.incrementBytesRead(1);
    }
    return byteRead;
  }

  @Override
  public synchronized int read(byte buf[], int off, int len) throws IOException {
    if (closed) {
      throw new IOException("Stream closed");
    }

View on GitHub (pinned to 2add963021)

Solutions

  1. Move every read inside the stream's lifecycle scope (try-with-resources block)
  2. Enforce single ownership: one reader per stream; pass Paths between components, not open streams
  3. For shared streams, synchronize close with readers (or use an external closed flag) so close waits for the read loop
  4. Reopen a fresh stream from the FileSystem if data must be read again

Example fix

// before
FSDataInputStream in = fs.open(path);
in.close();
int b = in.read();          // IOException: Stream closed

// after
try (FSDataInputStream in = fs.open(path)) {
  int b = in.read();        // all reads inside the scope
}
Defensive patterns

Strategy: try-catch

Try / catch

catch IOException from read() and treat "Stream closed" as a lifecycle bug in your code (not a data error) — locate the premature close rather than retrying the read.

Prevention

When it happens

Trigger: read() after close() in the same thread (finally-block ordering bug); a second consumer reading a stream the first consumer closed; a watchdog/cleanup thread closing streams still being drained by a reader thread.

Common situations: try-with-resources scope mistakes where parsing happens after the try block; frameworks closing input on task cancellation while a reader loop still drains; sharing one open stream across map tasks or threads.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a1687147ae7de3a8. Report an issue: GitHub.