apache/hadoop · error · IOException

Stream is closed!

Error message

Stream is closed!

What it means

DFSStripedInputStream.seek(long) checks an AtomicBoolean 'closed' flag after validating the target position; if the striped input stream has already been closed, it throws IOException('Stream is closed!'). There is no underlying stream left to reposition — the socket and stripe buffers are released by close() — so the only correct response is to open a new stream.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java:363

    dfsClient.updateFileSystemReadStats(stats.getNetworkDistance(),
        stats.getBytesRead(), readTimeMS);
    assert readStatistics.getBlockType() == BlockType.STRIPED;
    dfsClient.updateFileSystemECReadStats(stats.getBytesRead());
  }

  /**
   * Seek to a new arbitrary location.
   */
  @Override
  public synchronized void seek(long targetPos) throws IOException {
    if (targetPos > getFileLength()) {
      throw new EOFException("Cannot seek after EOF");
    }
    if (targetPos < 0) {
      throw new EOFException("Cannot seek to negative offset");
    }
    if (closed.get()) {
      throw new IOException("Stream is closed!");
    }
    if (targetPos <= blockEnd) {
      final long targetOffsetInBlk = getOffsetInBlockGroup(targetPos);
      if (curStripeRange.include(targetOffsetInBlk)) {
        int bufOffset = getStripedBufOffset(targetOffsetInBlk);
        curStripeBuf.position(bufOffset);
        pos = targetPos;
        return;
      }
    }
    pos = targetPos;
    blockEnd = -1;
  }

  private int getStripedBufOffset(long offsetInBlockGroup) {
    final long stripeLen = cellSize * dataBlkNum;
    // compute the position in the curStripeBuf based on "pos"
    return (int) (offsetInBlockGroup % stripeLen);

View on GitHub (pinned to 2add963021)

Solutions

1. Open a fresh stream with FileSystem.open(path) and seek it to the desired position instead of reusing the closed instance.
2. Make stream ownership explicit: exactly one owner, try-with-resources at the outermost user, nothing outside the block.
3. If you cache readers, evict them on close so no caller can obtain a dead stream.
4. Where late use is legitimate, catch the IOException around seek() and reopen the file transparently.

Example fix

// before
try (FSDataInputStream in = fs.open(path)) {
  process(in);
in.seek(offset); // stream already closed by try-with-resources

// after
try (FSDataInputStream in = fs.open(path)) {
  process(in);
  in.seek(offset);
  readAt(in, offset);
}

DFSInputStream internal = in.getWrappedStream() instanceof DFSInputStream
    ? (DFSInputStream) in.getWrappedStream() : null;
// note: isClosed() is package-private; track closure in YOUR code instead
if (streamClosedFlag.get()) {
  in = fs.open(path);
  in.seek(targetPos);
} else {
  in.seek(targetPos);
}

try {
  in.seek(targetPos);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("closed")) {
    try (FSDataInputStream fresh = fs.open(path)) {
      fresh.seek(targetPos);
      return readFrom(fresh);
    }
  }
  throw e;
}

When it happens

Trigger: Calling seek() on an FSDataInputStream for an erasure-coded file after close() ran: double-close patterns, seeking in a finally block after an earlier failure closed the stream, or using a stream outside the try-with-resources block that owns it.

Common situations: Streams stored in caches/pools and closed by one component while another still seeks; read-retry helpers that close on error and then try to reposition the same instance; large refactors where close() moved into a utility method but callers kept using the stream afterwards.

Related errors

• Stream closed
• key + ": Stream is closed!"
• Cannot seek to negative offset
• Not support enhanced byte buffer access.
• Stream closed

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/fff2e89afd9ed9fd. Report an issue: GitHub.