apache/hadoop · error · IOException

Stream is closed!

Error message

Stream is closed!

What it means

AbfsOutputStream.write(byte[], int, int) checks the closed flag first and throws IOException("Stream is closed!") for any write attempted after close(). close() itself is idempotent (a second close returns immediately), but a post-close write is always rejected before argument validation or lease checks run.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java:450

  public void write(final int byteVal) throws IOException {
    write(new byte[]{(byte) (byteVal & 0xFF)});
  }

  /**
   * Writes length bytes from the specified byte array starting at off to
   * this output stream.
   *
   * @param data   the byte array to write.
   * @param off the start off in the data.
   * @param length the number of bytes to write.
   * @throws IOException if an I/O error occurs. In particular, an IOException may be
   *                     thrown if the output stream has been closed.
   */
  @Override
  public synchronized void write(final byte[] data, final int off, final int length)
      throws IOException {
    if (closed) {
      throw new IOException(FSExceptionMessages.STREAM_IS_CLOSED);
    }
    // validate if data is not null and index out of bounds.
    DataBlocks.validateWriteArgs(data, off, length);
    maybeThrowLastError();

    if (off < 0 || length < 0 || length > data.length - off) {
      throw new IndexOutOfBoundsException();
    }

    if (hasLease() && isLeaseFreed()) {
      throw new PathIOException(path, ERR_WRITE_WITHOUT_LEASE);
    }
    if (length == 0) {
      LOG.debug("No data to write, length is 0 for path: {}", path);
      return;
    }

    AbfsBlock block = createBlockIfNeeded(position);

View on GitHub (pinned to 2add963021)

Solutions

1. Wrap the output stream so close() is the terminal operation — try-with-resources with no writes after the block
2. In multi-writer setups, coordinate: signal writers to stop BEFORE closing the stream
3. If data must be appended after close, open a new output stream in append mode (fs.append)

Example fix

// before
out.close();
out.write(trailer);   // IOException: Stream is closed!

// after
out.write(trailer);
out.close();

// or append later
try (FSDataOutputStream out2 = fs.append(path)) {
  out2.write(trailer);
}

// Own the lifecycle: no writes after close, ever
public class SafeWriter implements Closeable {
  private final FSDataOutputStream out;
  private boolean closed;
  SafeWriter(FSDataOutputStream out) { this.out = out; }
  public synchronized void write(byte[] b, int off, int len) throws IOException {
    if (closed) throw new IllegalStateException("writer already closed");
    out.write(b, off, len);
  }
  public synchronized void close() throws IOException {
    if (!closed) { closed = true; out.close(); }
  }
}

try {
  out.write(buf, 0, len);
} catch (IOException e) {
  if (FSExceptionMessages.STREAM_IS_CLOSED.equals(e.getMessage())) {
    // writer bug: data written after close — reopen in append mode if needed
  } else throw e;
}

When it happens

Trigger: Writing to an FSDataOutputStream after its close()/try-with-resources ended; producer threads still enqueuing data while the consumer thread closed the stream on error; framework retry logic that writes a trailer after the finally block closed the stream.

Common situations: Multi-threaded writers where one thread's failure closes the stream and another thread's write then fails; commit/trailer hooks running after close; buffered wrapper classes flushing on close after the inner ABFS stream was closed first.

Related errors

• Executor Service closed before writes could be completed.
• mark()/reset() not supported on this stream
• Lease desired but no lease threads configured, set fs.azure.
• - account name is not fully qualified.
• Exception while initializing metric credentials

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3144d409ca5d85e8. Report an issue: GitHub.