What causes "Stream closed" in apache/hadoop?

Confine the stream to one owner and one lifecycle: try-with-resources exactly around the writing code

apache/hadoop · error · IOException

Stream closed

Error message

Stream closed

What it means

BosOutputStream.write(int) is synchronized and checks the closed flag first; after close() (or the internal abort path) sets it, writing a single byte throws plain IOException('Stream closed') before flush() or buffer allocation run.

Source

Thrown at hadoop-cloud-storage-project/hadoop-bos/src/main/java/org/apache/hadoop/fs/bos/BosOutputStream.java:166

            "catch exception when allocating"
                + " BosBlockBuffer: ",
            throwable);
      }
    }
  }

  /**
   * Writes a single byte to this output stream.
   *
   * @param b the byte to write
   * @throws IOException if the stream is closed or an I/O
   *                     error occurs
   */
  @Override
  public synchronized void write(int b)
      throws IOException {
    if (this.closed) {
      throw new IOException("Stream closed");
    }

    flush();
    createBlockBufferIfNull();

    this.currBlock.getOutBuffer().write(b);
    this.bytesWrittenToBlock++;
    this.filePos++;
  }

  /**
   * Writes bytes from the specified buffer to this output
   * stream.
   *
   * @param b   the data buffer
   * @param off the start offset in the data
   * @param len the number of bytes to write
   * @throws IOException if the stream is closed or an I/O

View on GitHub (pinned to 2add963021)

Solutions

Confine the stream to one owner and one lifecycle: try-with-resources exactly around the writing code
In finally blocks, never write after close — restructure so close() is the last operation, exactly once
Wrap the stream and track a local closed flag if multiple components may close it

Example fix

// before
out.close();
out.write(b); // cleanup writes trailer after close

// after
writeTrailer(out);
out.close(); // close exactly once, last

Defensive patterns

Strategy: validation

Validate before calling

// wrapper that makes post-close writes explicit
if (!wrapperClosed) { out.write(b); } else { throw new IllegalStateException("writer already finished"); }

Type guard

static boolean isClosedWrite(IOException e) {
  return "Stream closed".equals(e.getMessage());
}

Try / catch

catch (IOException e) {
  if (isClosedWrite(e)) {
    out = fs.create(path, overwrite); // start a new object; old stream is finished
  } else { throw e; }
}

Prevention

Close exactly once, in finally, as the last operation
Never write from cleanup paths after close
One owner thread per output stream

When it happens

Trigger: Calling write(int) after close(); two threads sharing the output stream where one closes; writing from a callback/lambda that outlives the stream's scope.

Common situations: Cleanup code in finally that writes a trailer after an earlier close; framework writers (SequenceFile-style) double-closing wrapped streams; error paths that close early then continue appending.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/57c226244b232dc9. Report an issue: GitHub.