apache/beam · error · IOException

Channel is closed

Error message

Channel is closed

What it means

HadoopFileSystem's readable ByteChannel throws this IOException on any read() after the channel has been closed. It is a state guard: reading a closed channel is a programming error in the channel lifecycle.

Solutions

1. Stop using the channel after close(); scope it with try-with-resources.
2. Guard concurrent access — close only from the owning thread or after reads complete.
3. Reopen the channel via HadoopFileSystem.open() if further reads are needed.

Example fix

// before
ReadableByteChannel ch = fs.open(src);
ch.close();
ch.read(buf); // throws
// after
try (ReadableByteChannel ch = fs.open(src)) {
  ch.read(buf);
}

if (channel != null && !channel.isOpen()) {
  channel = fileSystem.open(src); // reopen before reading
}

static boolean isReadableChannel(ReadableByteChannel ch) { return ch != null && ch.isOpen(); }

try (ReadableByteChannel ch = fileSystem.open(src)) {
  while (ch.read(buf) != -1) { /* process */ }
} catch (IOException e) {
  if (e.getMessage().equals("Channel is closed")) {
    reopenAndResume();
  }
}

When it happens

Trigger: Calling read(ByteBuffer) on a HadoopFileSystem readable channel after close() has been called, e.g. reading in a finally block after an early close, or reusing a channel object post-close.

Common situations: Double-close in try-with-resources plus manual close; concurrent readers where one thread closes while another still reads; reusing a channel variable across iterations.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Unable to copy resource
• A schema is required to write non-schema'd data.
• A sink must inherit iobase.Sink, iobase.NativeSink, or be a…
• An explicit schema is required to write non-schema'd…
• AUTO is applicable only to reading files

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/64e753f42bf8c4d3. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystem.java:371

    lineage.add(scheme, segments.build(), "/");
  }

  /** An adapter around {@link FSDataInputStream} that implements {@link SeekableByteChannel}. */
  private static class HadoopSeekableByteChannel implements SeekableByteChannel {
    private final FileStatus fileStatus;
    private final FSDataInputStream inputStream;
    private boolean closed;

    private HadoopSeekableByteChannel(FileStatus fileStatus, FSDataInputStream inputStream) {
      this.fileStatus = fileStatus;
      this.inputStream = inputStream;
      this.closed = false;
    }

    @Override
    public int read(ByteBuffer dst) throws IOException {
      if (closed) {
        throw new IOException("Channel is closed");
      }
      // O length read must be supported
      int read = 0;
      // We avoid using the ByteBuffer based read for Hadoop because some FSDataInputStream
      // implementations are not ByteBufferReadable,
      // See https://issues.apache.org/jira/browse/HADOOP-14603
      if (dst.hasArray()) {
        // does the same as inputStream.read(dst):
        // stores up to dst.remaining() bytes into dst.array() starting at dst.position().
        // But dst can have an offset with its backing array hence the + dst.arrayOffset()
        read = inputStream.read(dst.array(), dst.position() + dst.arrayOffset(), dst.remaining());
      } else {
        // TODO: Add support for off heap ByteBuffers in case the underlying FSDataInputStream
        // does not support reading from a ByteBuffer.
        read = inputStream.read(dst);
      }
      if (read > 0) {
        dst.position(dst.position() + read);

View on GitHub (pinned to 12126d8942)