apache/hadoop · error · UnsupportedOperationException

zero-copy reads were not available, and you did not provide

Error message

zero-copy reads were not available, and you did not provide a fallback ByteBufferPool.

What it means

FSDataInputStream.read(ByteBufferPool, int, EnumSet) first tries the inner stream as HasEnhancedByteBufferAccess (zero-copy, implemented by DFSInputStream); on ClassCastException it falls back to ByteBufferUtil.fallbackRead, which requires a non-null ByteBufferPool and otherwise throws UnsupportedOperationException with this message. So the caller used the ByteBuffer read API on a stream without zero-copy support and supplied no pool to allocate the fallback buffer.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ByteBufferUtil.java:61

    }
    return ((FSDataInputStream)stream).getWrappedStream() 
        instanceof ByteBufferReadable;
  }

  /**
   * Perform a fallback read.
   *
   * @param stream input stream.
   * @param bufferPool bufferPool.
   * @param maxLength maxLength.
   * @throws IOException raised on errors performing I/O.
   * @return byte buffer.
   */
  public static ByteBuffer fallbackRead(
      InputStream stream, ByteBufferPool bufferPool, int maxLength)
          throws IOException {
    if (bufferPool == null) {
      throw new UnsupportedOperationException("zero-copy reads " +
          "were not available, and you did not provide a fallback " +
          "ByteBufferPool.");
    }
    boolean useDirect = streamHasByteBufferRead(stream);
    ByteBuffer buffer = bufferPool.getBuffer(useDirect, maxLength);
    if (buffer == null) {
      throw new UnsupportedOperationException("zero-copy reads " +
          "were not available, and the ByteBufferPool did not provide " +
          "us with " + (useDirect ? "a direct" : "an indirect") +
          "buffer.");
    }
    Preconditions.checkState(buffer.capacity() > 0);
    Preconditions.checkState(buffer.isDirect() == useDirect);
    maxLength = Math.min(maxLength, buffer.capacity());
    boolean success = false;
    try {
      if (useDirect) {
        buffer.clear();

View on GitHub (pinned to 2add963021)

Solutions

1. Always pass a real pool, e.g. new org.apache.hadoop.io.ElasticByteBufferPool(), which allocates on demand.
2. Or use the portable byte[] path (read(byte[]) / readFully) when the filesystem may not support zero-copy.
3. Feature-detect zero-copy via getWrappedStream() instanceof HasEnhancedByteBufferAccess before choosing the API.
4. On HDFS, read from the unwrapped DFSInputStream if you specifically need zero-copy semantics.

Example fix

// before
ByteBuffer buf = in.read(null, 4096); // no zero-copy on this stream -> UOE

// after
ByteBuffer buf = in.read(new ElasticByteBufferPool(), 4096);

static ByteBuffer readPortable(FSDataInputStream in, int maxLen)
    throws IOException {
  ByteBufferPool pool = new org.apache.hadoop.io.ElasticByteBufferPool(); // never null-dependant
  try {
    return in.read(pool, maxLen);
  } catch (UnsupportedOperationException e) {
    byte[] b = new byte[maxLen];
    int n = in.read(b); // classic fallback path
    return n <= 0 ? null : ByteBuffer.wrap(b, 0, n);
  }
}

static boolean supportsZeroCopy(FSDataInputStream in) {
  return in.getWrappedStream() instanceof org.apache.hadoop.fs.HasEnhancedByteBufferAccess;
}

try {
  buf = in.read(pool, maxLength);
} catch (UnsupportedOperationException e) {
  // zero-copy unavailable and no usable fallback: switch to byte[] reads
  buf = null;
}

When it happens

Trigger: fsin.read(null, maxLength) or read(null, maxLength, opts) on local, s3a, or any stream not implementing HasEnhancedByteBufferAccess; HDFS opened through a wrapping/filter stream that hides the DFSInputStream interface; code written against HDFS zero-copy reused on another filesystem.

Common situations: Applications that call read(pool, len) with null because DFSInputStream tolerated it on HDFS; parquet/erasure-coding style readers wanting ByteBuffers, run against local FS in tests or object stores in production.

Related errors

• zero-copy reads were not available, and the ByteBufferPool d
• ${className} does not support enhanced byte buffer access.
• ${className} does not support release buffer.
• {} doesn't support listXAttrs
• {} doesn't support removeXAttr

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3f284f5ea7f9dbbb. Report an issue: GitHub.