{"record":{"id":"3f284f5ea7f9dbbb","repo":"apache/hadoop","slug":"zero-copy-reads-were-not-available-and-you-did-no","errorCode":null,"errorMessage":"zero-copy reads were not available, and you did not provide a fallback ByteBufferPool.","messagePattern":"zero-copy reads were not available, and you did not provide a fallback ByteBufferPool\\.","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ByteBufferUtil.java","lineNumber":61,"sourceCode":"    }\n    return ((FSDataInputStream)stream).getWrappedStream() \n        instanceof ByteBufferReadable;\n  }\n\n  /**\n   * Perform a fallback read.\n   *\n   * @param stream input stream.\n   * @param bufferPool bufferPool.\n   * @param maxLength maxLength.\n   * @throws IOException raised on errors performing I/O.\n   * @return byte buffer.\n   */\n  public static ByteBuffer fallbackRead(\n      InputStream stream, ByteBufferPool bufferPool, int maxLength)\n          throws IOException {\n    if (bufferPool == null) {\n      throw new UnsupportedOperationException(\"zero-copy reads \" +\n          \"were not available, and you did not provide a fallback \" +\n          \"ByteBufferPool.\");\n    }\n    boolean useDirect = streamHasByteBufferRead(stream);\n    ByteBuffer buffer = bufferPool.getBuffer(useDirect, maxLength);\n    if (buffer == null) {\n      throw new UnsupportedOperationException(\"zero-copy reads \" +\n          \"were not available, and the ByteBufferPool did not provide \" +\n          \"us with \" + (useDirect ? \"a direct\" : \"an indirect\") +\n          \"buffer.\");\n    }\n    Preconditions.checkState(buffer.capacity() > 0);\n    Preconditions.checkState(buffer.isDirect() == useDirect);\n    maxLength = Math.min(maxLength, buffer.capacity());\n    boolean success = false;\n    try {\n      if (useDirect) {\n        buffer.clear();","sourceCodeStart":43,"sourceCodeEnd":79,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ByteBufferUtil.java#L43-L79","documentation":"FSDataInputStream.read(ByteBufferPool, int, EnumSet) first tries the inner stream as HasEnhancedByteBufferAccess (zero-copy, implemented by DFSInputStream); on ClassCastException it falls back to ByteBufferUtil.fallbackRead, which requires a non-null ByteBufferPool and otherwise throws UnsupportedOperationException with this message. So the caller used the ByteBuffer read API on a stream without zero-copy support and supplied no pool to allocate the fallback buffer.","triggerScenarios":"fsin.read(null, maxLength) or read(null, maxLength, opts) on local, s3a, or any stream not implementing HasEnhancedByteBufferAccess; HDFS opened through a wrapping/filter stream that hides the DFSInputStream interface; code written against HDFS zero-copy reused on another filesystem.","commonSituations":"Applications that call read(pool, len) with null because DFSInputStream tolerated it on HDFS; parquet/erasure-coding style readers wanting ByteBuffers, run against local FS in tests or object stores in production.","solutions":["Always pass a real pool, e.g. new org.apache.hadoop.io.ElasticByteBufferPool(), which allocates on demand.","Or use the portable byte[] path (read(byte[]) / readFully) when the filesystem may not support zero-copy.","Feature-detect zero-copy via getWrappedStream() instanceof HasEnhancedByteBufferAccess before choosing the API.","On HDFS, read from the unwrapped DFSInputStream if you specifically need zero-copy semantics."],"exampleFix":"// before\nByteBuffer buf = in.read(null, 4096); // no zero-copy on this stream -> UOE\n\n// after\nByteBuffer buf = in.read(new ElasticByteBufferPool(), 4096);","handlingStrategy":"fallback","validationCode":"static ByteBuffer readPortable(FSDataInputStream in, int maxLen)\n    throws IOException {\n  ByteBufferPool pool = new org.apache.hadoop.io.ElasticByteBufferPool(); // never null-dependant\n  try {\n    return in.read(pool, maxLen);\n  } catch (UnsupportedOperationException e) {\n    byte[] b = new byte[maxLen];\n    int n = in.read(b); // classic fallback path\n    return n <= 0 ? null : ByteBuffer.wrap(b, 0, n);\n  }\n}","typeGuard":"static boolean supportsZeroCopy(FSDataInputStream in) {\n  return in.getWrappedStream() instanceof org.apache.hadoop.fs.HasEnhancedByteBufferAccess;\n}","tryCatchPattern":"try {\n  buf = in.read(pool, maxLength);\n} catch (UnsupportedOperationException e) {\n  // zero-copy unavailable and no usable fallback: switch to byte[] reads\n  buf = null;\n}","preventionTips":["Never pass a null ByteBufferPool to read(ByteBufferPool, int)","Feature-detect with getWrappedStream() instanceof HasEnhancedByteBufferAccess before choosing the API","Default to the byte[] read path when the filesystem is configurable"],"tags":["hadoop","filesystem","zero-copy","bytebuffer","unsupportedoperationexception"],"backgroundTag":"zero-copy-read-unavailable","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}