apache/hadoop · error · IOException

Cannot mlock a non-direct ByteBuffer

Error message

Cannot mlock a non-direct ByteBuffer

What it means

NativeIO.mlock(ByteBuffer, long) pins memory with mlock(2), which requires a stable native address; only direct ByteBuffers (ByteBuffer.allocateDirect) have one. Heap buffers live on the movable GC heap, so the method rejects them with IOException before entering JNI. It is a precondition on buffer type, not an environmental failure.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java:477

    }

    static native void mlock_native(
        ByteBuffer buffer, long len) throws NativeIOException;

    /**
     * Locks the provided direct ByteBuffer into memory, preventing it from
     * swapping out. After a buffer is locked, future accesses will not incur
     * a page fault.
     * 
     * See the mlock(2) man page for more information.
     * 
     * @throws NativeIOException
     */
    static void mlock(ByteBuffer buffer, long len)
        throws IOException {
      assertCodeLoaded();
      if (!buffer.isDirect()) {
        throw new IOException("Cannot mlock a non-direct ByteBuffer");
      }
      mlock_native(buffer, len);
    }

    /**
     * Unmaps the block from memory. See munmap(2).
     *
     * There isn't any portable way to unmap a memory region in Java.
     * So we use the sun.nio method here.
     * Note that unmapping a memory region could cause crashes if code
     * continues to reference the unmapped code.  However, if we don't
     * manually unmap the memory, we are dependent on the finalizer to
     * do it, and we have no idea when the finalizer will run.
     *
     * @param buffer    The buffer to unmap.
     */
    public static void munmap(MappedByteBuffer buffer) {
      if (CleanerUtil.UNMAP_SUPPORTED) {

View on GitHub (pinned to 2add963021)

Solutions

1. Allocate with ByteBuffer.allocateDirect(capacity) before calling mlock.
2. Pool direct buffers — they are expensive to allocate, so reuse them instead of converting per call.
3. Enforce directness at the API boundary: check buffer.isDirect() (or assert) as soon as the buffer enters your module.

Example fix

// before
ByteBuffer buf = ByteBuffer.allocate(len);
NativeIO.mlock(buf, len); // IOException: non-direct

// after
ByteBuffer buf = ByteBuffer.allocateDirect(len);
NativeIO.mlock(buf, len);

if (!buffer.isDirect()) {
  throw new IllegalArgumentException("mlock requires a direct ByteBuffer");
}
NativeIO.mlock(buffer, len);

private static ByteBuffer requireDirect(ByteBuffer b) {
  if (b == null || !b.isDirect()) {
    throw new IllegalArgumentException("direct ByteBuffer required");
  }
  return b;
}

When it happens

Trigger: Passing ByteBuffer.allocate(n) or ByteBuffer.wrap(byte[]) to NativeIO.mlock — common when refactoring byte[]-based code to the ByteBuffer API or when a caller upstream chooses the buffer type (HDFS short-circuit / caching code paths that use mlock).

Common situations: Shared-memory and page-pinning features (short-circuit read caches, mlock-based block caching) receiving heap buffers from a refactored caller; unit tests allocating convenient heap buffers.

Related errors

• ${className} does not support positioned reads with byte buf
• ${className} does not support enhanced byte buffer access.
• ${className} does not support release buffer.
• zero-copy reads were not available, and you did not provide
• zero-copy reads were not available, and the ByteBufferPool d

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a28ba515fee21671. Report an issue: GitHub.