apache/flink · error · IllegalArgumentException

The given buffer is neither an array-backed heap ByteBuffer,

Error message

The given buffer is neither an array-backed heap ByteBuffer, nor a direct ByteBuffer.

What it means

Thrown as an IllegalArgumentException by DataInputDeserializer.setBuffer(ByteBuffer) when the buffer is neither array-backed (hasArray()==false) nor direct/read-only. This is the else branch after checking hasArray() and (isDirect()||isReadOnly()); it catches ByteBuffer instances that are views or wrappers without an accessible backing array and without the direct or read-only flag — a rare combination that the deserializer cannot efficiently read.

Source

Thrown at flink-core/src/main/java/org/apache/flink/core/memory/DataInputDeserializer.java:80

    // ------------------------------------------------------------------------
    //  Changing buffers
    // ------------------------------------------------------------------------

    public void setBuffer(@Nonnull ByteBuffer buffer) {
        if (buffer.hasArray()) {
            this.buffer = buffer.array();
            this.position = buffer.arrayOffset() + buffer.position();
            this.end = this.position + buffer.remaining();
        } else if (buffer.isDirect() || buffer.isReadOnly()) {
            // TODO: FLINK-8585 handle readonly and other non array based buffers more efficiently
            // without data copy
            this.buffer = new byte[buffer.remaining()];
            this.position = 0;
            this.end = this.buffer.length;

            buffer.get(this.buffer);
        } else {
            throw new IllegalArgumentException(
                    "The given buffer is neither an array-backed heap ByteBuffer, nor a direct ByteBuffer.");
        }
    }

    public void setBuffer(@Nonnull byte[] buffer, int start, int len) {

        if (start < 0 || len < 0 || start + len > buffer.length) {
            throw new IllegalArgumentException("Invalid bounds.");
        }

        setBufferInternal(buffer, start, len);
    }

    public void setBuffer(@Nonnull byte[] buffer) {
        setBufferInternal(buffer, 0, buffer.length);
    }

    private void setBufferInternal(@Nonnull byte[] buffer, int start, int len) {

View on GitHub (pinned to 2f3c205e92)

Solutions

Convert the ByteBuffer to a heap array before passing: copy bytes via buffer.get(byte[]) into a fresh byte[] and use the byte[] constructor.
Ensure the ByteBuffer is direct (ByteBuffer.allocateDirect) or array-backed (ByteBuffer.allocate) if you want zero-copy.
If the buffer is a view, call .duplicate() on an array-backed source or re-allocate.

Example fix

// before
DataInputDeserializer in = new DataInputDeserializer(weirdByteBuffer);

// after — copy to a heap array first
byte[] bytes = new byte[weirdByteBuffer.remaining()];
weirdByteBuffer.get(bytes);
DataInputDeserializer in = new DataInputDeserializer(bytes);

Defensive patterns

Strategy: validation

Validate before calling

if (!buffer.hasArray() && !buffer.isDirect() && !buffer.isReadOnly()) {
    // copy to heap array to satisfy DataInputDeserializer
    byte[] bytes = new byte[buffer.remaining()];
    buffer.get(bytes);
    new DataInputDeserializer(bytes);
}

Type guard

buffer.hasArray() || buffer.isDirect() || buffer.isReadOnly()

Prevention

Prefer ByteBuffer.allocate (array-backed) or ByteBuffer.allocateDirect (direct) when creating buffers for DataInputDeserializer.
Copy non-standard buffers to a heap byte[] before constructing the deserializer.
Avoid passing custom ByteBuffer subclasses without array or direct backing.

When it happens

Trigger: Constructing a DataInputDeserializer or calling setBuffer with a ByteBuffer that is a sliced/duplicated view of a non-direct buffer whose hasArray() returns false, or a custom ByteBuffer subclass not flagged direct or read-only.

Common situations: Passing a ByteBuffer obtained from NIO channel operations or third-party libraries that create non-array-backed, non-direct buffers; custom ByteBuffer implementations; edge cases with memory-mapped buffers on some JVMs.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/dcb1125995c5034f. Report an issue: GitHub.