prestodb/presto · error · IllegalArgumentException

Non-direct byte buffer backed by byte array required

Error message

Non-direct byte buffer backed by byte array required

What it means

The Zstd compressor's ByteBuffer overload requires both input and output buffers to be non-direct heap buffers backed by an accessible Java array. It throws IllegalArgumentException when either buffer is direct (off-heap) or lacks a backing array, because it reads/writes via arrayOffset()+position().

Source

Thrown at presto-spark-base/src/main/java/com/facebook/presto/spark/util/PrestoSparkUtils.java:174

            {
                return toIntExact(Zstd.compressBound(uncompressedSize));
            }

            @Override
            public int compress(byte[] input, int inputOffset, int inputLength, byte[] output, int outputOffset, int maxOutputLength)
            {
                long size = Zstd.compressByteArray(output, outputOffset, maxOutputLength, input, inputOffset, inputLength, COMPRESSION_LEVEL);
                if (Zstd.isError(size)) {
                    throw new RuntimeException(Zstd.getErrorName(size));
                }
                return toIntExact(size);
            }

            @Override
            public void compress(ByteBuffer input, ByteBuffer output)
            {
                if (input.isDirect() || output.isDirect() || !input.hasArray() || !output.hasArray()) {
                    throw new IllegalArgumentException("Non-direct byte buffer backed by byte array required");
                }
                int inputOffset = input.arrayOffset() + input.position();
                int outputOffset = output.arrayOffset() + output.position();

                int written = compress(input.array(), inputOffset, input.remaining(), output.array(), outputOffset, output.remaining());
                ((Buffer) output).position(output.position() + written);
            }
        };
    }

    private static PageDecompressor createPageDecompressor()
    {
        // based on ZstdJniDecompressor
        return new PageDecompressor()
        {
            @Override
            public int decompress(byte[] input, int inputOffset, int inputLength, byte[] output, int outputOffset, int maxOutputLength)
            {

View on GitHub (pinned to 55bb57d202)

Solutions

Ensure both ByteBuffers are created with ByteBuffer.allocate (heap, non-direct)
Call hasArray() to verify before compressing; copy direct buffers into heap buffers first
Pass raw byte arrays using the compress(byte[],...) overload instead of ByteBuffers

Example fix

// before
ByteBuffer input = ByteBuffer.allocateDirect(1024);
compressor.compress(input, output);
// after
ByteBuffer input = ByteBuffer.allocate(1024);
compressor.compress(input, output);

Defensive patterns

Strategy: validation

Validate before calling

if (input.isDirect() || output.isDirect() || !input.hasArray() || !output.hasArray()) { /* copy to heap buffers before compressing */ }

Try / catch

try { compressor.compress(input, output); } catch (IllegalArgumentException e) { ByteBuffer heapIn = toHeap(input); ByteBuffer heapOut = toHeap(output); compressor.compress(heapIn, heapOut); }

Prevention

Always allocate compression buffers with ByteBuffer.allocate, not allocateDirect
Check hasArray() before invoking buffer-based compress
Prefer the byte[] overloads when working with off-heap data

When it happens

Trigger: Calling PrestoSparkUtils compress with a direct ByteBuffer (allocateDirect) or a read-only/sliced buffer without a backing array for input or output.

Common situations: Passing Netty/off-heap buffers or buffers obtained from memory-mapped IO into the compression API; wrapping code that switched from allocate() to allocateDirect(); sliced buffers from larger heap buffers that lost their accessible array.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/c6b4b3039373ba21. Report an issue: GitHub.