prestodb/presto · error · SliceTooLargeException

Cannot allocate slice larger than %d bytes

Error message

Cannot allocate slice larger than %d bytes

What it means

BlockUtil.checkValidSliceRange guards against allocating a Slice larger than MAX_ARRAY_SIZE when extracting length bytes starting at sourceIndex, throwing SliceTooLargeException because sourceIndex+length can overflow int and JVM arrays are capped near Integer.MAX_VALUE. It prevents oversized or overflowed allocation requests from reaching the VM.

Source

Thrown at presto-common/src/main/java/com/facebook/presto/common/block/BlockUtil.java:83

    static void checkValidPositions(boolean[] positions, int positionCount)
    {
        if (positions.length != positionCount) {
            throw new IllegalArgumentException(format("Invalid positions array size %d, actual position count is %d", positions.length, positionCount));
        }
    }

    static void checkValidPosition(int position, int positionCount)
    {
        if (position < 0 || position >= positionCount) {
            throw new IllegalArgumentException(format("Invalid position %s in block with %s positions", position, positionCount));
        }
    }

    static void checkValidSliceRange(int sourceIndex, int length)
    {
        //sourceIndex + length can overflow integer range
        if (sourceIndex > MAX_ARRAY_SIZE - length) {
            throw new SliceTooLargeException(format("Cannot allocate slice larger than %d bytes", MAX_ARRAY_SIZE));
        }
    }

    static int calculateNewArraySize(int currentSize)
    {
        // grow array by 50%
        long newSize = (long) currentSize + (currentSize >> 1);

        // verify new size is within reasonable bounds
        if (newSize < DEFAULT_CAPACITY) {
            newSize = DEFAULT_CAPACITY;
        }
        else if (newSize > MAX_ARRAY_SIZE) {
            newSize = MAX_ARRAY_SIZE;
            if (newSize == currentSize) {
                throw new IllegalArgumentException(format("Can not grow array beyond '%s'", MAX_ARRAY_SIZE));
            }
        }

View on GitHub (pinned to 55bb57d202)

Solutions

  1. Validate length against MAX_ARRAY_SIZE (and remaining source bytes) before requesting the slice
  2. Fix the offset/length table so final offsets never exceed the underlying slice size
  3. Catch SliceTooLargeException and fail the operation with a data-corruption/too-large error instead of retrying

Example fix

// before
Slice value = slice.getBytes(0, declaredLength);
// after
checkState(declaredLength >= 0 && declaredLength <= slice.length(), "invalid length");
Slice value = slice.getBytes(0, Math.min(declaredLength, slice.length()));
Defensive patterns

Strategy: try-catch

Validate before calling

if (length < 0 || sourceIndex > BlockUtil.MAX_ARRAY_SIZE - length) {
    throw new PrestoException(GENERIC_INTERNAL_ERROR, "slice too large: " + length + " bytes at " + sourceIndex);
}

Type guard

boolean isValidSliceLength(long length) {
    return length >= 0 && length <= MAX_ARRAY_SIZE;
}

Try / catch

try {
    return checkValidSliceRange(sourceIndex, length);
} catch (SliceTooLargeException e) {
    throw new PrestoException(GENERIC_INTERNAL_ERROR, "requested slice exceeds " + MAX_ARRAY_SIZE + " bytes", e);
}

Prevention

When it happens

Trigger: Requesting a slice/copy whose sourceIndex + length exceeds MAX_ARRAY_SIZE (or overflows int), typically from an unvalidated length field in serialized data or a bad variable-width column offset computation.

Common situations: Deserializing corrupt or hostile serialized blocks where declared lengths are huge; buggy VARCHAR/VARBINARY offset tables whose last offset exceeds the slice capacity; accumulating unbounded concatenations.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/413a9ff07c6be7f8. Report an issue: GitHub.