prestodb/presto · error · PrestoException

GENERIC_INSUFFICIENT_RESOURCES

GENERIC_INSUFFICIENT_RESOURCES

Error message

Size of hash table cannot exceed 1 billion entries

What it means

Capacity guard in SetOfValues.rehash: the hash set backing a set aggregation would exceed one billion entries while growing, exceeding the implementation's addressing limit; the aggregation fails rather than corrupt state.

Source

Thrown at presto-main-base/src/main/java/com/facebook/presto/operator/aggregation/SetOfValues.java:144

    private boolean isContainedAt(Block firstBlock, int positionWithinFirstBlock, Block secondBlock, int positionWithinSecondBlock)
    {
        boolean firstValueNull = firstBlock.isNull(positionWithinFirstBlock);
        Object firstValue = firstValueNull ? defaultValue(valueType.getJavaType()) : readNativeValue(valueType, firstBlock, positionWithinFirstBlock);
        boolean secondValueNull = secondBlock.isNull(positionWithinSecondBlock);
        Object secondValue = secondValueNull ? defaultValue(valueType.getJavaType()) : readNativeValue(valueType, secondBlock, positionWithinSecondBlock);
        try {
            return !(boolean) elementIsDistinctFrom.invoke(firstValue, firstValueNull, secondValue, secondValueNull);
        }
        catch (Throwable t) {
            throw internalError(t);
        }
    }

    private void rehash()
    {
        long newCapacityLong = hashCapacity * 2L;
        if (newCapacityLong > Integer.MAX_VALUE) {
            throw new PrestoException(GENERIC_INSUFFICIENT_RESOURCES, "Size of hash table cannot exceed 1 billion entries");
        }
        int newCapacity = (int) newCapacityLong;
        hashCapacity = newCapacity;
        hashMask = newCapacity - 1;
        maxFill = calculateMaxFill(newCapacity);
        valuePositionByHash = new int[newCapacity];
        Arrays.fill(valuePositionByHash, EMPTY_SLOT);
        for (int position = 0; position < valueBlockBuilder.getPositionCount(); position++) {
            valuePositionByHash[getHashPositionOfValue(valueBlockBuilder, position)] = position;
        }
    }

    private static int calculateMaxFill(int hashSize)
    {
        checkArgument(hashSize > 0, "hashSize must be greater than 0");
        int maxFill = (int) Math.ceil(hashSize * FILL_RATIO);
        if (maxFill == hashSize) {
            maxFill--;

View on GitHub (pinned to 55bb57d202)

Solutions

1. Reduce distinct-value cardinality with filters or pre-aggregation
2. Use approx_set or a sketch function for very large distinct sets
3. Increase memory and accept that the one-billion-entry structural limit still applies

When it happens

Trigger: Thrown at presto-main-base/src/main/java/com/facebook/presto/operator/aggregation/SetOfValues.java:144 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/f5d50f0860d58a56. Report an issue: GitHub.