prestodb/presto · error · PrestoException

GENERIC_INSUFFICIENT_RESOURCES

GENERIC_INSUFFICIENT_RESOURCES

Error message

Size of hash table cannot exceed 1 billion entries

What it means

Capacity guard in StreamSummary.rehash (approx_most_frequent stream summary): growing the internal hash table would push it past one billion entries, beyond the implementation's structural limit, so the aggregation aborts.

Source

Thrown at presto-main-base/src/main/java/com/facebook/presto/operator/aggregation/approxmostfrequent/stream/StreamSummary.java:184

            int newBlockPos = newHeapBlockBuilder.getPositionCount();
            StreamDataEntity heapEntry = minHeap.get(heapPosition);
            int oldBlockPosition = getBlockPosition(heapEntry);
            type.appendTo(heapBlockBuilder, oldBlockPosition, newHeapBlockBuilder);
            newBlockPositionToCount.set(newBlockPos, blockPositionToCount.get(oldBlockPosition));
            newBlockToHeapIndex.set(newBlockPos, heapPosition);
            hashToBlockPosition.set(heapEntry.getHashPosition(), newBlockPos);
        }
        blockPositionToCount = newBlockPositionToCount;
        heapBlockBuilder = newHeapBlockBuilder;
        blockToHeapIndex = newBlockToHeapIndex;
        rehash();
    }

    private void rehash()
    {
        long newCapacityLong = hashCapacity * 2L;
        if (newCapacityLong > Integer.MAX_VALUE) {
            throw new PrestoException(GENERIC_INSUFFICIENT_RESOURCES, "Size of hash table cannot exceed 1 billion entries");
        }
        int newCapacity = (int) newCapacityLong;
        int newMask = newCapacity - 1;
        IntBigArray newHashToBlockPosition = new IntBigArray(EMPTY);
        newHashToBlockPosition.ensureCapacity(newCapacity);

        for (int heapPosition = 0; heapPosition < getHeapSize(); heapPosition++) {
            StreamDataEntity heapEntry = minHeap.get(heapPosition);
            int blockPosition = getBlockPosition(heapEntry);
            // find an empty slot for the address
            int hashPosition = getBucketId(TypeUtils.hashPosition(type, heapBlockBuilder, blockPosition), newMask);

            while (newHashToBlockPosition.get(hashPosition) != EMPTY) {
                hashPosition = (hashPosition + 1) & newMask;
            }
            // record the mapping
            newHashToBlockPosition.set(hashPosition, blockPosition);
            heapEntry.setHashPosition(hashPosition);

View on GitHub (pinned to 55bb57d202)

Solutions

1. Increase the capacity argument passed to approx_most_frequent appropriately or reduce input cardinality
2. Filter or pre-aggregate the input stream to fewer distinct keys
3. Use approx_set-based approaches if only heavy hitters matter

When it happens

Trigger: Thrown at presto-main-base/src/main/java/com/facebook/presto/operator/aggregation/approxmostfrequent/stream/StreamSummary.java:184 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/c8069d46dff22d88. Report an issue: GitHub.