prestodb/presto · error · PrestoException

GENERIC_INSUFFICIENT_RESOURCES

GENERIC_INSUFFICIENT_RESOURCES

Error message

Size of hash table cannot exceed 1 billion entries

What it means

Capacity guard in KeyValuePairs.rehash: the open-addressed hash table backing an aggregation (e.g. map keys or distinct values) would need to exceed one billion entries after doubling, which Presto refuses to allocate; the aggregation aborts.

Source

Thrown at presto-main-base/src/main/java/com/facebook/presto/operator/aggregation/KeyValuePairs.java:161

    private int getHashPositionOfKey(Block key, int position)
    {
        int hashPosition = getMaskedHash(hashPosition(keyType, key, position));
        while (true) {
            if (keyPositionByHash[hashPosition] == EMPTY_SLOT) {
                return hashPosition;
            }
            else if (positionEqualsPosition(keyType, keyBlockBuilder, keyPositionByHash[hashPosition], key, position)) {
                return hashPosition;
            }
            hashPosition = getMaskedHash(hashPosition + 1);
        }
    }

    private void rehash()
    {
        long newCapacityLong = hashCapacity * 2L;
        if (newCapacityLong > Integer.MAX_VALUE) {
            throw new PrestoException(GENERIC_INSUFFICIENT_RESOURCES, "Size of hash table cannot exceed 1 billion entries");
        }
        int newCapacity = (int) newCapacityLong;
        hashCapacity = newCapacity;
        hashMask = newCapacity - 1;
        maxFill = calculateMaxFill(newCapacity);
        keyPositionByHash = new int[newCapacity];
        Arrays.fill(keyPositionByHash, EMPTY_SLOT);
        for (int position = 0; position < keyBlockBuilder.getPositionCount(); position++) {
            keyPositionByHash[getHashPositionOfKey(keyBlockBuilder, position)] = position;
        }
    }

    private static int calculateMaxFill(int hashSize)
    {
        checkArgument(hashSize > 0, "hashSize must be greater than 0");
        int maxFill = (int) Math.ceil(hashSize * FILL_RATIO);
        if (maxFill == hashSize) {
            maxFill--;

View on GitHub (pinned to 55bb57d202)

Solutions

1. Reduce input cardinality by pre-aggregating or filtering the data
2. Use approx_set or other sketch-based aggregations for very high cardinality
3. Split the query or increase cluster memory so grouping can complete within limits

When it happens

Trigger: Thrown at presto-main-base/src/main/java/com/facebook/presto/operator/aggregation/KeyValuePairs.java:161 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/fcc26be4a0cacb51. Report an issue: GitHub.