prestodb/presto · error · PrestoException

GENERIC_INSUFFICIENT_RESOURCES

GENERIC_INSUFFICIENT_RESOURCES

Error message

Size of hash table cannot exceed 1 billion entries

What it means

This is an internal guard in JsonUtil's open-addressing hash table used during JSON-to-row casting. When the table's rehash would exceed Integer.MAX_VALUE capacity (about 1 billion entries after the cap), it throws PrestoException GENERIC_INSUFFICIENT_RESOURCES. Hitting it means an absurdly large number of distinct JSON field names is being processed, exhausting memory resources.

Source

Thrown at presto-main-base/src/main/java/com/facebook/presto/util/JsonUtil.java:1473

        private int getHashPosition(int position)
        {
            int hashPosition = getMaskedHash(hashPosition(type, block, position));
            while (true) {
                if (positionByHash[hashPosition] == EMPTY_SLOT) {
                    return hashPosition;
                }
                else if (positionEqualsPosition(type, block, positionByHash[hashPosition], block, position)) {
                    return hashPosition;
                }
                hashPosition = getMaskedHash(hashPosition + 1);
            }
        }

        private void rehash()
        {
            long newCapacityLong = hashCapacity * 2L;
            if (newCapacityLong > Integer.MAX_VALUE) {
                throw new PrestoException(GENERIC_INSUFFICIENT_RESOURCES, "Size of hash table cannot exceed 1 billion entries");
            }
            int newCapacity = (int) newCapacityLong;
            hashCapacity = newCapacity;
            hashMask = newCapacity - 1;
            maxFill = calculateMaxFill(newCapacity);
            int[] oldPositionByHash = positionByHash;
            positionByHash = new int[newCapacity];
            Arrays.fill(positionByHash, EMPTY_SLOT);
            for (int position : oldPositionByHash) {
                if (position != EMPTY_SLOT) {
                    positionByHash[getHashPosition(position)] = position;
                }
            }
        }

        private static int calculateMaxFill(int hashSize)
        {
            checkArgument(hashSize > 0, "hashSize must be greater than 0");

View on GitHub (pinned to 55bb57d202)

Solutions

Reduce the size/key-cardinality of the JSON input before casting
Split the payload and cast in smaller chunks
Limit payload size at ingestion (query max-parsed-tokens/max expression size limits)
Do not attempt to cast such JSON to fixed row types; process with a streaming/ETL tool instead

Defensive patterns

Strategy: validation

Validate before calling

// Bound payload cardinality before casting
if (json.length() > MAX_JSON_BYTES || countDistinctTopLevelKeys(json) > MAX_KEYS) {
    throw new IllegalArgumentException("JSON payload too large for row cast");
}

Try / catch

try {
    row = castJsonToRow(hugeJson, rowType);
} catch (PrestoException e) {
    if (GENERIC_INSUFFICIENT_RESOURCES.equals(e.getErrorCode())) {
        reject(e); // do not retry; input is pathological
    } else throw e;
}

Prevention

Enforce payload size limits at ingestion
Never cast unbounded/user-supplied JSON to row types directly
Monitor for anomalous key cardinality in incoming JSON
Do not retry this error; it is deterministic for the input

When it happens

Trigger: Casting JSON with an extreme number of distinct field names (hundreds of millions+) to a row/map type, causing the internal field-name hash table to grow past the capacity limit.

Common situations: Pathological or malicious inputs (giant JSON payloads with millions of unique keys); runaway data generation bugs feeding enormous JSON documents into casts.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/4bb1cac26b2eb075. Report an issue: GitHub.