{"record":{"id":"4bb1cac26b2eb075","repo":"prestodb/presto","slug":"generic-insufficient-resources-4bb1ca","errorCode":"GENERIC_INSUFFICIENT_RESOURCES","errorMessage":"Size of hash table cannot exceed 1 billion entries","messagePattern":"Size of hash table cannot exceed 1 billion entries","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-main-base/src/main/java/com/facebook/presto/util/JsonUtil.java","lineNumber":1473,"sourceCode":"        private int getHashPosition(int position)\n        {\n            int hashPosition = getMaskedHash(hashPosition(type, block, position));\n            while (true) {\n                if (positionByHash[hashPosition] == EMPTY_SLOT) {\n                    return hashPosition;\n                }\n                else if (positionEqualsPosition(type, block, positionByHash[hashPosition], block, position)) {\n                    return hashPosition;\n                }\n                hashPosition = getMaskedHash(hashPosition + 1);\n            }\n        }\n\n        private void rehash()\n        {\n            long newCapacityLong = hashCapacity * 2L;\n            if (newCapacityLong > Integer.MAX_VALUE) {\n                throw new PrestoException(GENERIC_INSUFFICIENT_RESOURCES, \"Size of hash table cannot exceed 1 billion entries\");\n            }\n            int newCapacity = (int) newCapacityLong;\n            hashCapacity = newCapacity;\n            hashMask = newCapacity - 1;\n            maxFill = calculateMaxFill(newCapacity);\n            int[] oldPositionByHash = positionByHash;\n            positionByHash = new int[newCapacity];\n            Arrays.fill(positionByHash, EMPTY_SLOT);\n            for (int position : oldPositionByHash) {\n                if (position != EMPTY_SLOT) {\n                    positionByHash[getHashPosition(position)] = position;\n                }\n            }\n        }\n\n        private static int calculateMaxFill(int hashSize)\n        {\n            checkArgument(hashSize > 0, \"hashSize must be greater than 0\");","sourceCodeStart":1455,"sourceCodeEnd":1491,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-main-base/src/main/java/com/facebook/presto/util/JsonUtil.java#L1455-L1491","documentation":"This is an internal guard in JsonUtil's open-addressing hash table used during JSON-to-row casting. When the table's rehash would exceed Integer.MAX_VALUE capacity (about 1 billion entries after the cap), it throws PrestoException GENERIC_INSUFFICIENT_RESOURCES. Hitting it means an absurdly large number of distinct JSON field names is being processed, exhausting memory resources.","triggerScenarios":"Casting JSON with an extreme number of distinct field names (hundreds of millions+) to a row/map type, causing the internal field-name hash table to grow past the capacity limit.","commonSituations":"Pathological or malicious inputs (giant JSON payloads with millions of unique keys); runaway data generation bugs feeding enormous JSON documents into casts.","solutions":["Reduce the size/key-cardinality of the JSON input before casting","Split the payload and cast in smaller chunks","Limit payload size at ingestion (query max-parsed-tokens/max expression size limits)","Do not attempt to cast such JSON to fixed row types; process with a streaming/ETL tool instead"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Bound payload cardinality before casting\nif (json.length() > MAX_JSON_BYTES || countDistinctTopLevelKeys(json) > MAX_KEYS) {\n    throw new IllegalArgumentException(\"JSON payload too large for row cast\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    row = castJsonToRow(hugeJson, rowType);\n} catch (PrestoException e) {\n    if (GENERIC_INSUFFICIENT_RESOURCES.equals(e.getErrorCode())) {\n        reject(e); // do not retry; input is pathological\n    } else throw e;\n}","preventionTips":["Enforce payload size limits at ingestion","Never cast unbounded/user-supplied JSON to row types directly","Monitor for anomalous key cardinality in incoming JSON","Do not retry this error; it is deterministic for the input"],"tags":["resources","memory","json","limits"],"backgroundTag":"resource-limit-exceeded","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}