{"record":{"id":"28c1403dc61e134b","repo":"prestodb/presto","slug":"memory-limit-exceeded","errorCode":"MEMORY_LIMIT_EXCEEDED","errorMessage":"Memory limit [%d] for memory connector exceeded","messagePattern":"Memory limit \\[(.+?)\\] for memory connector exceeded","errorType":"error_code","errorClass":"PrestoException","httpStatus":500,"severity":"error","filePath":"presto-memory/src/main/java/com/facebook/presto/plugin/memory/MemoryPagesStore.java","lineNumber":69,"sourceCode":"\n    public synchronized void initialize(long tableId)\n    {\n        if (!tables.containsKey(tableId)) {\n            tables.put(tableId, new TableData());\n        }\n    }\n\n    public synchronized void add(Long tableId, Page page)\n    {\n        if (!contains(tableId)) {\n            throw new PrestoException(MISSING_DATA, \"Failed to find table on a worker.\");\n        }\n\n        page.compact();\n\n        long newSize = currentBytes + page.getRetainedSizeInBytes();\n        if (maxBytes < newSize) {\n            throw new PrestoException(MEMORY_LIMIT_EXCEEDED, format(\"Memory limit [%d] for memory connector exceeded\", maxBytes));\n        }\n        currentBytes = newSize;\n\n        TableData tableData = tables.get(tableId);\n        tableData.add(page);\n    }\n\n    public synchronized List<Page> getPages(\n            Long tableId,\n            int partNumber,\n            int totalParts,\n            List<Integer> columnIndexes,\n            long expectedRows)\n    {\n        if (!contains(tableId)) {\n            throw new PrestoException(MISSING_DATA, \"Failed to find table on a worker.\");\n        }\n        TableData tableData = tables.get(tableId);","sourceCodeStart":51,"sourceCodeEnd":87,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-memory/src/main/java/com/facebook/presto/plugin/memory/MemoryPagesStore.java#L51-L87","documentation":"MemoryPagesStore.add throws MEMORY_LIMIT_EXCEEDED when adding a page would push currentBytes past maxBytes, the memory connector's per-store cap (memory.max-data-per-node / configured max bytes). The memory connector is in-memory only; it never spills, so exceeding the budget aborts the write.","triggerScenarios":"INSERT/CREATE TABLE AS into a memory table whose accumulated retained size plus the new page exceeds maxBytes on that worker.","commonSituations":"Loading large datasets into the memory connector for fast testing; underestimated row sizes (wide/complex types); concurrent inserts filling the same worker's store; max-bytes-per-node configured too low.","solutions":["Increase the connector's maxBytes (memory.max-data-per-node in catalog properties) and/or worker heap.","Load only a subset of the data, or switch to a spilling connector (Hive/Iceberg) for large datasets.","Page.compact() already trims retained size — reduce wide column usage (e.g. large VARCHAR/map/array) in the memory table.","Ensure no leaked/abandoned tables are holding memory; restart the worker or drop unused memory tables to reclaim space."],"exampleFix":"// catalog/memory.properties\n// before\nmemory.max-data-per-node=1GB\n// after\nmemory.max-data-per-node=16GB","handlingStrategy":"validation","validationCode":"if (estimatedRetainedBytes + currentUsageBytes > maxBytesPerNode) throw new IllegalArgumentException(\"Dataset too large for memory connector\");","typeGuard":null,"tryCatchPattern":"try { insert(...) } catch (PrestoException e) { if (MEMORY_LIMIT_EXCEEDED.equals(e.getErrorCode())) { /* switch to Hive/Iceberg or raise limit */ } else throw e; }","preventionTips":["Size memory.max-data-per-node to your largest expected table plus headroom","Use the memory connector only for small test datasets","Monitor currentBytes via connector stats/JMX"],"tags":["presto","memory-connector","memory-limit","resource-exhaustion"],"backgroundTag":"memory-limit-exceeded","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}