prestodb/presto · error · PrestoException

MEMORY_LIMIT_EXCEEDED

MEMORY_LIMIT_EXCEEDED

Error message

Memory limit [%d] for memory connector exceeded

What it means

MemoryPagesStore.add throws MEMORY_LIMIT_EXCEEDED when adding a page would push currentBytes past maxBytes, the memory connector's per-store cap (memory.max-data-per-node / configured max bytes). The memory connector is in-memory only; it never spills, so exceeding the budget aborts the write.

Source

Thrown at presto-memory/src/main/java/com/facebook/presto/plugin/memory/MemoryPagesStore.java:69

    public synchronized void initialize(long tableId)
    {
        if (!tables.containsKey(tableId)) {
            tables.put(tableId, new TableData());
        }
    }

    public synchronized void add(Long tableId, Page page)
    {
        if (!contains(tableId)) {
            throw new PrestoException(MISSING_DATA, "Failed to find table on a worker.");
        }

        page.compact();

        long newSize = currentBytes + page.getRetainedSizeInBytes();
        if (maxBytes < newSize) {
            throw new PrestoException(MEMORY_LIMIT_EXCEEDED, format("Memory limit [%d] for memory connector exceeded", maxBytes));
        }
        currentBytes = newSize;

        TableData tableData = tables.get(tableId);
        tableData.add(page);
    }

    public synchronized List<Page> getPages(
            Long tableId,
            int partNumber,
            int totalParts,
            List<Integer> columnIndexes,
            long expectedRows)
    {
        if (!contains(tableId)) {
            throw new PrestoException(MISSING_DATA, "Failed to find table on a worker.");
        }
        TableData tableData = tables.get(tableId);

View on GitHub (pinned to 55bb57d202)

Solutions

  1. Increase the connector's maxBytes (memory.max-data-per-node in catalog properties) and/or worker heap.
  2. Load only a subset of the data, or switch to a spilling connector (Hive/Iceberg) for large datasets.
  3. Page.compact() already trims retained size — reduce wide column usage (e.g. large VARCHAR/map/array) in the memory table.
  4. Ensure no leaked/abandoned tables are holding memory; restart the worker or drop unused memory tables to reclaim space.

Example fix

// catalog/memory.properties
// before
memory.max-data-per-node=1GB
// after
memory.max-data-per-node=16GB
Defensive patterns

Strategy: validation

Validate before calling

if (estimatedRetainedBytes + currentUsageBytes > maxBytesPerNode) throw new IllegalArgumentException("Dataset too large for memory connector");

Try / catch

try { insert(...) } catch (PrestoException e) { if (MEMORY_LIMIT_EXCEEDED.equals(e.getErrorCode())) { /* switch to Hive/Iceberg or raise limit */ } else throw e; }

Prevention

When it happens

Trigger: INSERT/CREATE TABLE AS into a memory table whose accumulated retained size plus the new page exceeds maxBytes on that worker.

Common situations: Loading large datasets into the memory connector for fast testing; underestimated row sizes (wide/complex types); concurrent inserts filling the same worker's store; max-bytes-per-node configured too low.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/28c1403dc61e134b. Report an issue: GitHub.