prestodb/presto · error · PrestoException

GENERIC_SPILL_FAILURE

GENERIC_SPILL_FAILURE

Error message

Failed to spill pages

What it means

Thrown by TempStorageStandaloneSpiller.spill when closing the TempDataSink in the finally block throws an IOException (there was no primary write failure recorded, or the close error is chained as suppressed). This is the post-write/close failure path of the stateless spiller: the spill file could not be finalized/closed cleanly, so Presto aborts with GENERIC_SPILL_FAILURE. The sink has been rolled back/closed and the spill is abandoned.

Source

Thrown at presto-main-base/src/main/java/com/facebook/presto/spiller/TempStorageStandaloneSpiller.java:130

                if (ioException != exception) {
                    ioException.addSuppressed(exception);
                }
            }
        }
        finally {
            try {
                if (tempDataSink != null) {
                    tempDataSink.close();
                }
            }
            catch (IOException e) {
                if (ioException == null) {
                    ioException = e;
                }
                else if (ioException != e) {
                    ioException.addSuppressed(e);
                }
                throw new PrestoException(GENERIC_SPILL_FAILURE, "Failed to spill pages", ioException);
            }
        }

        throw new PrestoException(GENERIC_SPILL_FAILURE, "Failed to spill pages", ioException);
    }

    private void flushBufferedPages(TempDataSink tempDataSink, List<DataOutput> bufferedPages)
    {
        try {
            tempDataSink.write(bufferedPages);
        }
        catch (UncheckedIOException | IOException e) {
            throw new PrestoException(GENERIC_SPILL_FAILURE, "Failed to spill pages", e);
        }

        bufferedPages.clear();
    }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Check the server log for the root IOException/suppressed exceptions to see if this is ENOSPC, file-descriptor exhaustion, or a storage backend failure
2. Free spill-disk space or add additional spiller-spill-path directories; raise the open-file limit (ulimit -n) for the Presto process
3. Retry the query — this path often involves transient storage errors; enable spiller retry/configured temp storage with retries
4. Monitor spiller-max-used-space and per-node spill disk utilization to prevent mid-spill failures

Example fix

// before
ulimit -n 4096
// after
ulimit -n 65536  # in the systemd unit or launch script for the Presto server

// verify file descriptor headroom and spill disk before large spilling queries
import java.io.File;
boolean fdAndDiskOk(String spillPath, long minFreeBytes) {
    File f = new File(spillPath);
    return f.isDirectory() && f.canWrite() && f.getUsableSpace() > minFreeBytes;
}

boolean isClosePhaseSpillFailure(Throwable t) {
    return t instanceof com.facebook.presto.spi.PrestoException
        && ((com.facebook.presto.spi.PrestoException) t).getErrorCode().getName().equals("GENERIC_SPILL_FAILURE")
        && t.getCause() instanceof java.io.IOException;
}

try {
    SerializedStorageHandle h = standaloneSpiller.spill(pageIterator);
} catch (PrestoException e) {
    if (e.getErrorCode().getName().equals("GENERIC_SPILL_FAILURE")) {
        // cause IOException includes suppressed close errors; log full cause + suppressed
        // free disk / raise ulimit, then retry the spilling operation
    } else throw e;
}

When it happens

Trigger: Calling spill(Iterator<Page>) and either the primary write fails AND tempDataSink.close() then also throws (close exception wrapped as cause with the primary as suppressed), or the primary path succeeds partially and close() itself fails (disk flush error, commit-adjacent failure).

Common situations: Disk fills mid-spill so both write and close fail; spill files on NFS/ephemeral storage that disappears mid-operation; too many concurrently open spill files exhausting file descriptors; storage backend transient errors under load.

Related errors

• GENERIC_INTERNAL_ERROR
• spill path %s is not writable; adjust experimental.spiller-s
• OUT_OF_SPILL_SPACE
• could not close some single stream spillers
• could not create spill path %s; adjust experimental.spiller-

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/4397532c35198414. Report an issue: GitHub.