{"record":{"id":"4397532c35198414","repo":"prestodb/presto","slug":"generic-spill-failure-439753","errorCode":"GENERIC_SPILL_FAILURE","errorMessage":"Failed to spill pages","messagePattern":"Failed to spill pages","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-main-base/src/main/java/com/facebook/presto/spiller/TempStorageStandaloneSpiller.java","lineNumber":130,"sourceCode":"                if (ioException != exception) {\n                    ioException.addSuppressed(exception);\n                }\n            }\n        }\n        finally {\n            try {\n                if (tempDataSink != null) {\n                    tempDataSink.close();\n                }\n            }\n            catch (IOException e) {\n                if (ioException == null) {\n                    ioException = e;\n                }\n                else if (ioException != e) {\n                    ioException.addSuppressed(e);\n                }\n                throw new PrestoException(GENERIC_SPILL_FAILURE, \"Failed to spill pages\", ioException);\n            }\n        }\n\n        throw new PrestoException(GENERIC_SPILL_FAILURE, \"Failed to spill pages\", ioException);\n    }\n\n    private void flushBufferedPages(TempDataSink tempDataSink, List<DataOutput> bufferedPages)\n    {\n        try {\n            tempDataSink.write(bufferedPages);\n        }\n        catch (UncheckedIOException | IOException e) {\n            throw new PrestoException(GENERIC_SPILL_FAILURE, \"Failed to spill pages\", e);\n        }\n\n        bufferedPages.clear();\n    }\n","sourceCodeStart":112,"sourceCodeEnd":148,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-main-base/src/main/java/com/facebook/presto/spiller/TempStorageStandaloneSpiller.java#L112-L148","documentation":"Thrown by TempStorageStandaloneSpiller.spill when closing the TempDataSink in the finally block throws an IOException (there was no primary write failure recorded, or the close error is chained as suppressed). This is the post-write/close failure path of the stateless spiller: the spill file could not be finalized/closed cleanly, so Presto aborts with GENERIC_SPILL_FAILURE. The sink has been rolled back/closed and the spill is abandoned.","triggerScenarios":"Calling spill(Iterator<Page>) and either the primary write fails AND tempDataSink.close() then also throws (close exception wrapped as cause with the primary as suppressed), or the primary path succeeds partially and close() itself fails (disk flush error, commit-adjacent failure).","commonSituations":"Disk fills mid-spill so both write and close fail; spill files on NFS/ephemeral storage that disappears mid-operation; too many concurrently open spill files exhausting file descriptors; storage backend transient errors under load.","solutions":["Check the server log for the root IOException/suppressed exceptions to see if this is ENOSPC, file-descriptor exhaustion, or a storage backend failure","Free spill-disk space or add additional spiller-spill-path directories; raise the open-file limit (ulimit -n) for the Presto process","Retry the query — this path often involves transient storage errors; enable spiller retry/configured temp storage with retries","Monitor spiller-max-used-space and per-node spill disk utilization to prevent mid-spill failures"],"exampleFix":"// before\nulimit -n 4096\n// after\nulimit -n 65536  # in the systemd unit or launch script for the Presto server","handlingStrategy":"try-catch","validationCode":"// verify file descriptor headroom and spill disk before large spilling queries\nimport java.io.File;\nboolean fdAndDiskOk(String spillPath, long minFreeBytes) {\n    File f = new File(spillPath);\n    return f.isDirectory() && f.canWrite() && f.getUsableSpace() > minFreeBytes;\n}","typeGuard":"boolean isClosePhaseSpillFailure(Throwable t) {\n    return t instanceof com.facebook.presto.spi.PrestoException\n        && ((com.facebook.presto.spi.PrestoException) t).getErrorCode().getName().equals(\"GENERIC_SPILL_FAILURE\")\n        && t.getCause() instanceof java.io.IOException;\n}","tryCatchPattern":"try {\n    SerializedStorageHandle h = standaloneSpiller.spill(pageIterator);\n} catch (PrestoException e) {\n    if (e.getErrorCode().getName().equals(\"GENERIC_SPILL_FAILURE\")) {\n        // cause IOException includes suppressed close errors; log full cause + suppressed\n        // free disk / raise ulimit, then retry the spilling operation\n    } else throw e;\n}","preventionTips":["Raise the worker process open-file limit (ulimit -n) to >65536 for heavy spill workloads","Keep spill disks under ~80% utilization; add multiple spiller-spill-path entries","Avoid NFS/network filesystems for spill; prefer fast local SSDs","Track spillerStats (total spilled bytes, spill failures) for early warning"],"tags":["spill","disk-io","presto","temp-storage"],"backgroundTag":"disk-full-io-error","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}