{"record":{"id":"c8833159bab1a2b0","repo":"prestodb/presto","slug":"storage-error","errorCode":"STORAGE_ERROR","errorMessage":"Unable to delete broadcast spill file","messagePattern":"Unable to delete broadcast spill file","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"warning","filePath":"presto-spark-base/src/main/java/com/facebook/presto/spark/PrestoSparkStorageBasedBroadcastDependency.java","lineNumber":137,"sourceCode":"    }\n\n    @Override\n    public void destroy()\n    {\n        if (broadcastVariable == null) {\n            return;\n        }\n\n        try {\n            // Delete the files\n            for (PrestoSparkStorageHandle diskPage : broadcastVariable.getValue()) {\n                TempStorageHandle storageHandle = tempStorage.deserialize(diskPage.getSerializedStorageHandle());\n                tempStorage.remove(tempDataOperationContext, storageHandle);\n                log.info(\"Deleted broadcast spill file: \" + storageHandle.toString());\n            }\n        }\n        catch (IOException e) {\n            throw new PrestoException(STORAGE_ERROR, \"Unable to delete broadcast spill file\", e);\n        }\n\n        // Destroy broadcast variable\n        broadcastVariable.destroy();\n    }\n}\n","sourceCodeStart":119,"sourceCodeEnd":144,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-spark-base/src/main/java/com/facebook/presto/spark/PrestoSparkStorageBasedBroadcastDependency.java#L119-L144","documentation":"Thrown from PrestoSparkStorageBasedBroadcastDependency.destroy() when an IOException occurs while removing a broadcast spill file from the configured TempStorage. The broadcast table was spilled to temporary storage; during destruction each disk page's storage handle is deserialized and its file removed. Failure to delete indicates the underlying temp storage is unhealthy or the file is already gone.","triggerScenarios":"Calling destroy() on a storage-based broadcast dependency when tempStorage.remove() fails due to an IOException — e.g. the spill file was already deleted, the storage backend is unreachable, or permissions are wrong.","commonSituations":"HDFS/S3/local-disk temp storage cleanup failures at query teardown; external cleanup jobs deleting spill files while the query is still running; misconfigured temp storage credentials or disk-full conditions on workers.","solutions":["Check the wrapped IOException cause to identify whether the file is missing, access is denied, or the storage backend is down","Verify the TempStorage configuration (base directory, credentials, connectivity) on all Spark workers","Ensure no external retention/cleanup jobs delete temp storage files while queries are active","Retry the query; cleanup failure usually does not affect results, only leaks the spill file"],"exampleFix":"// before: destroy aborts on missing file\nTempStorageHandle storageHandle = tempStorage.deserialize(diskPage.getSerializedStorageHandle());\ntempStorage.remove(tempDataOperationContext, storageHandle);\n// after: tolerate already-deleted files by pre-checking/catching in storage layer (or handle at cleanup job level)\nif (tempStorage.exists(tempDataOperationContext, storageHandle)) {\n    tempStorage.remove(tempDataOperationContext, storageHandle);\n}","handlingStrategy":"try-catch","validationCode":"// verify temp storage is reachable before the query runs\nTempStorage ts = ...;\n// attempt a probe write/read/delete\nts.create(new TempDataOperationContext(...), OptionalLong.empty());","typeGuard":"null","tryCatchPattern":"try {\n    dependency.destroy();\n}\ncatch (PrestoException e) {\n    if (STORAGE_ERROR.toErrorCode().getCode().equals(e.getErrorCode().getCode())) {\n        log.warn(e, \"broadcast spill cleanup failed; file may leak\");\n        // do not fail the query for cleanup issues\n    } else { throw e; }\n}","preventionTips":["Do not run external cleanup jobs against temp storage during query lifetime","Monitor temp storage backend health and capacity","Keep TempStorage configuration consistent across driver and executors","Accept that cleanup errors are warnings: results are intact, files may leak"],"tags":["io","temp-storage","spill","cleanup","spark"],"backgroundTag":"temp-storage-cleanup-failure","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}