{"record":{"id":"9bdc58627bf6f699","repo":"MyCATApache/Mycat-Server","slug":"error-while-calling-spill-on-spiller-reas","errorCode":null,"errorMessage":"error while calling spill() on ${spiller} : ${reason}","messagePattern":"error while calling spill\\(\\) on (.+?) : (.+?)","errorType":"exception","errorClass":"OutOfMemoryError","httpStatus":null,"severity":"critical","filePath":"src/main/java/io/mycat/memory/unsafe/memory/mm/DataNodeMemoryManager.java","lineNumber":148,"sourceCode":"        // Call spill() on other consumers to release memory\n        for (MemoryConsumer c: consumers) {\n          if (c != consumer && c.getUsed() > 0) {\n            try {\n               /**\n               * 调用spill函数，写数据到磁盘中\n               */\n              long released = c.spill(required - got, consumer);\n                if (released > 0 && mode == tungstenMemoryMode) {\n                logger.info(\"Thread \"+connectionAttemptId+\" released \"+ JavaUtils.bytesToString(released) +\n                        \" from \"+ c +\" for\" + consumer);\n                got += memoryManager.acquireExecutionMemory(required - got, connectionAttemptId, mode);\n                if (got >= required) {\n                  break;\n                }\n              }\n            } catch (IOException e) {\n              logger.error(\"error while calling spill() on \" + c, e);\n              throw new OutOfMemoryError(\"error while calling spill() on \" + c + \" : \"\n                + e.getMessage());\n            }\n          }\n        }\n      }\n\n      // call spill() on itself\n      if (got < required && consumer != null) {\n        try {\n          long released = consumer.spill(required - got, consumer);\n          if (released > 0 && mode == tungstenMemoryMode) {\n            logger.info(\"Thread \" + connectionAttemptId +\n                    \" released \"+   JavaUtils.bytesToString(released) +\"from itself (\"+consumer+ \")\");\n            got += memoryManager.acquireExecutionMemory(required - got, connectionAttemptId, mode);\n          }\n        } catch (IOException e) {\n          logger.error(\"error while calling spill() on \" + consumer, e);\n          throw new OutOfMemoryError(\"error while calling spill() on \" + consumer + \" : \"","sourceCodeStart":130,"sourceCodeEnd":166,"githubUrl":"https://github.com/MyCATApache/Mycat-Server/blob/65f8d8beb752f935752f2a0eec0ab017facab9ef/src/main/java/io/mycat/memory/unsafe/memory/mm/DataNodeMemoryManager.java#L130-L166","documentation":"During acquireExecutionMemory, DataNodeMemoryManager asks Spillable consumers to spill() to disk to free execution memory. If a consumer's spill() throws IOException, the manager logs the failure and throws OutOfMemoryError, because without spilling it cannot satisfy the memory request.","triggerScenarios":"Calling acquireExecutionMemory (directly or via allocatePage/allocateLongArray on a MemoryConsumer) when execution memory is exhausted and one of the registered spillable consumers throws IOException from its spill() method (e.g. disk full, I/O error, bad temp directory).","commonSituations":"Disk full or disk failure on the spill directory; temp directory not writable or removed at runtime; a misconfigured spark-style local dir; large sorts/aggregations/joins exceeding available execution memory so spilling is required.","solutions":["Check the wrapped IOException in the log ('error while calling spill() on <consumer>') to find the root cause (disk full, permission, path).","Free disk space on the spill directories and verify they are writable by the process.","Point the spill/local directories at a healthy, larger disk via configuration.","Reduce per-task memory pressure (smaller batches, fewer concurrent consumers) or increase execution memory so spilling is not required."],"exampleFix":"// before (spill target on a full/unwritable disk)\nconf.set(\"mycat.local.dir\", \"/var/tmp\");\n// after\n// point spill dirs at a writable disk with free space\nconf.set(\"mycat.local.dir\", \"/mnt/bigdisk/spill\");","handlingStrategy":"try-catch","validationCode":"// before requesting memory, verify spill dirs are usable\nfor (String dir : spillDirs) {\n  File d = new File(dir);\n  if (!d.canWrite() || d.getUsableSpace() < minSpillBytes)\n    throw new IllegalStateException(\"spill dir unusable: \" + dir);\n}","typeGuard":null,"tryCatchPattern":"try {\n  long got = memoryManager.acquireExecutionMemory(required, ctx, mode);\n} catch (OutOfMemoryError e) {\n  // inspect cause/logging for the spill IOException; shrink workload or fail task cleanly\n  logger.error(\"execution memory unavailable after spill failure\", e);\n  throw new TaskKilledException(e);\n}","preventionTips":["Monitor free space on spill/local directories.","Keep spill dirs on reliable, non-full disks with writable permissions.","Limit concurrent consumers to the memory pool.","Reduce per-task memory demand so spilling is rare."],"tags":["java","memory","spill","out-of-memory","io"],"backgroundTag":"out-of-memory","analyzedSha":"65f8d8beb752f935752f2a0eec0ab017facab9ef","analyzedAt":"2026-09-11T00:12:21.696Z","contentChangedAt":"2026-09-11T00:12:21.696Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}