{"record":{"id":"8194851ed50edc8d","repo":"MyCATApache/Mycat-Server","slug":"error-while-calling-spill-on-consumer-rea","errorCode":null,"errorMessage":"error while calling spill() on ${consumer} : ${reason}","messagePattern":"error while calling spill\\(\\) on (.+?) : (.+?)","errorType":"exception","errorClass":"OutOfMemoryError","httpStatus":null,"severity":"critical","filePath":"src/main/java/io/mycat/memory/unsafe/memory/mm/DataNodeMemoryManager.java","lineNumber":166,"sourceCode":"              throw new OutOfMemoryError(\"error while calling spill() on \" + c + \" : \"\n                + e.getMessage());\n            }\n          }\n        }\n      }\n\n      // call spill() on itself\n      if (got < required && consumer != null) {\n        try {\n          long released = consumer.spill(required - got, consumer);\n          if (released > 0 && mode == tungstenMemoryMode) {\n            logger.info(\"Thread \" + connectionAttemptId +\n                    \" released \"+   JavaUtils.bytesToString(released) +\"from itself (\"+consumer+ \")\");\n            got += memoryManager.acquireExecutionMemory(required - got, connectionAttemptId, mode);\n          }\n        } catch (IOException e) {\n          logger.error(\"error while calling spill() on \" + consumer, e);\n          throw new OutOfMemoryError(\"error while calling spill() on \" + consumer + \" : \"\n            + e.getMessage());\n\n        }\n      }\n\n      if (consumer != null) {\n        consumers.add(consumer);\n      }\n     // logger.info(\"Thread\" + connectionAttemptId + \" acquire \"+  JavaUtils.bytesToString(got) +\" for \"+ consumer+\"\");\n      return got;\n    }\n  }\n\n  /**\n   * Release N bytes of execution memory for a MemoryConsumer.\n   */\n  public void releaseExecutionMemory(long size, MemoryMode mode, MemoryConsumer consumer) {\n    logger.debug   (\"Thread\" + connectionAttemptId + \" release \"+  JavaUtils.bytesToString(size) +\" from \"+ consumer+\"\");","sourceCodeStart":148,"sourceCodeEnd":184,"githubUrl":"https://github.com/MyCATApache/Mycat-Server/blob/65f8d8beb752f935752f2a0eec0ab017facab9ef/src/main/java/io/mycat/memory/unsafe/memory/mm/DataNodeMemoryManager.java#L148-L184","documentation":"In acquireExecutionMemory, when the requested memory cannot be obtained directly, DataNodeMemoryManager calls the consumer's own spill() to free space. If that spill() call throws IOException, the manager wraps it into an OutOfMemoryError stating that spill failed on that consumer, since memory cannot be freed.","triggerScenarios":"A MemoryConsumer requests execution memory (acquireExecutionMemory or allocatePage/allocateArray on it) while under memory pressure, and the consumer's own spill() implementation throws IOException.","commonSituations":"Sorters/aggregators spilling to a full, failed, or unwritable disk; temp files deleted by a cleaner job mid-run; heavy concurrent workloads forcing frequent spills on marginal storage.","solutions":["Inspect the chained IOException cause in logs to identify the spill I/O failure (ENOSPC, EACCES, missing dir).","Free space and ensure write permissions on the spill directory.","Reconfigure spill directories to reliable storage with sufficient free space.","Reduce memory demand (lower concurrency, smaller page sizes/batches) or raise the execution memory limit."],"exampleFix":"// before\nlong got = memoryManager.acquireExecutionMemory(required, taskContext, mode);\n// after\n// guard memory pressure and spill health beforehand\nif (diskUsableBytes(spillDir) < minSpillSpace) {\n  throw new IllegalStateException(\"spill dir too small: \" + spillDir);\n}\nlong got = memoryManager.acquireExecutionMemory(required, taskContext, mode);","handlingStrategy":"try-catch","validationCode":"// ensure the consumer's spill target is healthy before acquiring\nif (!spillDir.canWrite() || spillDir.getUsableSpace() < minSpillBytes) {\n  throw new IllegalStateException(\"cannot spill to \" + spillDir);\n}","typeGuard":null,"tryCatchPattern":"try {\n  long got = memoryManager.acquireExecutionMemory(required, attemptId, mode);\n} catch (OutOfMemoryError e) {\n  // consumer spill() threw IOException; log and retry with less demand or kill task\n  logger.error(\"spill failed while acquiring execution memory\", e);\n  throw new TaskKilledException(e);\n}","preventionTips":["Implement consumer.spill() defensively: check disk space and permissions first.","Monitor spill-directory health and free space.","Back off / retry with reduced memory demand on failure.","Avoid placing temp/spill dirs on ephemeral or cleaned paths."],"tags":["java","memory","spill","out-of-memory","io"],"backgroundTag":"out-of-memory","analyzedSha":"65f8d8beb752f935752f2a0eec0ab017facab9ef","analyzedAt":"2026-09-11T00:12:21.696Z","contentChangedAt":"2026-09-11T00:12:21.696Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}