apache/hadoop · error · MemoryLimitException

Can not allocate memory, memory used {memoryUsed}, allocate

Error message

Can not allocate memory, memory used {memoryUsed}, allocate size {size}, memory limit {memoryLimit}

What it means

Error "Can not allocate memory, memory used {memoryUsed}, allocate size {size}, memory limit {memoryLimit}" thrown in apache/hadoop.

Source

Thrown at hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/OSSDataBlocks.java:462

    void setMemoryLimit(long memoryLimit) {
      this.memoryLimit = memoryLimit;
      if (memoryLimit > 0) {
        checkMemory = true;
      }
    }

    void allocateMemory(long size) throws MemoryLimitException {
      if (!checkMemory) {
        return;
      }
      long next = memoryUsed.addAndGet(size);
      if (next > memoryLimit) {
        memoryUsed.getAndAdd(-size);
        String msg = "Can not allocate memory"
                + ", memory used " + memoryUsed
                + ", allocate size " + size
                + ", memory limit " + memoryLimit;
        throw new MemoryLimitException(msg);
      }
    }

    void releaseMemory(long size) {
      if (!checkMemory) {
        return;
      }
      memoryUsed.getAndAdd(-size);
    }

    long getMemoryUsed() {
      return memoryUsed.get();
    }
  }

  /**
   * Use byte arrays on the heap for storage.
   */

View on GitHub (pinned to 2add963021)

Solutions

1. Increase fs.oss.data.blocks.buffer.memory.limit or reduce the number of active upload streams so buffered memory stays under the limit.

When it happens

Trigger: Thrown at hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/OSSDataBlocks.java:462 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8e18ace2054cd535. Report an issue: GitHub.