apache/hadoop · error · IOException

There is no enough persistent memory space for caching. The

Error message

There is no enough persistent memory space for caching. The current max available space is ${maxAvailableSpace}, but ${bytesCount}is required.

What it means

Error "There is no enough persistent memory space for caching. The current max available space is ${maxAvailableSpace}, but ${bytesCount}is required." thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/PmemVolumeManager.java:414

    if (count == 0) {
      throw new IOException("No usable persistent memory is found");
    }
    int k = 0;
    long maxAvailableSpace = 0L;
    while (k++ != count) {
      if (nextIndex == count) {
        nextIndex = 0;
      }
      byte index = nextIndex++;
      long availableBytes = usedBytesCounts.get(index).getAvailableBytes();
      if (availableBytes >= bytesCount) {
        return index;
      }
      if (availableBytes > maxAvailableSpace) {
        maxAvailableSpace = availableBytes;
      }
    }
    throw new IOException("There is no enough persistent memory space " +
        "for caching. The current max available space is " +
        maxAvailableSpace + ", but " + bytesCount + "is required.");
  }

  @VisibleForTesting
  String getVolumeByIndex(Byte index) {
    return pmemVolumes.get(index);
  }

  ArrayList<String> getVolumes() {
    return pmemVolumes;
  }

  /**
   * A cache file is named after the corresponding BlockId.
   * Thus, cache file name can be inferred according to BlockId.
   */
  public String idToCacheFileName(ExtendedBlockId key) {

View on GitHub (pinned to 2add963021)

Solutions

1. Free space in the pmem cache (evict cached blocks or clear the cache directories) before caching larger blocks.
2. Add more or larger pmem volumes to dfs.datanode.pmem.cache.dirs.
3. Cache smaller blocks or reduce the number of blocks directed to pmem cache.

When it happens

Trigger: Caching a block to persistent memory when the requested byte count exceeds the maximum available pmem space.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9dc222c401f4f2c3. Report an issue: GitHub.