apache/hadoop · error · InvalidRequestException

Caching path path of size stats.getBytesNeeded() / replicati

Error message

Caching path path of size stats.getBytesNeeded() / replication bytes at replication replication would exceed pool pool.getPoolName()'s remaining capacity of (pool.getLimit() - pool.getBytesNeeded()) bytes.

What it means

Error "Caching path path of size stats.getBytesNeeded() / replication bytes at replication replication would exceed pool pool.getPoolName()'s remaining capacity of (pool.getLimit() - pool.getBytesNeeded()) bytes." thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CacheManager.java:464

  }

  /**
   * Throws an exception if the CachePool does not have enough capacity to
   * cache the given path at the replication factor.
   *
   * @param pool CachePool where the path is being cached
   * @param path Path that is being cached
   * @param replication Replication factor of the path
   * @throws InvalidRequestException if the pool does not have enough capacity
   */
  private void checkLimit(CachePool pool, String path,
      short replication) throws InvalidRequestException {
    CacheDirectiveStats stats = computeNeeded(path, replication);
    if (pool.getLimit() == CachePoolInfo.LIMIT_UNLIMITED) {
      return;
    }
    if (pool.getBytesNeeded() + stats.getBytesNeeded() > pool.getLimit()) {
      throw new InvalidRequestException("Caching path " + path + " of size "
          + stats.getBytesNeeded() / replication + " bytes at replication "
          + replication + " would exceed pool " + pool.getPoolName()
          + "'s remaining capacity of "
          + (pool.getLimit() - pool.getBytesNeeded()) + " bytes.");
    }
  }

  /**
   * Computes the needed number of bytes and files for a path.
   * @return CacheDirectiveStats describing the needed stats for this path
   */
  private CacheDirectiveStats computeNeeded(String path, short replication) {
    FSDirectory fsDir = namesystem.getFSDirectory();
    INode node;
    long requestedBytes = 0;
    long requestedFiles = 0;
    CacheDirectiveStats.Builder builder = new CacheDirectiveStats.Builder();
    try {

View on GitHub (pinned to 2add963021)

Solutions

1. Increase the cache pool's limit with hdfs cacheadmin -modifyPool -limit, or reduce the path's size/replication so it fits in remaining capacity.
2. Remove other directives from the pool to free capacity.

When it happens

Trigger: addCacheDirective when caching the path at the given replication would exceed the cache pool's remaining byte limit.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/07dd6a960802c641. Report an issue: GitHub.