apache/hadoop · error · IOException

Exceeded the configured number of objects {} in the filesyst

Error message

Exceeded the configured number of objects {} in the filesystem.

What it means

checkFsObjectLimit throws when total inodes plus blocks has reached dfs.namenode.max.objects (maxFsObjects; 0 means unlimited). This caps namespace growth, so file creation, mkdir, and new block allocation fail once the ceiling is hit.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:5507

    if (isPermissionEnabled) {
      FSPermissionChecker pc = getPermissionChecker();
      pc.checkSuperuserPrivilege(null);
    }
  }

  void checkSuperuserPrivilege(String operationName)
      throws IOException {
    checkSuperuserPrivilege(operationName, null);
  }

  /**
   * Check to see if we have exceeded the limit on the number
   * of inodes.
   */
  void checkFsObjectLimit() throws IOException {
    if (maxFsObjects != 0 &&
        maxFsObjects <= dir.totalInodes() + getBlocksTotal()) {
      throw new IOException("Exceeded the configured number of objects " +
                             maxFsObjects + " in the filesystem.");
    }
  }

  @Override // FSNamesystemMBean
  public long getMaxObjects() {
    return maxFsObjects;
  }

  @Override // FSNamesystemMBean
  @Metric
  public long getFilesTotal() {
    // There is no need to take fSNamesystem's lock as
    // FSDirectory has its own lock.
    return this.dir.totalInodes();
  }

  /**

View on GitHub (pinned to 2add963021)

Solutions

Raise dfs.namenode.max.objects (or set 0 to disable) in hdfs-site.xml and restart the NameNode
Reduce object count: compact small files (HAR, sequence files, ORC), delete stale data
Increase dfs.blocksize so fewer blocks represent the same bytes
Monitor filesTotal/blocksTotal JMX metrics against the configured cap

Example fix

<!-- before -->
<property><name>dfs.namenode.max.objects</name><value>1000000</value></property>

<!-- after -->
<property><name>dfs.namenode.max.objects</name><value>0</value></property> <!-- unlimited, or a larger ceiling -->

Defensive patterns

Strategy: validation

Validate before calling

// via JMX or ClientProtocol.getMBean-backed metrics before mass creates
long maxObjects = dfsAdmin.getMetrics().getLong("MaxObjects");
long used = filesTotal + blocksTotal;
if (maxObjects > 0 && used >= maxObjects * 0.9) {
  alertOrThrottle("Approaching dfs.namenode.max.objects");
}

Try / catch

try {
  fs.create(path);
} catch (IOException e) {
  if (e.getMessage().contains("Exceeded the configured number of objects")) {
    // quota ceiling: stop creating, compact files, or raise max.objects
  }
}

Prevention

Set dfs.namenode.max.objects to 0 or a realistic ceiling for production
Compact small files (HAR/ORC/sequence files) instead of accumulating millions
Watch filesTotal and blocksTotal trends against the cap

When it happens

Trigger: Any create/mkdir/addBlock once dir.totalInodes() + getBlocksTotal() >= maxFsObjects, typically with a small configured cap or a many-small-files workload.

Common situations: A low dfs.namenode.max.objects set for testing left in production; millions of small files; small block size inflating block count faster than inode count.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0af85ec6216c6104. Report an issue: GitHub.