apache/hadoop · error · NSQuotaExceededException

The NameSpace quota (directories and files) of directory pat

Error message

The NameSpace quota (directories and files) of directory pathName is exceeded: quota=quota file count=count

What it means

NSQuotaExceededException thrown from DirectoryWithQuotaFeature.verifyNamespaceQuota(), which the NameNode calls (via FSDirectory.verifyQuota) before applying any write that adds names to a directory tree. Quota.isViolated(quota, usage, delta) is true when a namespace quota is set (quota >= 0), delta > 0, and usage.getNameSpace() + delta would exceed quota.getNameSpace(). Namespace quota counts files AND directories (each contributes 1) and is set with 'hdfs dfsadmin -setQuota N path'.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/DirectoryWithQuotaFeature.java:177

    usage.setNameSpace(c.getNameSpace());
    usage.setStorageSpace(c.getStorageSpace());
    usage.setTypeSpaces(c.getTypeSpaces());
  }

  /** @return the namespace and storagespace and typespace allowed. */
  public QuotaCounts getSpaceAllowed() {
    return new QuotaCounts.Builder().quotaCount(quota).build();
  }

  /** @return the namespace and storagespace and typespace consumed. */
  public QuotaCounts getSpaceConsumed() {
    return new QuotaCounts.Builder().quotaCount(usage).build();
  }

  /** Verify if the namespace quota is violated after applying delta. */
  private void verifyNamespaceQuota(long delta) throws NSQuotaExceededException {
    if (Quota.isViolated(quota.getNameSpace(), usage.getNameSpace(), delta)) {
      throw new NSQuotaExceededException(quota.getNameSpace(),
          usage.getNameSpace() + delta);
    }
  }
  /** Verify if the storagespace quota is violated after applying delta. */
  private void verifyStoragespaceQuota(long delta) throws DSQuotaExceededException {
    if (Quota.isViolated(quota.getStorageSpace(), usage.getStorageSpace(), delta)) {
      throw new DSQuotaExceededException(quota.getStorageSpace(),
          usage.getStorageSpace() + delta);
    }
  }

  private void verifyQuotaByStorageType(EnumCounters<StorageType> typeDelta)
      throws QuotaByStorageTypeExceededException {
    if (!isQuotaByStorageTypeSet()) {
      return;
    }
    for (StorageType t: StorageType.getTypesSupportingQuota()) {
      if (!isQuotaByStorageTypeSet(t)) {

View on GitHub (pinned to 2add963021)

Solutions

1. Raise or clear the quota: hdfs dfsadmin -setQuota <bigger-N> <path>, or hdfs dfsadmin -clrQuota <path>.
2. Reduce name count: delete stale files/partitions, or compact many small files into fewer larger ones (Har/compaction) under the quota'd tree.
3. Check current usage first: hdfs dfs -count -q <path> (QUOTA, REMAINING_QUOTA columns) to size the new limit.
4. For applications: catch QuotaExceededException, shed input or spill to an unquota'd path.

Example fix

# before
hdfs dfsadmin -setQuota 1000 /data/ingest   # ingest writes millions of small files
# after
hdfs dfs -count -q /data/ingest             # see QUOTA vs REMAINING_QUOTA
hdfs dfsadmin -setQuota 2000000 /data/ingest

ContentSummary cs = dfs.getContentSummary(dir);
long namesUsed = cs.getFileCount() + cs.getDirectoryCount();
long delta = filesToAdd + dirsToAdd;             // each file/dir counts 1
if (cs.getQuota() >= 0 && namesUsed + delta > cs.getQuota()) {
  throw new IllegalStateException("namespace quota " + cs.getQuota() + " would be exceeded");
}

static boolean isNamespaceQuotaExceeded(IOException e) {
  return e instanceof NSQuotaExceededException
      || (e instanceof RemoteException re
          && re.getClassName().endsWith("NSQuotaExceededException"));
}

try {
  fs.create(dst);
} catch (RemoteException re) {
  if (re.getClassName().endsWith("QuotaExceededException")) {
    // read hdfs dfs -count -q, raise dfsadmin -setQuota or shed files, then retry
  } else { throw re; }
}

When it happens

Trigger: create(), mkdir(), symlink creation, or rename() of a subtree into a directory whose nearest quota'd ancestor has name usage + delta over the limit; verifyNamespaceQuota is invoked with counts.getNameSpace() from the proposed change and throws before the inode is added.

Common situations: Massive small-file jobs (every file and each partition dir counts 1) under a quota set for byte-scale thinking; bulk ingest or Hive partition explosion; renaming a large subtree into a quota'd area; usage already near the limit with concurrent writers racing the check.

Related errors

• The DiskSpace quota of pathName is exceeded: quota = quota B
• Exceeded the configured number of objects {} in the filesyst
• Directory does not exist: {}
• {}
• File does not exist: {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/2a3cee78a52a08a2. Report an issue: GitHub.