apache/hadoop · error · HadoopIllegalArgumentException

"Cannot find a block policy with the name " + policyName

Error message

"Cannot find a block policy with the name " + policyName

What it means

setStoragePolicy first resolves the policy name through the NameNode's BlockStoragePolicySuite; an unknown name returns null and FSDirAttrOp throws HadoopIllegalArgumentException. Names are case-sensitive and limited to the suite the NN loaded - the built-ins are HOT, WARM, COLD, ALL_SSD, ONE_SSD, LAZY_PERSIST, plus version-dependent ones like ALL_NVDIMM and PROVIDED.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAttrOp.java:172

      }
    } finally {
      fsd.writeUnlock();
    }
    return isFile;
  }

  static FileStatus unsetStoragePolicy(FSDirectory fsd, FSPermissionChecker pc,
      BlockManager bm, String src) throws IOException {
    return setStoragePolicy(fsd, pc, bm, src,
        HdfsConstants.BLOCK_STORAGE_POLICY_ID_UNSPECIFIED);
  }

  static FileStatus setStoragePolicy(FSDirectory fsd, FSPermissionChecker pc,
      BlockManager bm, String src, final String policyName) throws IOException {
    // get the corresponding policy and make sure the policy name is valid
    BlockStoragePolicy policy = bm.getStoragePolicy(policyName);
    if (policy == null) {
      throw new HadoopIllegalArgumentException(
          "Cannot find a block policy with the name " + policyName);
    }
    return setStoragePolicy(fsd, pc, bm, src, policy.getId());
  }

  static FileStatus setStoragePolicy(FSDirectory fsd, FSPermissionChecker pc,
      BlockManager bm, String src, final byte policyId)
      throws IOException {
    INodesInPath iip;
    fsd.writeLock();
    try {
      iip = fsd.resolvePath(pc, src, DirOp.WRITE);

      if (fsd.isPermissionEnabled()) {
        fsd.checkPathAccess(pc, iip, FsAction.WRITE);
      }

      unprotectedSetStoragePolicy(fsd, bm, iip, policyId);

View on GitHub (pinned to 2add963021)

Solutions

List what the NameNode supports: 'hdfs storagepolicies -listStoragePolicies' (or DistributedFileSystem.getStoragePolicies()) and use one of those names exactly.
Check spelling and case - names are the uppercase constants from HdfsConstants.StoragePolicy.
If a custom/newer policy is expected, confirm the NameNode actually loaded it; the suite is NN-side, not client config.

Example fix

// before
fs.setStoragePolicy(path, policyName); // HadoopIllegalArgumentException on typo

// after: validate against the NN's suite
BlockStoragePolicy[] available = ((DistributedFileSystem) fs).getStoragePolicies();
boolean known = Arrays.stream(available)
    .anyMatch(p -> p.getName().equals(policyName));
if (!known) {
  throw new IllegalArgumentException("Unknown policy " + policyName
      + "; available: " + Arrays.toString(available));
}
fs.setStoragePolicy(path, policyName);

Defensive patterns

Strategy: validation

Validate before calling

BlockStoragePolicy[] available = ((DistributedFileSystem) fs).getStoragePolicies();
boolean known = Arrays.stream(available).anyMatch(sp -> sp.getName().equals(policyName));
if (!known) {
  throw new IllegalArgumentException("Unknown storage policy " + policyName
      + "; available on this NameNode: " + Arrays.toString(available));
}
fs.setStoragePolicy(path, policyName);

Try / catch

try {
  fs.setStoragePolicy(path, policyName);
} catch (HadoopIllegalArgumentException e) {
  if (e.getMessage().contains("Cannot find a block policy")) {
    // typo or version mismatch: list policies with getStoragePolicies() and re-run
  }
}

Prevention

Fetch the policy list at job start instead of hardcoding names.
Pin policy names per cluster version in deployment config, not in code.

When it happens

Trigger: DistributedFileSystem.setStoragePolicy(path, name) or 'hdfs storagepolicies -setStoragePolicy -policy <name>' with a typo, wrong case ('hot'), or a policy the NN suite does not define (e.g. PROVIDED on an older cluster).

Common situations: Policy scripts hardcoded across Hadoop versions; clusters with custom policy suites where the client and NN disagree; case mismatches or trailing whitespace in names from config/CLI input.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/cde07f9db4b14b73. Report an issue: GitHub.