apache/hadoop · error · HadoopIllegalArgumentException

Cannot find a block policy with the name <storagePolicy>

Error message

Cannot find a block policy with the name <storagePolicy>

What it means

addFile resolves the caller-supplied storagePolicy string through BlockManager.getStoragePolicy(name) before creating the inode. An unknown name (misspelled, wrong case, or not defined on this cluster) returns null and startFile fails immediately with HadoopIllegalArgumentException; no file is created.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java:568

  private static INodesInPath addFile(
      FSDirectory fsd, INodesInPath existing, byte[] localName,
      PermissionStatus permissions, short replication, long preferredBlockSize,
      String clientName, String clientMachine, boolean shouldReplicate,
      String ecPolicyName, String storagePolicy) throws IOException {

    Preconditions.checkNotNull(existing);
    long modTime = now();
    INodesInPath newiip;
    fsd.writeLock();
    try {
      boolean isStriped = false;
      ErasureCodingPolicy ecPolicy = null;
      byte storagepolicyid = 0;
      if (storagePolicy != null && !storagePolicy.isEmpty()) {
        BlockStoragePolicy policy =
            fsd.getBlockManager().getStoragePolicy(storagePolicy);
        if (policy == null) {
          throw new HadoopIllegalArgumentException(
              "Cannot find a block policy with the name " + storagePolicy);
        }
        storagepolicyid = policy.getId();
      }
      if (!shouldReplicate) {
        ecPolicy = FSDirErasureCodingOp.getErasureCodingPolicy(
            fsd.getFSNamesystem(), ecPolicyName, existing);
        if (ecPolicy != null && (!ecPolicy.isReplicationPolicy())) {
          isStriped = true;
        }
      }
      final BlockType blockType = isStriped ?
          BlockType.STRIPED : BlockType.CONTIGUOUS;
      final Short replicationFactor = (!isStriped ? replication : null);
      final Byte ecPolicyID = (isStriped ? ecPolicy.getId() : null);
      INodeFile newNode = newINodeFile(fsd.allocateNewInodeId(), permissions,
          modTime, modTime, replicationFactor, ecPolicyID, preferredBlockSize,
          storagepolicyid, blockType);

View on GitHub (pinned to 2add963021)

Solutions

1. List valid policies with 'hdfs storagepolicies -listPolicies' and use an exact name (HOT, WARM, COLD, ALL_SSD, ONE_SSD, LAZY_PERSIST, PROVIDED)
2. Drop the policy argument at create time and set it afterwards with DistributedFileSystem.setStoragePolicy, which fails with a clearer error
3. Register the custom policy on the NameNode before referencing it

Example fix

// before: unknown/case-wrong policy at create
fs.create(path, perm, flags, bufSize, replication, blockSize, progress, null, "all_ssd", null);

// after: create plainly, then set policy with validation
FSDataOutputStream out = fs.create(path, true);
((DistributedFileSystem) fs).setStoragePolicy(path, "ALL_SSD");

static final Set<String> KNOWN = new HashSet<>(Arrays.asList(
    "HOT", "WARM", "COLD", "ALL_SSD", "ONE_SSD", "LAZY_PERSIST", "PROVIDED"));
if (!KNOWN.contains(storagePolicy)) {
  throw new IllegalArgumentException("Unknown HDFS storage policy: " + storagePolicy
      + "; run 'hdfs storagepolicies -listPolicies'");
}

static boolean isValidStoragePolicy(String name) {
  return KNOWN.contains(name); // extend with cluster-registered custom policies
}

try {
  fs.create(path, perm, flags, bufSize, repl, bs, progress, null, storagePolicy, null);
} catch (HadoopIllegalArgumentException e) {
  // invalid policy name: create plainly, then set a validated policy
  ((DistributedFileSystem) fs).setStoragePolicy(path, "HOT");
}

When it happens

Trigger: FileSystem.create with a storagePolicy parameter that does not match a registered policy. Names are case-sensitive: ALL_SSD, not all_ssd; custom policies must be defined in NameNode configuration to resolve.

Common situations: Hard-coded policy names that only exist on another cluster; version drift (PROVIDED exists only on newer Hadoop); typos and case errors in job or client configs.

Related errors

• {} doesn't support satisfyStoragePolicy
• {} doesn't support setStoragePolicy
• {} doesn't support unsetStoragePolicy
• {} doesn't support getStoragePolicy
• {} doesn't support getAllStoragePolicies

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0c21ea3019c4e599. Report an issue: GitHub.