What causes "iip.getPath() + " is not a file or directory"" in apache/hadoop?

Apply the policy to the symlink's target (resolve the link) or to the parent directory, which real children inherit.

apache/hadoop · error · FileNotFoundException

iip.getPath() + " is not a file or directory"

Error message

iip.getPath() + " is not a file or directory"

What it means

unprotectedSetStoragePolicy dispatches on inode type: files get per-file policy handling, directories get recursive/inherited handling, and anything else - which in HDFS means a symlink - throws FileNotFoundException stating the path 'is not a file or directory'. The path exists; it simply cannot carry a storage policy. The FileNotFound type is reused for 'unusable target', which makes it look like a missing-file bug when it is not.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAttrOp.java:468

              + " cannot be set after file creation.");
        }
      }

      BlockStoragePolicy currentPolicy =
          bm.getStoragePolicy(inode.getLocalStoragePolicyID());

      if (currentPolicy != null && currentPolicy.isCopyOnCreateFile()) {
        throw new HadoopIllegalArgumentException(
            "Existing policy " + currentPolicy.getName() +
                " cannot be changed after file creation.");
      }
      inode.asFile().setStoragePolicyID(policyId, snapshotId);
    } else if (inode.isDirectory()) {
      FSDirectory.LOG.debug("DIR* FSDirAAr.unprotectedSetStoragePolicy for " +
              "Directory.");
      setDirStoragePolicy(fsd, iip, policyId);
    } else {
      throw new FileNotFoundException(iip.getPath()
          + " is not a file or directory");
    }
  }

  private static void setDirStoragePolicy(
      FSDirectory fsd, INodesInPath iip, byte policyId) throws IOException {
    INode inode = FSDirectory.resolveLastINode(iip);
    List<XAttr> existingXAttrs = XAttrStorage.readINodeXAttrs(inode);
    XAttr xAttr = BlockStoragePolicySuite.buildXAttr(policyId);
    List<XAttr> newXAttrs = null;
    if (policyId == HdfsConstants.BLOCK_STORAGE_POLICY_ID_UNSPECIFIED) {
      List<XAttr> toRemove = Lists.newArrayList();
      toRemove.add(xAttr);
      List<XAttr> removed = Lists.newArrayList();
      newXAttrs = FSDirXAttrOp.filterINodeXAttrs(existingXAttrs, toRemove,
          removed);
    } else {
      newXAttrs = FSDirXAttrOp.setINodeXAttrs(fsd, existingXAttrs,

View on GitHub (pinned to 2add963021)

Solutions

Apply the policy to the symlink's target (resolve the link) or to the parent directory, which real children inherit.
Skip symlinks during bulk operations: check fs.getFileLinkStatus(path).isSymlink() before setting policy.
Avoid placing symlinks in trees you intend to manage with storage policies.

Example fix

// before
fs.setStoragePolicy(linkPath, "COLD"); // FileNotFoundException: not a file or directory

// after: resolve the link, set policy on the real target
FileStatus st = fs.getFileLinkStatus(linkPath);
Path target = st.isSymlink() ? st.getSymlink() : linkPath;
fs.setStoragePolicy(target.makeQualified(fs.getUri(), fs.getWorkingDirectory()), "COLD");

Defensive patterns

Strategy: type-guard

Validate before calling

FileStatus st = fs.getFileLinkStatus(path); // does not follow the link
if (st.isSymlink()) {
  throw new IllegalArgumentException(path + " is a symlink; apply the policy to "
      + st.getSymlink() + " or the parent directory");
}

Type guard

static Path storagePolicyTarget(FileSystem fs, Path p) throws IOException {
  FileStatus st = fs.getFileLinkStatus(p); // lstat semantics
  return st.isSymlink()
      ? st.getSymlink().makeQualified(fs.getUri(), fs.getWorkingDirectory())
      : p;
}

Try / catch

try {
  fs.setStoragePolicy(path, policyName);
} catch (FileNotFoundException e) {
  if (e.getMessage().endsWith("is not a file or directory")) {
    // path is a symlink: resolve it and set the policy on the real target
  }
}

Prevention

Skip symlinks explicitly in bulk policy walks (getFileLinkStatus + isSymlink).
Prefer applying policies to parent directories, which children inherit regardless of type.

When it happens

Trigger: setStoragePolicy / 'hdfs storagepolicies -setStoragePolicy' on an HDFS symlink: the resolved INode is an INodeSymlink, failing both isFile() and isDirectory() - e.g. a 'latest -> 2026-08-22' convenience link inside a managed tree.

Common situations: Policy rollouts over trees containing compatibility symlinks; manifests mixing real directories and links; tooling that walks listings without distinguishing link entries.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e7e21c517f5d9b0c. Report an issue: GitHub.