apache/hadoop · error · FileNotFoundException

Path not found: {}

Error message

Path not found: {}

What it means

FileNotFoundException thrown by setErasureCodingPolicy when the resolved path has no final inode: the target directory does not exist. The EC policy is stored as an xattr on an existing inode, so resolution failure aborts before any xattr is written.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java:169

      src = iip.getPath();
      xAttrs = setErasureCodingPolicyXAttr(fsn, iip, ecPolicy);
    } finally {
      fsd.writeUnlock();
    }
    fsn.getEditLog().logSetXAttrs(src, xAttrs, logRetryCache);
    return fsd.getAuditFileInfo(iip);
  }

  private static List<XAttr> setErasureCodingPolicyXAttr(final FSNamesystem fsn,
      final INodesInPath srcIIP, ErasureCodingPolicy ecPolicy) throws IOException {
    FSDirectory fsd = fsn.getFSDirectory();
    assert fsd.hasWriteLock();
    Preconditions.checkNotNull(srcIIP, "INodes cannot be null");
    Preconditions.checkNotNull(ecPolicy, "EC policy cannot be null");
    String src = srcIIP.getPath();
    final INode inode = srcIIP.getLastINode();
    if (inode == null) {
      throw new FileNotFoundException("Path not found: " + srcIIP.getPath());
    }
    if (!inode.isDirectory()) {
      throw new IOException("Attempt to set an erasure coding policy " +
          "for a file " + src);
    }

    final XAttr ecXAttr;
    DataOutputStream dOut = null;
    try {
      ByteArrayOutputStream bOut = new ByteArrayOutputStream();
      dOut = new DataOutputStream(bOut);
      WritableUtils.writeString(dOut, ecPolicy.getName());
      ecXAttr = XAttrHelper.buildXAttr(XATTR_ERASURECODING_POLICY,
          bOut.toByteArray());
    } finally {
      IOUtils.closeStream(dOut);
    }
    // check whether the directory already has an erasure coding policy

View on GitHub (pinned to 2add963021)

Solutions

1. Create the directory first: `hdfs dfs -mkdir -p <path>`, then `hdfs ec -setPolicy -path <path>`.
2. Check spelling and case of every component with `hdfs dfs -ls <parent>`.
3. If a concurrent delete is possible, re-check existence immediately before the call and retry once.

Example fix

# before
hdfs ec -setPolicy -policy RS-6-3-1024k -path /warehouse/db/part=missing
# -> FileNotFoundException: Path not found

# after
hdfs dfs -mkdir -p /warehouse/db/part=missing
hdfs ec -setPolicy -policy RS-6-3-1024k -path /warehouse/db/part=missing

if (!fs.exists(dir)) {
  throw new FileNotFoundException("Cannot set EC policy; path missing: " + dir);
}
dfs.setErasureCodingPolicy(dir, policyName);

try {
  dfs.setErasureCodingPolicy(dir, policyName);
} catch (FileNotFoundException e) {
  fs.mkdirs(dir);                     // create then retry once
  dfs.setErasureCodingPolicy(dir, policyName);
}

When it happens

Trigger: `hdfs ec -setPolicy -path /ec/typo`; setPolicy racing with a concurrent delete of the directory; setPolicy executed before the directory is created; wrong parent in a Hive/HBase-on-HDFS layout template.

Common situations: Scripts that create a directory and set its policy in one job but skip `mkdir -p`; paths read from config with a missing component; multi-step provisioners where the mkdir step failed earlier.

Related errors

• No erasure coding policy explicitly set on {}
• Policy '%s' does not match any enabled erasure coding polici
• The given erasure coding policy {} does not exist.
• Attempt to set an erasure coding policy for a file {}
• Cannot unset an erasure coding policy on a file {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b02e88162770209f. Report an issue: GitHub.