apache/hadoop · error · IOException

Attempt to set an erasure coding policy for a file {}

Error message

Attempt to set an erasure coding policy for a file {}

What it means

IOException thrown when setErasureCodingPolicy targets an inode that is a file. EC policies are set on directories and inherited by files at write time; a file inode cannot carry the raw.ec.policy xattr.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java:172

      fsd.writeUnlock();
    }
    fsn.getEditLog().logSetXAttrs(src, xAttrs, logRetryCache);
    return fsd.getAuditFileInfo(iip);
  }

  private static List<XAttr> setErasureCodingPolicyXAttr(final FSNamesystem fsn,
      final INodesInPath srcIIP, ErasureCodingPolicy ecPolicy) throws IOException {
    FSDirectory fsd = fsn.getFSDirectory();
    assert fsd.hasWriteLock();
    Preconditions.checkNotNull(srcIIP, "INodes cannot be null");
    Preconditions.checkNotNull(ecPolicy, "EC policy cannot be null");
    String src = srcIIP.getPath();
    final INode inode = srcIIP.getLastINode();
    if (inode == null) {
      throw new FileNotFoundException("Path not found: " + srcIIP.getPath());
    }
    if (!inode.isDirectory()) {
      throw new IOException("Attempt to set an erasure coding policy " +
          "for a file " + src);
    }

    final XAttr ecXAttr;
    DataOutputStream dOut = null;
    try {
      ByteArrayOutputStream bOut = new ByteArrayOutputStream();
      dOut = new DataOutputStream(bOut);
      WritableUtils.writeString(dOut, ecPolicy.getName());
      ecXAttr = XAttrHelper.buildXAttr(XATTR_ERASURECODING_POLICY,
          bOut.toByteArray());
    } finally {
      IOUtils.closeStream(dOut);
    }
    // check whether the directory already has an erasure coding policy
    // directly on itself.
    final Boolean hasEcXAttr =
        getErasureCodingPolicyXAttrForINode(fsn, inode) == null ? false : true;

View on GitHub (pinned to 2add963021)

Solutions

1. Apply the policy to the containing directory instead: `hdfs ec -setPolicy -path <parent-dir>`.
2. If the path must be a directory: `hdfs dfs -rm <file>`, then `hdfs dfs -mkdir -p <path>`, then setPolicy.
3. Guard callers: require isDirectory() before invoking setErasureCodingPolicy.

Example fix

// before
Path p = new Path("/data/part-00000");
dfs.setErasureCodingPolicy(p, "RS-6-3-1024k"); // IOException: ...for a file

// after
Path p = new Path("/data");
dfs.setErasureCodingPolicy(p, "RS-6-3-1024k");

FileStatus st = fs.getFileStatus(target);
if (!st.isDirectory()) {
  throw new IllegalArgumentException("EC policy target must be a directory: " + target);
}
dfs.setErasureCodingPolicy(target, policyName);

static boolean isDirectoryTarget(FileSystem fs, Path p) throws IOException {
  return fs.exists(p) && fs.getFileStatus(p).isDirectory();
}

try {
  dfs.setErasureCodingPolicy(target, policyName);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("for a file")) {
    // retarget the parent directory instead
    dfs.setErasureCodingPolicy(target.getParent(), policyName);
  } else {
    throw e;
  }
}

When it happens

Trigger: `hdfs ec -setPolicy -path /data/part-00000` on a regular file; a directory path that was replaced by a same-named file; passing a file path from a manifest into setErasureCodingPolicy.

Common situations: Pointing at data files instead of their parent directory; path reuse across jobs where a directory was deleted and a file written with the same name; ETL steps that conflate dataset root files with dataset directories.

Related errors

• Cannot unset an erasure coding policy on a file {}
• Policy '%s' does not match any enabled erasure coding polici
• The given erasure coding policy {} does not exist.
• Path not found: {}
• No erasure coding policy explicitly set on {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0e0f6f4a10d9a71a. Report an issue: GitHub.