{"record":{"id":"0e0f6f4a10d9a71a","repo":"apache/hadoop","slug":"attempt-to-set-an-erasure-coding-policy-for-a-file","errorCode":null,"errorMessage":"Attempt to set an erasure coding policy for a file {}","messagePattern":"Attempt to set an erasure coding policy for a file (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java","lineNumber":172,"sourceCode":"      fsd.writeUnlock();\n    }\n    fsn.getEditLog().logSetXAttrs(src, xAttrs, logRetryCache);\n    return fsd.getAuditFileInfo(iip);\n  }\n\n  private static List<XAttr> setErasureCodingPolicyXAttr(final FSNamesystem fsn,\n      final INodesInPath srcIIP, ErasureCodingPolicy ecPolicy) throws IOException {\n    FSDirectory fsd = fsn.getFSDirectory();\n    assert fsd.hasWriteLock();\n    Preconditions.checkNotNull(srcIIP, \"INodes cannot be null\");\n    Preconditions.checkNotNull(ecPolicy, \"EC policy cannot be null\");\n    String src = srcIIP.getPath();\n    final INode inode = srcIIP.getLastINode();\n    if (inode == null) {\n      throw new FileNotFoundException(\"Path not found: \" + srcIIP.getPath());\n    }\n    if (!inode.isDirectory()) {\n      throw new IOException(\"Attempt to set an erasure coding policy \" +\n          \"for a file \" + src);\n    }\n\n    final XAttr ecXAttr;\n    DataOutputStream dOut = null;\n    try {\n      ByteArrayOutputStream bOut = new ByteArrayOutputStream();\n      dOut = new DataOutputStream(bOut);\n      WritableUtils.writeString(dOut, ecPolicy.getName());\n      ecXAttr = XAttrHelper.buildXAttr(XATTR_ERASURECODING_POLICY,\n          bOut.toByteArray());\n    } finally {\n      IOUtils.closeStream(dOut);\n    }\n    // check whether the directory already has an erasure coding policy\n    // directly on itself.\n    final Boolean hasEcXAttr =\n        getErasureCodingPolicyXAttrForINode(fsn, inode) == null ? false : true;","sourceCodeStart":154,"sourceCodeEnd":190,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java#L154-L190","documentation":"IOException thrown when setErasureCodingPolicy targets an inode that is a file. EC policies are set on directories and inherited by files at write time; a file inode cannot carry the raw.ec.policy xattr.","triggerScenarios":"`hdfs ec -setPolicy -path /data/part-00000` on a regular file; a directory path that was replaced by a same-named file; passing a file path from a manifest into setErasureCodingPolicy.","commonSituations":"Pointing at data files instead of their parent directory; path reuse across jobs where a directory was deleted and a file written with the same name; ETL steps that conflate dataset root files with dataset directories.","solutions":["Apply the policy to the containing directory instead: `hdfs ec -setPolicy -path <parent-dir>`.","If the path must be a directory: `hdfs dfs -rm <file>`, then `hdfs dfs -mkdir -p <path>`, then setPolicy.","Guard callers: require isDirectory() before invoking setErasureCodingPolicy."],"exampleFix":"// before\nPath p = new Path(\"/data/part-00000\");\ndfs.setErasureCodingPolicy(p, \"RS-6-3-1024k\"); // IOException: ...for a file\n\n// after\nPath p = new Path(\"/data\");\ndfs.setErasureCodingPolicy(p, \"RS-6-3-1024k\");","handlingStrategy":"type-guard","validationCode":"FileStatus st = fs.getFileStatus(target);\nif (!st.isDirectory()) {\n  throw new IllegalArgumentException(\"EC policy target must be a directory: \" + target);\n}\ndfs.setErasureCodingPolicy(target, policyName);","typeGuard":"static boolean isDirectoryTarget(FileSystem fs, Path p) throws IOException {\n  return fs.exists(p) && fs.getFileStatus(p).isDirectory();\n}","tryCatchPattern":"try {\n  dfs.setErasureCodingPolicy(target, policyName);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"for a file\")) {\n    // retarget the parent directory instead\n    dfs.setErasureCodingPolicy(target.getParent(), policyName);\n  } else {\n    throw e;\n  }\n}","preventionTips":["Apply EC policy at dataset-directory level, never per file.","Filter manifests with isDirectory() before policy assignment.","Watch for path reuse where files replace former directories."],"tags":["hdfs","erasure-coding","hdfs-ec","not-a-directory"],"backgroundTag":"not-a-directory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}