{"record":{"id":"f7a7df94faade6b9","repo":"apache/hadoop","slug":"concat-operation-doesn-t-support-fsdirectory-f7a7df","errorCode":null,"errorMessage":"\"Concat operation doesn't support \" + FSDirectory.DOT_RESERVED_STRING + \" relative path : \" + srcPath","messagePattern":"\"Concat operation doesn't support \" \\+ FSDirectory\\.DOT_RESERVED_STRING \\+ \" relative path : \" \\+ srcPath","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirConcatOp.java","lineNumber":93,"sourceCode":"    }\n    fsd.getEditLog().logConcat(target, srcs, timestamp, logRetryCache);\n    return fsd.getAuditFileInfo(targetIIP);\n  }\n\n  private static void validatePath(String target, String[] srcs)\n      throws IOException {\n    Preconditions.checkArgument(!target.isEmpty(), \"Target file name is empty\");\n    Preconditions.checkArgument(srcs != null && srcs.length > 0,\n        \"No sources given\");\n    if (FSDirectory.isReservedRawName(target)\n        || FSDirectory.isReservedInodesName(target)) {\n      throw new IOException(\"Concat operation doesn't support \"\n          + FSDirectory.DOT_RESERVED_STRING + \" relative path : \" + target);\n    }\n    for (String srcPath : srcs) {\n      if (FSDirectory.isReservedRawName(srcPath)\n          || FSDirectory.isReservedInodesName(srcPath)) {\n        throw new IOException(\"Concat operation doesn't support \"\n            + FSDirectory.DOT_RESERVED_STRING + \" relative path : \" + srcPath);\n      }\n    }\n  }\n\n  private static void verifyTargetFile(FSDirectory fsd, final String target,\n      final INodesInPath targetIIP) throws IOException {\n    // check the target\n    if (FSDirEncryptionZoneOp.getEZForPath(fsd, targetIIP) != null) {\n      throw new HadoopIllegalArgumentException(\n          \"concat can not be called for files in an encryption zone.\");\n    }\n    final INodeFile targetINode = INodeFile.valueOf(targetIIP.getLastINode(),\n        target);\n    if(targetINode.isUnderConstruction()) {\n      throw new HadoopIllegalArgumentException(\"concat: target file \"\n          + target + \" is under construction\");\n    }","sourceCodeStart":75,"sourceCodeEnd":111,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirConcatOp.java#L75-L111","documentation":"Identical guard to the target-path check, but applied to each SOURCE path: concat refuses any src under /.reserved/raw or /.reserved/inodes. validatePath loops over every element of srcs and throws before target/source file verification begins. The rationale is the same — reserved paths are virtual views and cannot participate in namespace mutation.","triggerScenarios":"Calling FileSystem.concat(target, srcs) where one or more entries in srcs contain the /.reserved/raw prefix — e.g. a manifest of raw paths produced by an encryption-zone-aware listing tool is fed straight into concat.","commonSituations":"Ingest pipelines that list an encryption zone through /.reserved/raw for verification and then reuse the same path list for compaction via concat; mixed toolchains where one component adds the prefix and another (correctly) does not.","solutions":["Strip the reserved prefix from every source path before calling concat (map the srcs array through a normalizer).","Keep two distinct path lists in EZ tooling: raw paths for reading bytes, plain paths for mutations — never interchange them.","Add a unit assertion in your pipeline that no path passed to a mutating FileSystem API starts with /.reserved."],"exampleFix":"// before\nPath[] srcs = rawManifest.stream().map(Path::new).toArray(Path[]::new);\nfs.concat(target, srcs);\n\n// after\nPath[] srcs = rawManifest.stream()\n    .map(p -> new Path(p).toUri().getPath().replaceFirst(\"^/\\\\.reserved/(raw|inodes)\", \"\"))\n    .map(Path::new).toArray(Path[]::new);\nfs.concat(target, srcs);","handlingStrategy":"validation","validationCode":"static boolean isReservedPath(Path p) {\n  String s = p.toUri().getPath();\n  return s != null && (s.equals(\"/.reserved\") || s.startsWith(\"/.reserved/\"));\n}\nfor (Path src : srcs) {\n  if (isReservedPath(src)) throw new IllegalArgumentException(\"strip /.reserved from src: \" + src);\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"doesn't support .reserved\")) {\n    srcs = Arrays.stream(srcs).map(p -> stripReserved(p)).toArray(Path[]::new);\n    fs.concat(target, srcs); // retry with cleaned sources\n  } else { throw e; }\n}","preventionTips":["Sanitize whole src arrays in one place before any mutating call, not per call site.","Never feed a manifest produced by /.reserved/raw listings directly into concat; map raw paths to plain paths explicitly.","Log the exact src list at DEBUG before concat so offending paths are identifiable on failure."],"tags":["hdfs","concat","reserved-path","encryption-zone"],"backgroundTag":"hdfs-reserved-path","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}