{"record":{"id":"367194e3a8aea5cf","repo":"apache/hadoop","slug":"concat-the-src-file-src-is-the-same-with","errorCode":null,"errorMessage":"\"concat: the src file \" + src + \" is the same with the target file \" + targetIIP.getPath()","messagePattern":"\"concat: the src file \" \\+ src \\+ \" is the same with the target file \" \\+ targetIIP\\.getPath\\(\\)","errorType":"exception","errorClass":"HadoopIllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirConcatOp.java","lineNumber":149,"sourceCode":"      if (srcINodeFile.getParent() != targetParent) {\n        throw new HadoopIllegalArgumentException(\"Source file \" + src\n            + \" is not in the same directory with the target \"\n            + targetIIP.getPath());\n      }\n      // make sure all the source files are not in snapshot\n      if (srcINode.isInLatestSnapshot(iip.getLatestSnapshotId())) {\n        throw new SnapshotException(\"Concat: the source file \" + src\n            + \" is in snapshot\");\n      }\n      // check if the file has other references.\n      if (srcINode.isReference() && ((INodeReference.WithCount)\n          srcINode.asReference().getReferredINode()).getReferenceCount() > 1) {\n        throw new SnapshotException(\"Concat: the source file \" + src\n            + \" is referred by some other reference in some snapshot.\");\n      }\n      // source file cannot be the same with the target file\n      if (srcINode.equals(targetINode)) {\n        throw new HadoopIllegalArgumentException(\"concat: the src file \" + src\n            + \" is the same with the target file \" + targetIIP.getPath());\n      }\n      // source file cannot be under construction or empty\n      if(srcINodeFile.isUnderConstruction() || srcINodeFile.numBlocks() == 0) {\n        throw new HadoopIllegalArgumentException(\"concat: source file \" + src\n            + \" is invalid or empty or underConstruction\");\n      }\n\n      // source file's preferred block size cannot be greater than the target\n      // file\n      if (srcINodeFile.getPreferredBlockSize() >\n          targetINode.getPreferredBlockSize()) {\n        throw new HadoopIllegalArgumentException(\"concat: source file \" + src\n            + \" has preferred block size \" + srcINodeFile.getPreferredBlockSize()\n            + \" which is greater than the target file's preferred block size \"\n            + targetINode.getPreferredBlockSize());\n      }\n      if(srcINodeFile.getErasureCodingPolicyID() !=","sourceCodeStart":131,"sourceCodeEnd":167,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirConcatOp.java#L131-L167","documentation":"The src array contained the target itself: verifySrcFiles compares srcINode.equals(targetINode) and rejects it. Concat appends each source's blocks onto the target and then deletes the sources; using the target as its own source is logically undefined (a file cannot absorb itself). Note the comparison is on INode identity, so an equivalent path, a snapshot path, or a different string resolving to the same inode also triggers it.","triggerScenarios":"Passing the target path (or any alias of it, such as '/dir/f' vs '/dir/./f', or a snapshot path resolving to the same inode) in the srcs array — often from a glob that includes the target or from not filtering the target out of a directory listing.","commonSituations":"Compaction code that lists a directory and concats 'all files in the directory' without excluding the chosen target; glob patterns like part-* that match the target too; config-driven file lists that accidentally repeat the target.","solutions":["Filter the target out of the src list before calling concat: srcs = list minus target (compare normalized absolute paths).","Dedupe and normalize all paths (Path.makeQualified + toUri().getPath()) so aliases of the target are caught too.","If the intent is 'merge everything', pick the oldest file as target and concat only the rest."],"exampleFix":"// before\nList<Path> all = listFiles(dir); // includes target\nfs.concat(target, all.toArray(new Path[0])); // target in srcs -> error\n\n// after\nString t = target.makeQualified(fs.getUri(), fs.getWorkingDirectory()).toUri().getPath();\nPath[] srcs = all.stream()\n    .filter(p -> !p.makeQualified(fs.getUri(), fs.getWorkingDirectory()).toUri().getPath().equals(t))\n    .toArray(Path[]::new);\nfs.concat(target, srcs);","handlingStrategy":"validation","validationCode":"String tq = target.makeQualified(fs.getUri(), fs.getWorkingDirectory()).toUri().getPath();\nPath[] safe = Arrays.stream(srcs)\n    .map(p -> p.makeQualified(fs.getUri(), fs.getWorkingDirectory()))\n    .filter(p -> !p.toUri().getPath().equals(tq))\n    .distinct()\n    .toArray(Path[]::new);","typeGuard":null,"tryCatchPattern":"catch (HadoopIllegalArgumentException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"same with the target file\")) {\n    // filter target alias out of srcs and retry once\n    fs.concat(target, withoutTargetAliases(target, srcs));\n  } else { throw e; }\n}","preventionTips":["When building 'merge all files in a directory' lists, always exclude the chosen target explicitly.","Normalize paths (makeQualified) before comparing so '.', double slashes, and URI-qualified aliases cannot slip through.","Unit-test compaction helpers with a src list that intentionally contains the target."],"tags":["hdfs","concat","self-reference","precondition"],"backgroundTag":"concat-invalid-source","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}