{"record":{"id":"3ba2231ca0d6ea11","repo":"apache/hadoop","slug":"cannot-concatenate-source-file-because-it-is-lo","errorCode":null,"errorMessage":"Cannot concatenate source file {} because it is located in a different namespace with nameservice {} from the target file with nameservice {}","messagePattern":"Cannot concatenate source file (.+?) because it is located in a different namespace with nameservice (.+?) from the target file with nameservice (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java","lineNumber":701,"sourceCode":"\n    // Concat only effects when all files in the same namespace.\n    RemoteLocation targetDestination = getFileRemoteLocation(trg);\n    if (targetDestination == null) {\n      throw new IOException(\"Cannot find target file - \" + trg);\n    }\n    String targetNameService = targetDestination.getNameserviceId();\n\n    String[] sourceDestinations = new String[src.length];\n    for (int i = 0; i < src.length; i++) {\n      String sourceFile = src[i];\n      RemoteLocation srcLocation = getFileRemoteLocation(sourceFile);\n      if (srcLocation == null) {\n        throw new IOException(\"Cannot find source file - \" + sourceFile);\n      }\n      sourceDestinations[i] = srcLocation.getDest();\n\n      if (!targetNameService.equals(srcLocation.getNameserviceId())) {\n        throw new IOException(\"Cannot concatenate source file \" + sourceFile\n            + \" because it is located in a different namespace\" + \" with nameservice \"\n            + srcLocation.getNameserviceId() + \" from the target file with nameservice \"\n            + targetNameService);\n      }\n    }\n    // Invoke\n    RemoteMethod method = new RemoteMethod(\"concat\",\n        new Class<?>[] {String.class, String[].class},\n        targetDestination.getDest(), sourceDestinations);\n    rpcClient.invokeSingle(targetDestination, method, Void.class);\n  }\n\n  @Override\n  public boolean truncate(String src, long newLength, String clientName)\n      throws IOException {\n    rpcServer.checkOperation(NameNode.OperationCategory.WRITE);\n\n    final List<RemoteLocation> locations =","sourceCodeStart":683,"sourceCodeEnd":719,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java#L683-L719","documentation":"HDFS concat physically appends blocks to the target file, which only works within a single NameNode namespace. The Router therefore checks every source's resolved nameservice against the target's; on mismatch it throws IOException naming the source file, its nameservice, and the target's nameservice. This is a structural limitation of concat under federation, not a transient error.","triggerScenarios":"hdfs dfs -concat /mntA/trg /mntB/src where /mntA and /mntB are mounts into different subclusters (different nameservice IDs); multi-destination mounts where the target resolves to ns0 but a source resolves to ns1.","commonSituations":"Users assume Router federation merges namespaces transparently and try to concat across mount points; data pipeline writing parts to two different subclusters then attempting to concat.","solutions":["Ensure target and ALL sources live in the same nameservice (same mount subtree) before concat","If data must be combined across subclusters, use distcp/copy to move sources into the target's namespace first, then concat","Re-examine mount layout: put the directory tree that needs concat behind a single mount"],"exampleFix":"# before: rejected, /mntB is a different nameservice\nhdfs dfs -fs hdfs://router -concat /mntA/trg /mntB/part-0\n\n# after: copy source into target's namespace, then concat\nhadoop distcp hdfs://router/mntB/part-0 hdfs://router/mntA/part-0\nhdfs dfs -fs hdfs://router -concat /mntA/trg /mntA/part-0","handlingStrategy":"validation","validationCode":"// Enforce single-namespace concat before calling the Router\nRemoteLocation t = clientProtocol.getFileRemoteLocation(trg);\nfor (String s : srcs) {\n  RemoteLocation l = clientProtocol.getFileRemoteLocation(s);\n  if (l == null || !l.getNameserviceId().equals(t.getNameserviceId())) {\n    throw new IOException(s + \" is in \"\n        + (l == null ? \"?\" : l.getNameserviceId())\n        + \", target is in \" + t.getNameserviceId());\n  }\n}\nclientProtocol.concat(trg, srcs);","typeGuard":null,"tryCatchPattern":"try {\n  clientProtocol.concat(trg, srcs);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"located in a different namespace\")) {\n    // copy the offending source into the target's namespace (distcp), then concat there\n  } else { throw e; }\n}","preventionTips":["Keep files destined for concat under one mount (one nameservice)","Teach pipelines that RBF is a router, not a merged namespace: cross-cluster concat is impossible"],"tags":["hdfs","router-based-federation","concat","cross-namespace","mount-table"],"backgroundTag":"cross-cluster-operation-unsupported","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}