{"record":{"id":"ac69e23d829019b1","repo":"apache/hadoop","slug":"cannot-concatenate-source-file-sourcefile-becaus","errorCode":null,"errorMessage":"Cannot concatenate source file {sourceFile} because it is located in a different namespace with nameservice {nameserviceId} from the target file with nameservice {targetNameService}","messagePattern":"Cannot concatenate source file (.+?) because it is located in a different namespace with nameservice (.+?) from the target file with nameservice (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/async/RouterAsyncClientProtocol.java","lineNumber":333,"sourceCode":"\n    // Concat only effects when all files in the same namespace.\n    getFileRemoteLocation(trg);\n    asyncApply((AsyncApplyFunction<RemoteLocation, Object>) targetDestination -> {\n      if (targetDestination == null) {\n        throw new IOException(\"Cannot find target file - \" + trg);\n      }\n      String targetNameService = targetDestination.getNameserviceId();\n      String[] sourceDestinations = new String[src.length];\n      int[] index = new int[1];\n      asyncForEach(Arrays.stream(src).iterator(), (forEachRun, sourceFile) -> {\n        getFileRemoteLocation(sourceFile);\n        asyncApply((ApplyFunction<RemoteLocation, Object>) srcLocation -> {\n          if (srcLocation == null) {\n            throw new IOException(\"Cannot find source file - \" + sourceFile);\n          }\n          sourceDestinations[index[0]++] = srcLocation.getDest();\n          if (!targetNameService.equals(srcLocation.getNameserviceId())) {\n            throw new IOException(\"Cannot concatenate source file \" + sourceFile\n                + \" because it is located in a different namespace\" + \" with nameservice \"\n                + srcLocation.getNameserviceId() + \" from the target file with nameservice \"\n                + targetNameService);\n          }\n          return null;\n        });\n      });\n      asyncApply((AsyncApplyFunction<Object, Object>) o -> {\n        // Invoke\n        RemoteMethod method = new RemoteMethod(\"concat\",\n            new Class<?>[] {String.class, String[].class},\n            targetDestination.getDest(), sourceDestinations);\n        rpcClient.invokeSingle(targetDestination, method, Void.class);\n      });\n    });\n  }\n\n  @Override","sourceCodeStart":315,"sourceCodeEnd":351,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/async/RouterAsyncClientProtocol.java#L315-L351","documentation":"During Router concat validation, each resolved source file must live in the same nameservice as the target (targetNameService.equals(srcLocation.getNameserviceId())). HDFS concat physically merges blocks on a single NameNode, so sources in another namespace cannot be merged; the Router throws IOException naming the source file and both nameservice IDs. This is a hard federation boundary, not a transient condition.","triggerScenarios":"concat where target and one or more source files resolve through different mount entries to different nameservices; sources on a different subcluster than the target after data placement or mount-table changes.","commonSituations":"Mount table splits one logical directory tree across subclusters (e.g. /logs/day1 on ns0, /logs/day2 on ns1) and a job tries to concat across the boundary; data redistributed by tiering/migration tools.","solutions":["Concat only files within one nameservice: pick target and sources inside the same mount entry / namespace","Move cross-namespace sources next to the target first (distcp/copy), then concat within the namespace","Review the mount table (hdfs dfsrouteradmin -ls) so writers place all parts of a concat set in the same subcluster"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Resolve namespace of target and all sources before concat;\n// all must map to the same nameservice (mount entry):\nString trgNs = resolverNamespaceOf(trg);   // e.g. via Router admin/REST mount info\nfor (String s : srcs) {\n  if (!trgNs.equals(resolverNamespaceOf(s))) {\n    // copy s next to trg (distcp) or refuse the concat\n  }\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) {\n  if (e.getMessage().contains(\"different namespace\")) {\n    // cross-namespace concat is impossible: copy sources into the target's\n    // namespace first, then concat within it\n  }\n}","preventionTips":["Design job outputs so all concat parts land under one mount entry","Review mount splits before enabling concat-based commiters on federated paths","Use distcp or copy for cross-namespace consolidation; never concat across namespaces"],"tags":["hadoop","hdfs","rbf","concat","cross-namespace","mount-table","async-rpc"],"backgroundTag":"cross-namespace-operation","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}