apache/hadoop · error · IOException

Cannot concatenate source file {sourceFile} because it is lo

Error message

Cannot concatenate source file {sourceFile} because it is located in a different namespace with nameservice {nameserviceId} from the target file with nameservice {targetNameService}

What it means

During Router concat validation, each resolved source file must live in the same nameservice as the target (targetNameService.equals(srcLocation.getNameserviceId())). HDFS concat physically merges blocks on a single NameNode, so sources in another namespace cannot be merged; the Router throws IOException naming the source file and both nameservice IDs. This is a hard federation boundary, not a transient condition.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/async/RouterAsyncClientProtocol.java:333

    // Concat only effects when all files in the same namespace.
    getFileRemoteLocation(trg);
    asyncApply((AsyncApplyFunction<RemoteLocation, Object>) targetDestination -> {
      if (targetDestination == null) {
        throw new IOException("Cannot find target file - " + trg);
      }
      String targetNameService = targetDestination.getNameserviceId();
      String[] sourceDestinations = new String[src.length];
      int[] index = new int[1];
      asyncForEach(Arrays.stream(src).iterator(), (forEachRun, sourceFile) -> {
        getFileRemoteLocation(sourceFile);
        asyncApply((ApplyFunction<RemoteLocation, Object>) srcLocation -> {
          if (srcLocation == null) {
            throw new IOException("Cannot find source file - " + sourceFile);
          }
          sourceDestinations[index[0]++] = srcLocation.getDest();
          if (!targetNameService.equals(srcLocation.getNameserviceId())) {
            throw new IOException("Cannot concatenate source file " + sourceFile
                + " because it is located in a different namespace" + " with nameservice "
                + srcLocation.getNameserviceId() + " from the target file with nameservice "
                + targetNameService);
          }
          return null;
        });
      });
      asyncApply((AsyncApplyFunction<Object, Object>) o -> {
        // Invoke
        RemoteMethod method = new RemoteMethod("concat",
            new Class<?>[] {String.class, String[].class},
            targetDestination.getDest(), sourceDestinations);
        rpcClient.invokeSingle(targetDestination, method, Void.class);
      });
    });
  }

  @Override

View on GitHub (pinned to 2add963021)

Solutions

Concat only files within one nameservice: pick target and sources inside the same mount entry / namespace
Move cross-namespace sources next to the target first (distcp/copy), then concat within the namespace
Review the mount table (hdfs dfsrouteradmin -ls) so writers place all parts of a concat set in the same subcluster

Defensive patterns

Strategy: validation

Validate before calling

// Resolve namespace of target and all sources before concat;
// all must map to the same nameservice (mount entry):
String trgNs = resolverNamespaceOf(trg);   // e.g. via Router admin/REST mount info
for (String s : srcs) {
  if (!trgNs.equals(resolverNamespaceOf(s))) {
    // copy s next to trg (distcp) or refuse the concat
  }
}

Try / catch

catch (IOException e) {
  if (e.getMessage().contains("different namespace")) {
    // cross-namespace concat is impossible: copy sources into the target's
    // namespace first, then concat within it
  }
}

Prevention

Design job outputs so all concat parts land under one mount entry
Review mount splits before enabling concat-based commiters on federated paths
Use distcp or copy for cross-namespace consolidation; never concat across namespaces

When it happens

Trigger: concat where target and one or more source files resolve through different mount entries to different nameservices; sources on a different subcluster than the target after data placement or mount-table changes.

Common situations: Mount table splits one logical directory tree across subclusters (e.g. /logs/day1 on ns0, /logs/day2 on ns1) and a job tries to concat across the boundary; data redistributed by tiering/migration tools.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ac69e23d829019b1. Report an issue: GitHub.