apache/hadoop · error · IOException

Cannot find source file - {}

Error message

Cannot find source file - {}

What it means

While resolving concat sources, RouterClientProtocol.concat() calls getFileRemoteLocation(src[i]) for every source file. A null result (file not found in any candidate subcluster) throws IOException('Cannot find source file - <src>'). concat requires the target and all sources to exist in the same nameservice, so the Router resolves each source up front and aborts on the first miss.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:696

  }

  @Override
  public void concat(String trg, String[] src) throws IOException {
    rpcServer.checkOperation(NameNode.OperationCategory.WRITE);

    // Concat only effects when all files in the same namespace.
    RemoteLocation targetDestination = getFileRemoteLocation(trg);
    if (targetDestination == null) {
      throw new IOException("Cannot find target file - " + trg);
    }
    String targetNameService = targetDestination.getNameserviceId();

    String[] sourceDestinations = new String[src.length];
    for (int i = 0; i < src.length; i++) {
      String sourceFile = src[i];
      RemoteLocation srcLocation = getFileRemoteLocation(sourceFile);
      if (srcLocation == null) {
        throw new IOException("Cannot find source file - " + sourceFile);
      }
      sourceDestinations[i] = srcLocation.getDest();

      if (!targetNameService.equals(srcLocation.getNameserviceId())) {
        throw new IOException("Cannot concatenate source file " + sourceFile
            + " because it is located in a different namespace" + " with nameservice "
            + srcLocation.getNameserviceId() + " from the target file with nameservice "
            + targetNameService);
      }
    }
    // Invoke
    RemoteMethod method = new RemoteMethod("concat",
        new Class<?>[] {String.class, String[].class},
        targetDestination.getDest(), sourceDestinations);
    rpcClient.invokeSingle(targetDestination, method, Void.class);
  }

  @Override

View on GitHub (pinned to 2add963021)

Solutions

1. List every source path through the Router (hdfs dfs -ls) and fix/remove missing ones before concat
2. Ensure source paths are under the same mount as the target and correctly namespaced
3. Retry if a transient rename/delete caused the miss

// All sources must exist before concat
for (String s : srcs) {
  if (clientProtocol.getFileInfo(s) == null) {
    throw new FileNotFoundException("concat source missing: " + s);
  }
}

try {
  clientProtocol.concat(trg, srcs);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Cannot find source file")) {
    // drop/recreate the missing source, or re-list and rebuild the source list
  } else { throw e; }
}

When it happens

Trigger: hdfs dfs -concat <trg> <src1> <src2> via the Router where one source path does not exist, was deleted, or its mount entry changed; source path outside any mounted subtree so resolution yields no locations.

Common situations: Race where a concurrent job deletes/appends a source between listing and concat; wrong relative path (missing mount prefix); mount table updated so source paths resolve elsewhere.

Related errors

• Cannot find target file - {}
• %s does not exist or is not file.
• ConcatDeleteOp can only have {} sources at most.
• File not found in downstream nameservices: {}
• Cannot concatenate source file {} because it is located in a

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/36bae3102b31a749. Report an issue: GitHub.