apache/hadoop · error · FileNotFoundException

Target path %s does not exist or is not file.

Error message

Target path %s does not exist or is not file.

What it means

Thrown by Concat.processArguments() when the first argument (the target) does not exist or is not a regular file: !target.exists || !target.stat.isFile() raises FileNotFoundException('Target path <p> does not exist or is not file.'). concat appends source blocks TO the existing target, so the target must already be a file; each source is validated the same way in the following loop ('<src> does not exist or is not file.').

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Concat.java:62

  public static final String DESCRIPTION = "Concatenate existing source files"
      + " into the target file. Target file and source files should be in the"
      + " same directory.";
  private static FileSystem testFs; // test only.

  @Override
  protected void processArguments(LinkedList<PathData> args)
      throws IOException {
    if (args.size() < 1) {
      throw new IOException("Target path not specified. " + USAGE);
    }
    if (args.size() < 3) {
      throw new IOException(
          "The number of source paths is less than 2. " + USAGE);
    }
    PathData target = args.removeFirst();
    LinkedList<PathData> srcList = args;
    if (!target.exists || !target.stat.isFile()) {
      throw new FileNotFoundException(String
          .format("Target path %s does not exist or is" + " not file.",
              target.path));
    }
    Path[] srcArray = new Path[srcList.size()];
    for (int i = 0; i < args.size(); i++) {
      PathData src = srcList.get(i);
      if (!src.exists || !src.stat.isFile()) {
        throw new FileNotFoundException(
            String.format("%s does not exist or is not file.", src.path));
      }
      srcArray[i] = src.path;
    }
    FileSystem fs = target.fs;
    if (testFs != null) {
      fs = testFs;
    }
    try {
      fs.concat(target.path, srcArray);

View on GitHub (pinned to 2add963021)

Solutions

1. Create the target file first ('hdfs dfs -touchz /data/target' or copy the first part to the target name), then concat the REMAINING parts into it
2. Verify each argument with 'hdfs dfs -ls' (or fs.getFileStatus().isFile() in Java) before invoking concat
3. Ensure sources and target are in the same directory and all are regular files

Example fix

# before
hdfs dfs concat /data/target /data/part-0 /data/part-1   # /data/target absent
# after
hdfs dfs -cp /data/part-0 /data/target && hdfs dfs concat /data/target /data/part-1

// every concat argument must be an existing regular file
static void validateConcatArgs(FileSystem fs, Path target, List<Path> srcs) throws IOException {
  if (!fs.exists(target) || !fs.getFileStatus(target).isFile())
    throw new FileNotFoundException("target must be an existing file: " + target);
  for (Path s : srcs) {
    if (!fs.exists(s) || !fs.getFileStatus(s).isFile())
      throw new FileNotFoundException("source must be an existing file: " + s);
  }
}

static boolean isExistingFile(FileSystem fs, Path p) throws IOException {
  return fs.exists(p) && !fs.getFileStatus(p).isDirectory();
}

try {
  fs.concat(target, srcArray); // hdfs dfs concat path
} catch (FileNotFoundException e) {
  // bootstrap the target from the first part, then concat the remainder
  fs.rename(srcArray[0], target);
  fs.concat(target, Arrays.copyOfRange(srcArray, 1, srcArray.length));
}

When it happens

Trigger: 'hdfs dfs concat /data/missing /a /b'; target is a directory; target created by a previous step that failed silently; sources include a directory or a path deleted between listing and concat.

Common situations: Compaction flows where the 'target' was meant to be created by an earlier job (e.g. via -touchz or getmerge) that did not run; passing the directory path instead of the base part file.

Related errors

• Target path not specified. <target path> <src path> <src pat
• The number of source paths is less than 2. <target path> <sr
• Illegal option {}
• Not enough arguments: expected {} but got {}
• Too many arguments: expected {} but got {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1c117989b2452584. Report an issue: GitHub.