apache/hadoop · error · FileNotFoundException

does not exist or is not file.

Error message

%s does not exist or is not file.

What it means

Thrown by the 'hadoop fs -concat' command (Concat.java:70) during argument pre-validation: one of the SOURCE paths passed after the target does not exist on the filesystem, or exists but is not a regular file (e.g. a directory). Concat only merges existing regular files into an existing target file in the same directory, so every src path is checked with PathData.exists and stat.isFile() before FileSystem.concat() is called.

Solutions

1. Verify every source exists and is a file before running the command: 'hdfs dfs -ls /data/part-*' or in code fs.getFileStatus(src).isFile()
2. Fix the typo'd or stale path in the command / script that assembles the src list
3. If the source is a directory, list its files and pass them explicitly, or use 'hadoop fs -getmerge' instead (it reads contents rather than merging blocks)
4. Ensure all src paths and the target are on the same filesystem and, per the command contract, in the same directory

Example fix

# before
hdfs dfs -concat /data/target /data/part-000 /data/part-999   # part-999 does not exist

# after
hdfs dfs -ls /data/part-*    # confirm each source exists and is a regular file
hdfs dfs -concat /data/target /data/part-000 /data/part-001

FileSystem fs = targetPath.getFileSystem(conf);
for (Path src : srcPaths) {
  if (!fs.exists(src) || !fs.getFileStatus(src).isFile()) {
    throw new FileNotFoundException(src + " does not exist or is not file.");
  }
}

static boolean isExistingRegularFile(FileSystem fs, Path p) throws IOException {
  return fs.exists(p) && fs.getFileStatus(p).isFile();
}

try {
  fs.concat(target, srcs);
} catch (FileNotFoundException e) {
  // message names the offending src path; re-validate and report which source vanished
}

When it happens

Trigger: Running 'hdfs dfs -concat /dir/target /dir/src1 /dir/src2' where any src path has a typo, was deleted/renamed by a concurrent job between listing and concat, is a directory, or is an empty glob expansion. The failure occurs before any block merge is attempted.

Common situations: Typos in generated shell scripts that build the src list dynamically; a MapReduce/Spark job renaming part files after the script listed them; passing a directory or a glob like '/data/part-*' that matched nothing; src paths that resolve on a different filesystem than the target.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Dest filesystem ' ' doesn't support concat.
• Target path not specified.
• The number of source paths is less than 2.
• Cannot find source file
• Cannot find target file

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8f130988a1dce719. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Concat.java:70

    if (args.size() < 1) {
      throw new IOException("Target path not specified. " + USAGE);
    }
    if (args.size() < 3) {
      throw new IOException(
          "The number of source paths is less than 2. " + USAGE);
    }
    PathData target = args.removeFirst();
    LinkedList<PathData> srcList = args;
    if (!target.exists || !target.stat.isFile()) {
      throw new FileNotFoundException(String
          .format("Target path %s does not exist or is" + " not file.",
              target.path));
    }
    Path[] srcArray = new Path[srcList.size()];
    for (int i = 0; i < args.size(); i++) {
      PathData src = srcList.get(i);
      if (!src.exists || !src.stat.isFile()) {
        throw new FileNotFoundException(
            String.format("%s does not exist or is not file.", src.path));
      }
      srcArray[i] = src.path;
    }
    FileSystem fs = target.fs;
    if (testFs != null) {
      fs = testFs;
    }
    try {
      fs.concat(target.path, srcArray);
    } catch (UnsupportedOperationException exception) {
      throw new PathIOException("Dest filesystem '" + fs.getUri().getScheme()
          + "' doesn't support concat.", exception);
    }
  }

  @VisibleForTesting
  static void setTestFs(FileSystem fs) {

View on GitHub (pinned to 2add963021)