apache/hadoop · error · PathNotFoundException

No such file or directory

Error message

No such file or directory

What it means

Thrown by CommandWithDestination.getRemoteDestination() when the DESTINATION argument of -put/-cp/-moveFromLocal/-appendToFile is a glob pattern and PathData.expandAsGlob() returns zero matches. PathNotFoundException (a PathIOException subclass) renders as 'dest: No such file or directory'. Important nuance: a plain non-glob destination that does not exist is fine (globStatus returns null and a placeholder PathData with null stat is created) - the exception requires glob metacharacters that match nothing.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java:213

  }

  /**
   *  The last arg is expected to be a remote path, if only one argument is
   *  given then the destination will be the remote user's directory 
   *  @param args is the list of arguments
   *  @throws PathIOException if path doesn't exist or matches too many times 
   */
  protected void getRemoteDestination(LinkedList<String> args)
  throws IOException {
    if (args.size() < 2) {
      dst = new PathData(Path.CUR_DIR, getConf());
    } else {
      String pathString = args.removeLast();
      // if the path is a glob, then it must match one and only one path
      PathData[] items = PathData.expandAsGlob(pathString, getConf());
      switch (items.length) {
        case 0:
          throw new PathNotFoundException(pathString);
        case 1:
          dst = items[0];
          break;
        default:
          throw new PathIOException(pathString, "Too many matches");
      }
    }
  }

  @Override
  protected void processArguments(LinkedList<PathData> args)
  throws IOException {
    // if more than one arg, the destination must be a directory
    // if one arg, the dst must not exist or must be a directory
    if (args.size() > 1) {
      if (!dst.exists) {
        throw new PathNotFoundException(dst.toString());
      }

View on GitHub (pinned to 2add963021)

Solutions

1. Remove glob metacharacters from the destination and give the exact target path or directory ('/user/me/newname' or an existing dir)
2. Verify the parent directory exists and the glob is spelled correctly ('hdfs dfs -ls /user/me/backup-*')
3. If the destination must be pattern-generated, pre-resolve it in the shell before calling put

Example fix

# before
hdfs dfs -put metrics.tsv /warehouse/daily-*
# after
hdfs dfs -put metrics.tsv /warehouse/daily/2026-08-22.tsv

// if the destination is a glob, verify it matches exactly one path before copying
FileSystem fs = dstPath.getFileSystem(conf);
FileStatus[] m = fs.globStatus(dstPath);
if (m == null) { /* not a glob: fine, new path allowed */ }
else if (m.length == 0) throw new FileNotFoundException("destination glob matches nothing: " + dstPattern);

try {
  shellRun("-put", localSrc, dstPattern);
} catch (PathNotFoundException e) {
  // e.getPath() is the destination glob that matched nothing; fall back to an explicit path
  shellRun("-put", localSrc, defaultDstPath());
}

When it happens

Trigger: 'hdfs dfs -put local.txt /user/me/backup-*' when nothing matches backup-*; a mistyped destination glob like '/dat/*' instead of '/data/*'; brace or character-class patterns matching zero entries.

Common situations: Scripts templating the destination with wildcards that assume at least one file already exists; renamed directories making a previously-matching glob stale; users expecting put-to-new-file semantics while accidentally including '*' or '?' in the destination.

Related errors

• Too many matches
• No such file or directory
• Is not a directory
• File exists
• Operation not supported

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/21158eadd140844a. Report an issue: GitHub.