apache/hadoop · error · PathNotFoundException

No such file or directory

Error message

No such file or directory

What it means

Command.expandArgument throws PathNotFoundException('No such file or directory') when PathData.expandAsGlob returns zero matches for an argument containing glob metacharacters. FsShell treats a matched-nothing glob as an error rather than an empty expansion, so the command aborts before processing.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java:268

        displayError(e);
      }
    }
    return expandedArgs;
  }

  /**
   * Expand the given argument into a list of {@link PathData} objects.
   * The default behavior is to expand globs.  Commands may override to
   * perform other expansions on an argument.
   * @param arg string pattern to expand
   * @return list of {@link PathData} objects
   * @throws IOException if anything goes wrong...
   */
  protected List<PathData> expandArgument(String arg) throws IOException {
    PathData[] items = PathData.expandAsGlob(arg, getConf());
    if (items.length == 0) {
      // it's a glob that failed to match
      throw new PathNotFoundException(arg);
    }
    return Arrays.asList(items);
  }

  /**
   *  Processes the command's list of expanded arguments.
   *  {@link #processArgument(PathData)} will be invoked with each item
   *  in the list.  The loop catches IOExceptions, increments the error
   *  count, and displays the exception.
   *  @param args a list of {@link PathData} to process
   *  @throws IOException if anything goes wrong... 
   */
  protected void processArguments(LinkedList<PathData> args)
  throws IOException {
    for (PathData arg : args) {
      try {
        processArgument(arg);
      } catch (IOException e) {

View on GitHub (pinned to 2add963021)

Solutions

1. Verify the pattern matches by listing its parent ('hadoop fs -ls /data/2026/08/') and adjust the expression.
2. In code, use FileSystem.globStatus(pattern) and branch on an empty result instead of letting a shell command fail.
3. Check the URI scheme and authority spelling in the glob; a wrong host can only ever match nothing.

Example fix

# before
hadoop fs -ls /data/2026/08/*.json   # throws when partition empty

# after
if hadoop fs -test -d /data/2026/08; then
  hadoop fs -ls /data/2026/08/*.json || echo "no .json files yet"
fi

FileStatus[] matches = fs.globStatus(pattern);
if (matches == null || matches.length == 0) {
  LOG.warn("glob matched nothing: {}", pattern);
  return; // or raise a domain-specific error
}

try {
  runCommand(new String[]{"-ls", pattern});
} catch (PathNotFoundException e) {
  // glob matched nothing: data not landed yet or pattern wrong
}

When it happens

Trigger: Any fs shell command with a non-matching glob: 'hadoop fs -ls "sftp://host/*.tmp"' when no such file exists, 'hadoop fs -rm /data/2026/08/*' before the daily partition has been written, or a glob whose scheme/authority is misspelled so nothing can match.

Common situations: Scheduled jobs whose glob targets date-partitioned data that has not landed yet; wrong authority or scheme in the glob; case-sensitivity mismatches in the pattern; globs written for a different cluster layout.

Related errors

• No such file or directory
• Illegal option {}
• Not enough arguments: expected {} but got {}
• Too many arguments: expected {} but got {}
• Too many matches

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4b7abfc703aa857a. Report an issue: GitHub.