apache/hadoop · error · IllegalArgumentException

Invalid DFS directory name {0}

Error message

Invalid DFS directory name {0}

What it means

HttpFSFileSystem.setWorkingDirectory() validates the candidate directory with DFSUtilClient.isValidName(), which rejects relative paths and any path containing ':', '//', or '.'/'..' components. When validation fails it throws IllegalArgumentException (unchecked) with the offending path, and the working directory is left unchanged. This mirrors HDFS name rules on the WebHDFS client because relative paths must resolve against a DFS-legal working directory.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSFileSystem.java:832

          .getBytes(StandardCharsets.UTF_8);
    }
    // Parse the remainingEntries boolean into hasMore
    final long remainingEntries = (Long) listing.get(REMAINING_ENTRIES_JSON);
    final boolean hasMore = remainingEntries > 0 ? true : false;
    return new DirectoryEntries(statuses, newToken, hasMore);
  }

  /**
   * Set the current working directory for the given file system. All relative
   * paths will be resolved relative to it.
   *
   * @param newDir new directory.
   */
  @Override
  public void setWorkingDirectory(Path newDir) {
    String result = newDir.toUri().getPath();
    if (!DFSUtilClient.isValidName(result)) {
      throw new IllegalArgumentException(
          "Invalid DFS directory name " + result);
    }
    workingDir = newDir;
  }

  /**
   * Get the current working directory for the given file system
   *
   * @return the directory pathname
   */
  @Override
  public Path getWorkingDirectory() {
    if (workingDir == null) {
      workingDir = getHomeDirectory();
    }
    return workingDir;
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Pass an absolute normalized path: new Path("/user/me/data/warehouse")
2. Normalize before calling: path.makeQualified(fs.getUri(), fs.getWorkingDirectory()) then validate
3. Strip scheme/authority first when you hold a qualified path: new Path(path.toUri().getPath())
4. Validate with DFSUtilClient.isValidName(path.toUri().getPath()) and surface a clear error to the caller before touching the FileSystem

Example fix

// before
fs.setWorkingDirectory(new Path("data/warehouse"));
// after
Path wd = new Path("/user/me/data/warehouse");
if (!DFSUtilClient.isValidName(wd.toUri().getPath())) {
  throw new IllegalArgumentException("Invalid working directory: " + wd);
}
fs.setWorkingDirectory(wd);

Path candidate = new Path(rawInput).makeQualified(fs.getUri(), fs.getWorkingDirectory());
if (!org.apache.hadoop.hdfs.DFSUtilClient.isValidName(candidate.toUri().getPath())) {
  throw new IllegalArgumentException("Refusing invalid DFS working directory: " + rawInput);
}
fs.setWorkingDirectory(candidate);

static boolean isValidDfsWorkingDir(Path p) {
  String s = p.toUri().getPath();
  return s != null && s.startsWith("/") && org.apache.hadoop.hdfs.DFSUtilClient.isValidName(s);
}

try {
  fs.setWorkingDirectory(dir);
} catch (IllegalArgumentException e) {
  Path qualified = new Path(dir.toUri().getPath()).makeQualified(fs.getUri(), fs.getWorkingDirectory());
  fs.setWorkingDirectory(qualified); // one normalization retry, then fail loudly
}

When it happens

Trigger: fs.setWorkingDirectory(new Path("data/warehouse")) (relative), new Path("hdfs://nn/user") (contains ':' in authority when fed as path), new Path("/a//b") (empty component), or new Path("/a/./b") / "../x" constructed from raw user input.

Common situations: Code ported from LocalFileSystem where a relative cwd is accepted; building a Path from unvalidated CLI/env input; passing a fully-qualified URI form Path to a method that expects an absolute DFS path.

Related errors

• Invalid scheme [{0}] it should be 'webhdfs' or 'swebhdfs'
• Bad rule definition: {bad_lines}
• Invalid parameter value: destination = "{str}" is not an abs
• Failed to parse "{str}" for the parameter {varName}. The va
• Invalid value: "{str}" does not belong to the domain {domain

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b909af2c34e5d981. Report an issue: GitHub.