apache/hadoop · error · IOException

Invalid path string

Error message

Invalid path string 

What it means

PathData.normalizePath's Windows branch (PathData.java:523) throws IOException('Invalid path string <path>') when the input matches windowsNonUriAbsolutePath1 (backslash-separated drive-absolute path like D:\dir) yet contains at least one forward slash. This is the same mixed-separator rule as checkIfSchemeInferredFromPath, applied when normalizing Windows paths to file: URIs for display/comparison.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/PathData.java:523

   *    @param pathString Path string supplied by the user.
   *    @return normalized absolute path string. Returns the input string
   *            if it is not a Windows absolute path.
   */
  private static String normalizeWindowsPath(String pathString)
  throws IOException
  {
    if (!Path.WINDOWS) {
      return pathString;
    }

    boolean slashed =
        ((pathString.length() >= 1) && (pathString.charAt(0) == '/'));

    // Is it a backslash-separated absolute path?
    if (windowsNonUriAbsolutePath1.matcher(pathString).find()) {
      // Forward slashes disallowed in a backslash-separated path.
      if (pathString.indexOf('/') != -1) {
        throw new IOException("Invalid path string " + pathString);
      }

      pathString = pathString.replace('\\', '/');
      return "file:" + (slashed ? "" : "/") + pathString;
    }

    // Is it a forward slash-separated absolute path?
    if (windowsNonUriAbsolutePath2.matcher(pathString).find()) {
      return "file:" + (slashed ? "" : "/") + pathString;
    }

    // Is it a backslash-separated relative file path (no scheme and
    // no drive-letter specifier)?
    if ((pathString.indexOf(':') == -1) && (pathString.indexOf('\\') != -1)) {
      pathString = pathString.replace('\\', '/');
    }

    return pathString;

View on GitHub (pinned to 2add963021)

Solutions

1. Use exclusively forward slashes on Windows paths: 'D:/logs/current' (matches windowsNonUriAbsolutePath2 and normalizes cleanly)
2. Pre-normalize inputs: pathString = pathString.replace('\\', '/') before passing to shell commands
3. Pass full URIs ('file:///D:/logs/current') so the Windows inference/normalization path is bypassed

Example fix

# before
hadoop fs -ls D:\logs/2026\aug  # mixed separators
# ls: Invalid path string D:\logs/2026\aug

# after
hadoop fs -ls 'D:/logs/2026/aug'

String norm = raw.contains(":\\") || raw.contains("\\") ? raw.replace('\\', '/') : raw;
// pass norm to shell commands / Path construction on Windows

catch (IOException e) when 'Invalid path string' -> re-issue with uniform '/' separators or a file: URI

When it happens

Trigger: Any hadoop fs shell path handling that routes through normalizePath on Windows (Path.WINDOWS) with strings like 'D:\logs/current' — e.g., command output formatting, path comparisons in copy/move target resolution. The throw happens before backslash-to-slash replacement can run.

Common situations: Windows clients building paths by concatenating a backslash base with forward-slash relative parts; scripts generated by tools with different separator conventions; paths logged and re-fed into commands after partial normalization.

Related errors

• Invalid path string ${pathString}
• {}
• Cannot create a file whose name looks like a directory: '%s'
• A file cannot be created in root.
• Rename to subdir is forbidden

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/fa708a7ca63a634b. Report an issue: GitHub.