apache/hadoop · error · IllegalArgumentException

Not a local path:

Error message

Not a local path: 

What it means

PathData.toFile() (PathData.java:496) returns a java.io.File for the wrapped path, but only if the PathData's FileSystem is a LocalFileSystem; otherwise it throws IllegalArgumentException('Not a local path: <path>'). It exists because shell commands like get/copyToLocal must obtain a local File handle from a PathData, and an HDFS (or S3A, etc.) FileSystem has no local File representation.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/PathData.java:496

      }
      return decodedRemainder;
    } else {
      StringBuilder buffer = new StringBuilder();
      buffer.append(scheme)
          .append(":")
          .append(decodedRemainder);
      return buffer.toString();
    }
  }
  
  /**
   * Get the path to a local file
   * @return File representing the local path
   * @throws IllegalArgumentException if this.fs is not the LocalFileSystem
   */
  public File toFile() {
    if (!(fs instanceof LocalFileSystem)) {
       throw new IllegalArgumentException("Not a local path: " + path);
    }
    return ((LocalFileSystem)fs).pathToFile(path);
  }

  /** Normalize the given Windows path string. This does the following:
   *    1. Adds "file:" scheme for absolute paths.
   *    2. Ensures the scheme-specific part starts with '/' per RFC2396.
   *    3. Replaces backslash path separators with forward slashes.
   *    @param pathString Path string supplied by the user.
   *    @return normalized absolute path string. Returns the input string
   *            if it is not a Windows absolute path.
   */
  private static String normalizeWindowsPath(String pathString)
  throws IOException
  {
    if (!Path.WINDOWS) {
      return pathString;
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Qualify the local side with the file: scheme: 'hadoop fs -get /hdfs/path file:///tmp/local/path' (or a relative path, which resolves locally)
2. In code, construct the local PathData against the local filesystem: new PathData(new Path('file:///tmp/f'), conf) or FileSystem.getLocal(conf)
3. Before calling toFile(), branch on 'pd.fs instanceof LocalFileSystem' and handle the remote case separately (copy via IOUtils)

Example fix

// before
File f = pathData.toFile(); // fs is hdfs -> IllegalArgumentException

// after
if (pathData.fs instanceof LocalFileSystem) {
  File f = pathData.toFile();
} else {
  Path tmp = new Path('file:///tmp/staging', pathData.path.getName());
  pathData.fs.copyToLocalFile(pathData.path, tmp);
  File f = new File(tmp.toUri().getPath());
}

if (!(pd.fs instanceof LocalFileSystem)) {
  Path local = new Path("file:///tmp/staging", pd.path.getName());
  pd.fs.copyToLocalFile(pd.path, local);
  // proceed with local.toUri().getPath()
}

boolean isLocalPathData(PathData pd) {
  return pd.fs instanceof LocalFileSystem;
}

catch (IllegalArgumentException e) when 'Not a local path' -> qualify the path with file:/// (or make it relative) so it resolves on the local filesystem, then retry

When it happens

Trigger: Calling toFile() (directly or via shell code paths such as CopyCommands' local-side IO that calls source.toFile().toPath()) on a PathData whose fs resolved to a non-local scheme: hdfs://, s3a://, viewfs across clusters. Typical trigger: the 'local' argument of a command accidentally resolving against the default (HDFS) filesystem because it lacked a file:// scheme or the fs.defaultFS is HDFS.

Common situations: Running commands where the local side is given as a bare absolute path '/tmp/file' while fs.defaultFS is hdfs://... so it resolves to HDFS, not the local FS; code that holds a PathData for a remote path and calls toFile(); custom FsCommand subclasses reusing toFile for convenience.

Related errors

• Path must be absolute: " + path
• Append is not supported by BaiduBosFileSystem
• min sleep time must be positive number and max sleep time mu
• Copy destination must be different from source for %s.
• GCS path must not have consecutive '/' characters: '%s'

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1353f6df5ef3c972. Report an issue: GitHub.