apache/hadoop · error · IOException

Source path %s does not exist

Error message

Source path %s does not exist

What it means

Thrown by SFTPFileSystem.rename (private overload) when the source path of a rename over the sftp:// scheme does not exist on the SFTP server. Existence is verified with an lstat against makeAbsolute(workDir, src), where workDir is the SFTP channel's pwd(). Unlike most FileSystem implementations that return false for a missing source, the SFTP adapter throws a plain IOException.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPFileSystem.java:476

   * @param channel
   * @param src
   * @param dst
   * @return rename successful?
   * @throws IOException
   */
  private boolean rename(ChannelSftp channel, Path src, Path dst)
      throws IOException {
    Path workDir;
    try {
      workDir = new Path(channel.pwd());
    } catch (SftpException e) {
      throw new IOException(e);
    }
    Path absoluteSrc = makeAbsolute(workDir, src);
    Path absoluteDst = makeAbsolute(workDir, dst);

    if (!exists(channel, absoluteSrc)) {
      throw new IOException(String.format(E_SPATH_NOTEXIST, src));
    }
    if (exists(channel, absoluteDst)) {
      throw new IOException(String.format(E_DPATH_EXIST, dst));
    }
    boolean renamed = true;
    try {
      final String previousCwd = channel.pwd();
      channel.cd("/");
      channel.rename(src.toUri().getPath(), dst.toUri().getPath());
      channel.cd(previousCwd);
    } catch (SftpException e) {
      renamed = false;
    }
    return renamed;
  }

  @Override
  public void initialize(URI uriInfo, Configuration conf) throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

1. Check fs.exists(src) before calling rename and treat a missing source as a no-op instead of an error.
2. Use fully-qualified absolute sftp:// paths (e.g. sftp://host/export/data/file) so resolution never depends on the channel working directory.
3. If you rely on the boolean contract of FileSystem.rename, wrap the call in try/catch (IOException) and map the 'Source path ... does not exist' failure to a false return.

Example fix

// before
fs.rename(new Path("data/file.csv"), new Path("data/file.done")); // relative -> resolved against SFTP pwd

// after
Path src = new Path("sftp://myhost/export/data/file.csv");
Path dst = new Path("sftp://myhost/export/data/file.done");
if (fs.exists(src)) {
  fs.rename(src, dst);
}

if (!fs.exists(src)) {
  LOG.warn("rename skipped, source missing: {}", src);
  return false;
}

try {
  fs.rename(src, dst);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("does not exist")) {
    // source vanished: treat as already moved
  } else {
    throw e;
  }
}

When it happens

Trigger: Calling fs.rename(src, dst) on an sftp:// URI when exists(channel, absoluteSrc) is false: the source was typo'd, already moved/deleted, or a relative path was resolved against the SFTP server's login directory (channel pwd) instead of the directory the caller assumed.

Common situations: Using relative paths like data/file when the channel's pwd is the user's SFTP home directory; the file was consumed by another process between the check and the rename; code ported from LocalFileSystem or HDFS that expects rename() to return false instead of throwing; logged in as a different SFTP user whose home differs.

Related errors

• Destination path %s already exist, cannot rename!
• Cannot overwrite an existing file: %s
• Cannot rename because path does not exist: %s
• Rename unsuccessful : '%s' to '%s'
• The source {src} and destination {dst} are the same

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b9864ff08948c6ca. Report an issue: GitHub.