apache/hadoop · error · IOException

Unable to open file: {file}, Aborting

Error message

Unable to open file: {file}, Aborting

What it means

In open(), after changing to the parent directory and issuing RETR via retrieveFileStream, the server's reply was not 1xx (positive preliminary), so no data transfer was started. The half-open stream is closed (which also logs out and disconnects) and the IOException aborts the open.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FTPFileSystem.java:303

    }
    client.allocate(bufferSize);
    Path parent = absolute.getParent();
    // Change to parent directory on the
    // server. Only then can we read the
    // file
    // on the server by opening up an InputStream. As a side effect the working
    // directory on the server is changed to the parent directory of the file.
    // The FTP client connection is closed when close() is called on the
    // FSDataInputStream.
    client.changeWorkingDirectory(parent.toUri().getPath());
    InputStream is = client.retrieveFileStream(file.getName());
    FSDataInputStream fis = new FSDataInputStream(new FTPInputStream(is,
        client, statistics));
    if (!FTPReply.isPositivePreliminary(client.getReplyCode())) {
      // The ftpClient is an inconsistent state. Must close the stream
      // which in turn will logout and disconnect from FTP server
      fis.close();
      throw new IOException("Unable to open file: " + file + ", Aborting");
    }
    return fis;
  }

  /**
   * A stream obtained via this call must be closed before using other APIs of
   * this class or else the invocation will block.
   */
  @Override
  public FSDataOutputStream create(Path file, FsPermission permission,
      boolean overwrite, int bufferSize, short replication, long blockSize,
      Progressable progress) throws IOException {
    final FTPClient client = connect();
    Path workDir = new Path(client.printWorkingDirectory());
    Path absolute = makeAbsolute(workDir, file);
    FileStatus status;
    try {
      status = getFileStatus(client, file);

View on GitHub (pinned to 2add963021)

Solutions

1. Reproduce manually with the same credentials: curl ftp://user@host/dir/file or lftp
2. Check read permission via fs.getFileStatus(path).getPermission() (FTP READ_PERMISSION is mapped to FsAction.READ)
3. If active mode is blocked, set conf.set("fs.ftp.data.connection.mode", "PASSIVE_LOCAL_DATA_CONNECTION_MODE")
4. Check the FTP server log for the RETR reply code to get the exact refusal reason

Example fix

// before
Configuration conf = new Configuration();
conf.set("fs.ftp.host", "ftp.example.com");
FSDataInputStream in = fs.open(path); // RETR refused

// after
Configuration conf = new Configuration();
conf.set("fs.ftp.host", "ftp.example.com");
conf.set("fs.ftp.data.connection.mode", "PASSIVE_LOCAL_DATA_CONNECTION_MODE");
FSDataInputStream in = fs.open(path);

FileStatus st = fs.getFileStatus(path);
if (!st.getPermission().getUserAction().and(FsAction.READ).equals(FsAction.READ)) {
  throw new IOException("No FTP read permission for " + path);
}
FSDataInputStream in = fs.open(path);

try {
  in = fs.open(path);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Unable to open file")) {
    // server refused RETR: check permission / data-connection mode, then fail with context
    throw new IOException("FTP server refused RETR for " + path, e);
  }
  throw e;
}

When it happens

Trigger: The FTP user lacks read permission on the file; the file was removed/renamed between getFileStatus and RETR; server policy (quota, IP allowlist on data connections) rejects the download; active-mode data channel blocked so the server refuses to start the transfer.

Common situations: Firewalls blocking FTP data connections (the default fs.ftp.data.connection.mode is ACTIVE_LOCAL_DATA_CONNECTION_MODE — try PASSIVE_LOCAL_DATA_CONNECTION_MODE); files with restrictive ownership on the server; antivirus/policy engines rejecting specific file types.

Related errors

• Unable to create file: {file}, Aborting
• create(): Mkdirs failed to create: {parent}
• Could not complete transfer, Reply Code -
• mkdir of ${f} failed
• rename from {} to {} failed.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/bfab2307f57f4cb3. Report an issue: GitHub.