apache/hadoop · error · IOException

{src} should be a directory.

Error message

{src} should be a directory.

What it means

FedBalance's DistCpProcedure.preCheck() validates the source before the balance procedure starts: fedbalance can only balance directories, because it recursively walks src, enables snapshots on it, and diff-copies it to dst. If srcFs.getFileStatus(src) exists but is not a directory, the job aborts immediately with this IOException. It is a precondition failure raised before any data is moved.

Source

Thrown at hadoop-tools/hadoop-federation-balance/src/main/java/org/apache/hadoop/tools/fedbalance/DistCpProcedure.java:185

      return false;
    case FINAL_DISTCP:
      finalDistCp();
      return false;
    case FINISH:
      finish();
      return true;
    default:
      throw new IOException("Unexpected stage=" + stage);
    }
  }

  /**
   * Pre check of src and dst.
   */
  void preCheck() throws IOException {
    FileStatus status = srcFs.getFileStatus(src);
    if (!status.isDirectory()) {
      throw new IOException(src + " should be a directory.");
    }
    if (dstFs.exists(dst)) {
      throw new IOException(dst + " already exists.");
    }
    if (srcFs.exists(new Path(src, HdfsConstants.DOT_SNAPSHOT_DIR))) {
      throw new IOException(src + " shouldn't enable snapshot.");
    }
    updateStage(Stage.INIT_DISTCP);
  }

  /**
   * The initial distcp. Copying src to dst.
   */
  void initDistCp() throws IOException, RetryException {
    RunningJobStatus job = getCurrentJob();
    if (job != null) {
      // the distcp has been submitted.
      if (job.isComplete()) {

View on GitHub (pinned to 2add963021)

Solutions

1. Point -src at the directory that contains the data (e.g. the parent directory of the file you named).
2. Verify before submitting: 'hdfs dfs -test -d <src> && echo dir || echo not-a-dir'.
3. Re-check the exact -src string for typos; note this error means the path EXISTS but is a file (a wrong path normally gives NoSuchFileException instead).

Example fix

# before
hdfs fedbalance -src hdfs://ns1/user/alice/part-00000 -dst hdfs://ns2/balance/alice

# after
hdfs fedbalance -src hdfs://ns1/user/alice -dst hdfs://ns2/balance/alice

Path src = new Path("hdfs://ns1/data");
FileSystem srcFs = src.getFileSystem(conf);
FileStatus st = srcFs.getFileStatus(src);
if (!st.isDirectory()) {
  throw new IllegalArgumentException(src + " is a file; pass its parent directory to fedbalance");
}

When it happens

Trigger: Submitting a fedbalance job (hdfs fedbalance) whose -src argument resolves to a regular file rather than a directory: status.isDirectory() is false in preCheck(). Also a symlink whose target is a file, or a path string with a trailing slash that still resolves to a file.

Common situations: Balancing a single file instead of its parent directory; scripts parameterized with file paths from an upstream stage; -src copied from an 'hdfs dfs -ls' row where a file path was picked instead of the directory.

Related errors

• {dst} already exists.
• {src} shouldn't enable snapshot.
• DistCp failed. jobId={jobId} failure={failureInfo}
• Final DistCp failed. Failure: {failureInfo}
• The dst path={dst} already exists. The admin should delete i

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/cf7e243562b596fb. Report an issue: GitHub.