apache/hadoop · error · InvalidInputException

Cannot copy {srcPath}, which is not a file to {targetPath}

Error message

Cannot copy {srcPath}, which is not a file to {targetPath}

What it means

The target exists and is a file, there is exactly one source path, but sourceFS.isFile(srcPath) is false - the single source is a directory (or something that is not a regular file). File-to-file copy requires the source to be a file; directory-onto-file is rejected at listing time.

Source

Thrown at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java:168

      targetExists = true;
    } catch (FileNotFoundException ignored) {
    }
    targetPath = targetFS.makeQualified(targetPath);
    final boolean targetIsReservedRaw =
        Path.getPathWithoutSchemeAndAuthority(targetPath).toString().
            startsWith(HDFS_RESERVED_RAW_DIRECTORY_NAME);

    //If target is a file, then source has to be single file
    if (targetIsFile) {
      if (context.getSourcePaths().size() > 1) {
        throw new InvalidInputException("Multiple source being copied to a file: " +
            targetPath);
      }

      Path srcPath = context.getSourcePaths().get(0);
      FileSystem sourceFS = srcPath.getFileSystem(getConf());
      if (!sourceFS.isFile(srcPath)) {
        throw new InvalidInputException("Cannot copy " + srcPath +
            ", which is not a file to " + targetPath);
      }
    }

    if (context.shouldAtomicCommit() && targetExists) {
      throw new InvalidInputException("Target path for atomic-commit already exists: " +
        targetPath + ". Cannot atomic-commit to pre-existing target-path.");
    }

    for (Path path: context.getSourcePaths()) {
      FileSystem fs = path.getFileSystem(getConf());
      if (!fs.exists(path)) {
        throw new InvalidInputException(path + " doesn't exist");
      }
      if (Path.getPathWithoutSchemeAndAuthority(path).toString().
          startsWith(HDFS_RESERVED_RAW_DIRECTORY_NAME)) {
        if (!targetIsReservedRaw) {
          final String msg = "The source path '" + path + "' starts with " +

View on GitHub (pinned to 2add963021)

Solutions

If you meant dir-to-dir: delete or rename the file target, then copy onto a nonexistent or directory target.
If you meant file-to-file: point the source at the actual file, e.g. hdfs://a/somedir/file.ext.
Verify both endpoints: hadoop fs -ls on source and target to see which one has the wrong type.

Example fix

# before: source is a directory, target is a file
hadoop distcp hdfs://a/somedir hdfs://b/existing.bin

# after: copy the file, or target a directory
hadoop distcp hdfs://a/somedir/actual.bin hdfs://b/existing.bin

Defensive patterns

Strategy: validation

Validate before calling

// Pre-flight: file-to-file copy requires the single source to be a file
FileSystem tfs = targetPath.getFileSystem(conf);
if (tfs.exists(targetPath) && tfs.getFileStatus(targetPath).isFile()
    && context.getSourcePaths().size() == 1) {
  Path src = context.getSourcePaths().get(0);
  FileSystem sfs = src.getFileSystem(conf);
  if (!sfs.isFile(src)) {
    throw new InvalidInputException(
        "Source " + src + " is not a file but target " + targetPath + " is");
  }
}

Try / catch

try {
  copyListing.doBuildListing(listingFile, context);
} catch (InvalidInputException e) {
  if (e.getMessage().startsWith("Cannot copy") && e.getMessage().contains("which is not a file")) {
    // point the source at the actual file, or make the target a directory
    fixSourceOrTargetType(e.getMessage());
  } else {
    throw e;
  }
}

Prevention

Check hadoop fs -ls on both endpoints when copying onto an existing path.
Include the filename in the source path when the target is a file.

When it happens

Trigger: hadoop distcp hdfs://a/somedir hdfs://b/existingFile; a source that is a symlink-to-directory or a mount point resolving as a directory; intending to copy one file but pointing at its parent directory.

Common situations: forgetting the filename portion of the source path; targets auto-created as files by earlier tools; globs that collapsed to a directory.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3fb17a4253b7d7ce. Report an issue: GitHub.