apache/hadoop · error · IOException

Can't replace . Target is , Source is

Error message

Can't replace {target}. Target is {targetType}, Source is {sourceType}

What it means

CopyMapper.statu­­s-checks each target path before copying: it fetches targetFS.getFileStatus(target), and if the target exists while its type (directory vs file) differs from the source's isDirectory(), distcp refuses to replace a file with a directory or vice versa. The guard fires before any data is written, so nothing on the target is modified.

Solutions

1. Identify the conflicting path from the message and remove or rename the mismatched entry on the target (hdfs dfs -rm / -mv)
2. Clean or change the destination so source and target layouts agree, then re-run with -update
3. Note that -overwrite does NOT bypass this check; the conflicting entry must be removed explicitly
4. Add a pre-flight scan comparing source/target types when reusing destinations

Example fix

# before: /dst/a is a FILE but /src/a is a DIRECTORY -> CopyMapper refuses
hadoop distcp -update hdfs://nn/src/a hdfs://nn/dst/a

# after: clear the conflicting entry, then re-run
hdfs dfs -rm hdfs://nn/dst/a
hadoop distcp -update hdfs://nn/src/a hdfs://nn/dst/a

// pre-flight: reject the copy if source/target types conflict
for (CopyListingFileStatus s : sourceList) {
  Path target = new Path(targetRoot, relativize(targetRoot, s.getPath()));
  if (targetFS.exists(target)
      && targetFS.getFileStatus(target).isDirectory() != s.isDirectory()) {
    // remove the conflicting entry or exclude this path from the listing
  }
}

catch (IOException e) {  // wraps 'Can't replace <target>. Target is X, Source is Y'
  // parse the target path from the message; delete/rename it and re-run with -update
}

When it happens

Trigger: The source path is a directory but a regular file exists at the corresponding target path (or the reverse). Typical causes: the target tree was previously populated with a different layout, a leftover file sits where a directory is expected, or the destination is reused across dissimilar sources.

Common situations: Reusing one destination directory for different source layouts; a partially populated target from an earlier tool; copying /src/a (directory) onto /dst/a where 'a' is a file from a prior flat copy; upstream processes replacing directories with files.

Related errors

• Mkdirs failed to create
• : Path is not a directory; its status is
• Source Destination both should be either file or directory
• absolutePath + " is a file"
• ACLs not supported for file system

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/bffaf1c89c013256. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java:203

            preserveXAttrs, preserveRawXattrs,
            sourceFileStatus.getChunkOffset(),
            sourceFileStatus.getChunkLength());
      } catch (FileNotFoundException e) {
        throw new IOException(new RetriableFileCopyCommand.CopyReadException(e));
      }

      FileStatus targetStatus = null;

      try {
        targetStatus = targetFS.getFileStatus(target);
      } catch (FileNotFoundException ignore) {
        if (LOG.isDebugEnabled())
          LOG.debug("Path could not be found: " + target, ignore);
      }

      if (targetStatus != null &&
          (targetStatus.isDirectory() != sourceCurrStatus.isDirectory())) {
        throw new IOException("Can't replace " + target + ". Target is " +
            getFileType(targetStatus) + ", Source is " + getFileType(sourceCurrStatus));
      }

      if (sourceCurrStatus.isDirectory()) {
        createTargetDirsWithRetry(description, target, context, sourceStatus,
            sourceFS);
        return;
      }

      FileAction action = checkUpdate(sourceFS, sourceCurrStatus, target,
          targetStatus);

      Path tmpTarget = target;
      if (action == FileAction.SKIP) {
        LOG.info("Skipping copy of " + sourceCurrStatus.getPath()
                 + " to " + target);
        updateSkipCounters(context, sourceCurrStatus);
        context.write(null, new Text("SKIP: " + sourceCurrStatus.getPath()));

View on GitHub (pinned to 2add963021)