apache/hadoop · error · IOException

Can't replace {target}. Target is {targetType}, Source is {s

Error message

Can't replace {target}. Target is {targetType}, Source is {sourceType}

What it means

CopyMapper.statu­­s-checks each target path before copying: it fetches targetFS.getFileStatus(target), and if the target exists while its type (directory vs file) differs from the source's isDirectory(), distcp refuses to replace a file with a directory or vice versa. The guard fires before any data is written, so nothing on the target is modified.

Source

Thrown at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java:203

            preserveXAttrs, preserveRawXattrs,
            sourceFileStatus.getChunkOffset(),
            sourceFileStatus.getChunkLength());
      } catch (FileNotFoundException e) {
        throw new IOException(new RetriableFileCopyCommand.CopyReadException(e));
      }

      FileStatus targetStatus = null;

      try {
        targetStatus = targetFS.getFileStatus(target);
      } catch (FileNotFoundException ignore) {
        if (LOG.isDebugEnabled())
          LOG.debug("Path could not be found: " + target, ignore);
      }

      if (targetStatus != null &&
          (targetStatus.isDirectory() != sourceCurrStatus.isDirectory())) {
        throw new IOException("Can't replace " + target + ". Target is " +
            getFileType(targetStatus) + ", Source is " + getFileType(sourceCurrStatus));
      }

      if (sourceCurrStatus.isDirectory()) {
        createTargetDirsWithRetry(description, target, context, sourceStatus,
            sourceFS);
        return;
      }

      FileAction action = checkUpdate(sourceFS, sourceCurrStatus, target,
          targetStatus);

      Path tmpTarget = target;
      if (action == FileAction.SKIP) {
        LOG.info("Skipping copy of " + sourceCurrStatus.getPath()
                 + " to " + target);
        updateSkipCounters(context, sourceCurrStatus);
        context.write(null, new Text("SKIP: " + sourceCurrStatus.getPath()));

View on GitHub (pinned to 2add963021)

Solutions

1. Identify the conflicting path from the message and remove or rename the mismatched entry on the target (hdfs dfs -rm / -mv)
2. Clean or change the destination so source and target layouts agree, then re-run with -update
3. Note that -overwrite does NOT bypass this check; the conflicting entry must be removed explicitly
4. Add a pre-flight scan comparing source/target types when reusing destinations

Example fix

# before: /dst/a is a FILE but /src/a is a DIRECTORY -> CopyMapper refuses
hadoop distcp -update hdfs://nn/src/a hdfs://nn/dst/a

# after: clear the conflicting entry, then re-run
hdfs dfs -rm hdfs://nn/dst/a
hadoop distcp -update hdfs://nn/src/a hdfs://nn/dst/a

// pre-flight: reject the copy if source/target types conflict
for (CopyListingFileStatus s : sourceList) {
  Path target = new Path(targetRoot, relativize(targetRoot, s.getPath()));
  if (targetFS.exists(target)
      && targetFS.getFileStatus(target).isDirectory() != s.isDirectory()) {
    // remove the conflicting entry or exclude this path from the listing
  }
}

catch (IOException e) {  // wraps 'Can't replace <target>. Target is X, Source is Y'
  // parse the target path from the message; delete/rename it and re-run with -update
}

When it happens

Trigger: The source path is a directory but a regular file exists at the corresponding target path (or the reverse). Typical causes: the target tree was previously populated with a different layout, a leftover file sits where a directory is expected, or the destination is reused across dissimilar sources.

Common situations: Reusing one destination directory for different source layouts; a partially populated target from an earlier tool; copying /src/a (directory) onto /dst/a where 'a' is a file from a prior flat copy; upstream processes replacing directories with files.

Related errors

• Mkdirs failed to create {} (exists={}, cwd={})
• Source ${src} Destination ${dst} both should be either file
• {operation}: Path is not a directory; its status is :{status
• absolutePath + " is a file"
• f.toString()

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/bffaf1c89c013256. Report an issue: GitHub.