{"record":{"id":"863c50eb96e4b066","repo":"apache/hadoop","slug":"multiple-source-being-copied-to-a-file-targetpat","errorCode":null,"errorMessage":"Multiple source being copied to a file: {targetPath}","messagePattern":"Multiple source being copied to a file: (.+?)","errorType":"exception","errorClass":"InvalidInputException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java","lineNumber":161,"sourceCode":"\n    Path targetPath = context.getTargetPath();\n    FileSystem targetFS = targetPath.getFileSystem(getConf());\n    boolean targetExists = false;\n    boolean targetIsFile = false;\n    try {\n      targetIsFile = targetFS.getFileStatus(targetPath).isFile();\n      targetExists = true;\n    } catch (FileNotFoundException ignored) {\n    }\n    targetPath = targetFS.makeQualified(targetPath);\n    final boolean targetIsReservedRaw =\n        Path.getPathWithoutSchemeAndAuthority(targetPath).toString().\n            startsWith(HDFS_RESERVED_RAW_DIRECTORY_NAME);\n\n    //If target is a file, then source has to be single file\n    if (targetIsFile) {\n      if (context.getSourcePaths().size() > 1) {\n        throw new InvalidInputException(\"Multiple source being copied to a file: \" +\n            targetPath);\n      }\n\n      Path srcPath = context.getSourcePaths().get(0);\n      FileSystem sourceFS = srcPath.getFileSystem(getConf());\n      if (!sourceFS.isFile(srcPath)) {\n        throw new InvalidInputException(\"Cannot copy \" + srcPath +\n            \", which is not a file to \" + targetPath);\n      }\n    }\n\n    if (context.shouldAtomicCommit() && targetExists) {\n      throw new InvalidInputException(\"Target path for atomic-commit already exists: \" +\n        targetPath + \". Cannot atomic-commit to pre-existing target-path.\");\n    }\n\n    for (Path path: context.getSourcePaths()) {\n      FileSystem fs = path.getFileSystem(getConf());","sourceCodeStart":143,"sourceCodeEnd":179,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java#L143-L179","documentation":"SimpleCopyListing detected that the target path exists and is a regular file. A file target can receive exactly one source file; with multiple source paths (after glob expansion, which happens first) there is nowhere to put the remaining files, so listing aborts before job submission.","triggerScenarios":"hadoop distcp hdfs://a/f1 hdfs://a/f2 hdfs://b/existingFile; a single glob source that expands to 2+ files with a pre-existing file target, e.g. hadoop distcp 'hdfs://a/*.txt' hdfs://b/file (GlobbedCopyListing expands first, then SimpleCopyListing sees multiple sourcePaths).","commonSituations":"assuming distcp merges/overwrites multiple files into one file target; a glob matching more than expected; a target that was meant to be a directory but was created as a file by an earlier step.","solutions":["Make the target a directory: hadoop fs -rm <targetFile> && hadoop fs -mkdir <targetDir>, then re-run with the directory as target.","Or reduce the inputs to exactly one source file when the file target is intended.","Check what the target really is first: hadoop fs -ls <target> (isFile() on the target decided this)."],"exampleFix":"# before: target exists as a file but two inputs are given\nhadoop distcp -overwrite hdfs://a/f1 hdfs://a/f2 hdfs://b/t\n\n# after: target is a directory\nhadoop fs -rm hdfs://b/t && hadoop fs -mkdir hdfs://b/t\nhadoop distcp -overwrite hdfs://a/f1 hdfs://a/f2 hdfs://b/t","handlingStrategy":"validation","validationCode":"// Pre-flight: a file target may receive exactly one source file\nFileSystem tfs = targetPath.getFileSystem(conf);\nboolean targetIsFile = tfs.exists(targetPath) && tfs.getFileStatus(targetPath).isFile();\nint nSources = context.getSourcePaths().size(); // post-glob count if you expand first\nif (targetIsFile && nSources > 1) {\n  throw new InvalidInputException(\n      \"Target \" + targetPath + \" is a file but \" + nSources + \" sources given\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  copyListing.doBuildListing(listingFile, context);\n} catch (InvalidInputException e) {\n  if (e.getMessage().startsWith(\"Multiple source being copied to a file\")) {\n    // remove/rename the file target or cut inputs to one, then rebuild listing\n    fixTargetTypeAndRerun(e.getMessage());\n  } else {\n    throw e;\n  }\n}","preventionTips":["Before multi-source copies, verify the target is a directory (or absent): hadoop fs -ls <target>.","Remember globs expand before this check - one glob source can still become many files."],"tags":["distcp","target-path","file-target","input-validation"],"backgroundTag":"invalid-source-target-combination","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}