{"record":{"id":"37cb0f768113a9dc","repo":"apache/hadoop","slug":"the-target-path-targetpath-starts-with-reser","errorCode":null,"errorMessage":"The target path '{targetPath}' starts with /.reserved/raw but the source path '{path}' does not. Either all or none of the paths must have this prefix.","messagePattern":"The target path '(.+?)' starts with /\\.reserved/raw but the source path '(.+?)' does not\\. Either all or none of the paths must have this prefix\\.","errorType":"exception","errorClass":"InvalidInputException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java","lineNumber":197,"sourceCode":"      FileSystem fs = path.getFileSystem(getConf());\n      if (!fs.exists(path)) {\n        throw new InvalidInputException(path + \" doesn't exist\");\n      }\n      if (Path.getPathWithoutSchemeAndAuthority(path).toString().\n          startsWith(HDFS_RESERVED_RAW_DIRECTORY_NAME)) {\n        if (!targetIsReservedRaw) {\n          final String msg = \"The source path '\" + path + \"' starts with \" +\n              HDFS_RESERVED_RAW_DIRECTORY_NAME + \" but the target path '\" +\n              targetPath + \"' does not. Either all or none of the paths must \" +\n              \"have this prefix.\";\n          throw new InvalidInputException(msg);\n        }\n      } else if (targetIsReservedRaw) {\n        final String msg = \"The target path '\" + targetPath + \"' starts with \" +\n                HDFS_RESERVED_RAW_DIRECTORY_NAME + \" but the source path '\" +\n                path + \"' does not. Either all or none of the paths must \" +\n                \"have this prefix.\";\n        throw new InvalidInputException(msg);\n      }\n    }\n\n    if (targetIsReservedRaw) {\n      context.setPreserveRawXattrs(true);\n      getConf().setBoolean(DistCpConstants.CONF_LABEL_PRESERVE_RAWXATTRS, true);\n    }\n\n    /* This is requires to allow map tasks to access each of the source\n       clusters. This would retrieve the delegation token for each unique\n       file system and add them to job's private credential store\n     */\n    Credentials credentials = getCredentials();\n    if (credentials != null) {\n      Path[] inputPaths = context.getSourcePaths()\n          .toArray(new Path[1]);\n      TokenCache.obtainTokensForNamenodes(credentials, inputPaths, getConf());\n    }","sourceCodeStart":179,"sourceCodeEnd":215,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java#L179-L215","documentation":"The inverse raw-path mismatch: the target starts with /.reserved/raw but this source path does not. DistCp requires all-or-none use of the raw prefix; copying a normal source into a raw destination would produce a raw view without the corresponding raw xattrs, so listing aborts.","triggerScenarios":"hadoop distcp hdfs://nn/ez/src hdfs://nn2/.reserved/raw/backup; restore jobs where the destination was templated with the raw prefix but one source (of several, or one line in the -f file) lacks it.","commonSituations":"disaster-recovery restores into a raw-prefixed backup area; -f listings mixing raw and normal paths; incremental scripts where the target changed to raw but sources were not updated.","solutions":["Add /.reserved/raw to every source: hdfs://nn/.reserved/raw/ez/src hdfs://nn2/.reserved/raw/backup.","Or remove the prefix from the target if raw semantics are not intended.","Grep the -f listing file for inconsistent prefixes: grep -c '^/.reserved/raw\\|hdfs://[^/]*/.reserved/raw' list.txt vs total lines."],"exampleFix":"# before: normal source, raw target\nhadoop distcp hdfs://nn/zone/src hdfs://nn2/.reserved/raw/backup\n\n# after: both sides raw\nhadoop distcp hdfs://nn/.reserved/raw/zone/src hdfs://nn2/.reserved/raw/backup","handlingStrategy":"validation","validationCode":"// Pre-flight: same check as source-side; catch a raw target with normal sources\nString RAW = \"/.reserved/raw\";\nboolean targetRaw = Path.getPathWithoutSchemeAndAuthority(targetPath).toString().startsWith(RAW);\nfor (Path p : context.getSourcePaths()) {\n  boolean srcRaw = Path.getPathWithoutSchemeAndAuthority(p).toString().startsWith(RAW);\n  if (targetRaw && !srcRaw) {\n    throw new InvalidInputException(\n        \"Target is raw but source \" + p + \" is not\");\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  copyListing.doBuildListing(listingFile, context);\n} catch (InvalidInputException e) {\n  if (e.getMessage().contains(\"Either all or none of the paths must have this prefix\")) {\n    // either strip /.reserved/raw from the target or add it to all sources\n    alignRawPrefixesAndRerun();\n  } else {\n    throw e;\n  }\n}","preventionTips":["Template restore jobs so the raw prefix is applied to all sources and the target consistently.","Audit -f listings for mixed prefixes: count lines matching /.reserved/raw vs total."],"tags":["distcp","reserved-raw","encryption-zone","input-validation","hdfs"],"backgroundTag":"reserved-raw-prefix-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}