{"record":{"id":"1193e4764299921e","repo":"apache/hadoop","slug":"the-source-path-path-starts-with-reserved-ra","errorCode":null,"errorMessage":"The source path '{path}' starts with /.reserved/raw but the target path '{targetPath}' does not. Either all or none of the paths must have this prefix.","messagePattern":"The source path '(.+?)' starts with /\\.reserved/raw but the target path '(.+?)' does not\\. Either all or none of the paths must have this prefix\\.","errorType":"exception","errorClass":"InvalidInputException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java","lineNumber":190,"sourceCode":"\n    if (context.shouldAtomicCommit() && targetExists) {\n      throw new InvalidInputException(\"Target path for atomic-commit already exists: \" +\n        targetPath + \". Cannot atomic-commit to pre-existing target-path.\");\n    }\n\n    for (Path path: context.getSourcePaths()) {\n      FileSystem fs = path.getFileSystem(getConf());\n      if (!fs.exists(path)) {\n        throw new InvalidInputException(path + \" doesn't exist\");\n      }\n      if (Path.getPathWithoutSchemeAndAuthority(path).toString().\n          startsWith(HDFS_RESERVED_RAW_DIRECTORY_NAME)) {\n        if (!targetIsReservedRaw) {\n          final String msg = \"The source path '\" + path + \"' starts with \" +\n              HDFS_RESERVED_RAW_DIRECTORY_NAME + \" but the target path '\" +\n              targetPath + \"' does not. Either all or none of the paths must \" +\n              \"have this prefix.\";\n          throw new InvalidInputException(msg);\n        }\n      } else if (targetIsReservedRaw) {\n        final String msg = \"The target path '\" + targetPath + \"' starts with \" +\n                HDFS_RESERVED_RAW_DIRECTORY_NAME + \" but the source path '\" +\n                path + \"' does not. Either all or none of the paths must \" +\n                \"have this prefix.\";\n        throw new InvalidInputException(msg);\n      }\n    }\n\n    if (targetIsReservedRaw) {\n      context.setPreserveRawXattrs(true);\n      getConf().setBoolean(DistCpConstants.CONF_LABEL_PRESERVE_RAWXATTRS, true);\n    }\n\n    /* This is requires to allow map tasks to access each of the source\n       clusters. This would retrieve the delegation token for each unique\n       file system and add them to job's private credential store","sourceCodeStart":172,"sourceCodeEnd":208,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java#L172-L208","documentation":"DistCp requires the /.reserved/raw prefix to be used consistently on source and target. A source under /.reserved/raw exposes the raw block-level view including raw* xattrs (used for encryption-zone key material), while a normal target cannot store them; copying in that direction would silently drop or corrupt raw xattrs, so listing aborts.","triggerScenarios":"hadoop distcp hdfs://nn/.reserved/raw/ez/src hdfs://nn2/backup - the source carries the raw prefix, the target does not. Any -sync or plain copy mixing one raw-prefixed source with a non-raw target.","commonSituations":"backing up encryption zones: operators quote the raw source path but a normal destination; scripts templating the target without the raw prefix; partially migrated runbooks where only one side was updated.","solutions":["Add the same prefix to the target: hadoop distcp hdfs://nn/.reserved/raw/ez/src hdfs://nn2/.reserved/raw/backup.","Or drop /.reserved/raw from the source if you intentionally want the normal (encrypted/transparent) view copied.","Verify every source path when using globs or -f listings - all of them must match the target's raw-ness."],"exampleFix":"# before: raw source, normal target\nhadoop distcp hdfs://nn/.reserved/raw/zone/src hdfs://nn2/backup\n\n# after: both sides raw\nhadoop distcp hdfs://nn/.reserved/raw/zone/src hdfs://nn2/.reserved/raw/backup","handlingStrategy":"validation","validationCode":"// Pre-flight: raw prefix must match between every source and the target\nString RAW = \"/.reserved/raw\";\nString tp = Path.getPathWithoutSchemeAndAuthority(targetPath).toString();\nboolean targetRaw = tp.startsWith(RAW);\nfor (Path p : context.getSourcePaths()) {\n  boolean srcRaw = Path.getPathWithoutSchemeAndAuthority(p).toString().startsWith(RAW);\n  if (srcRaw != targetRaw) {\n    throw new InvalidInputException(\n        \"Raw prefix mismatch between \" + p + \" and \" + targetPath);\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  copyListing.doBuildListing(listingFile, context);\n} catch (InvalidInputException e) {\n  if (e.getMessage().contains(\"Either all or none of the paths must have this prefix\")) {\n    // add or strip /.reserved/raw so source and target match, then re-run\n    alignRawPrefixesAndRerun();\n  } else {\n    throw e;\n  }\n}","preventionTips":["When copying encryption zones at block level, put /.reserved/raw on BOTH source and target.","Check every path in -f listings and globs, not just the first source.","Keep raw-path policies in one script constant so both sides are updated together."],"tags":["distcp","reserved-raw","encryption-zone","input-validation","hdfs"],"backgroundTag":"reserved-raw-prefix-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}