{"record":{"id":"7959b6e624179615","repo":"apache/hadoop","slug":"copy-from-reserved-raw-to-non-reserved","errorCode":null,"errorMessage":"'{}' copy from '/.reserved/raw' to non '/.reserved/raw'. Either both source and target must be in '/.reserved/raw' or neither.","messagePattern":"'(.+?)' copy from '/\\.reserved/raw' to non '/\\.reserved/raw'\\. Either both source and target must be in '/\\.reserved/raw' or neither\\.","errorType":"exception","errorClass":"PathOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java","lineNumber":396,"sourceCode":"   *            path, not relative.\n   * @param target The target path to check. This should be a fully-qualified\n   *               path, not relative.\n   * @return true if raw.* xattrs should be preserved.\n   * @throws PathOperationException is only one of src/target are in\n   * /.reserved/raw.\n   */\n  private boolean checkPathsForReservedRaw(Path src, Path target)\n      throws PathOperationException {\n    final boolean srcIsRR = Path.getPathWithoutSchemeAndAuthority(src).\n        toString().startsWith(RESERVED_RAW);\n    final boolean dstIsRR = Path.getPathWithoutSchemeAndAuthority(target).\n        toString().startsWith(RESERVED_RAW);\n    boolean preserveRawXattrs = false;\n    if (srcIsRR && !dstIsRR) {\n      final String s = \"' copy from '\" + RESERVED_RAW + \"' to non '\" +\n          RESERVED_RAW + \"'. Either both source and target must be in '\" +\n          RESERVED_RAW + \"' or neither.\";\n      throw new PathOperationException(\"'\" + src.toString() + s);\n    } else if (!srcIsRR && dstIsRR) {\n      final String s = \"' copy from non '\" + RESERVED_RAW +\"' to '\" +\n          RESERVED_RAW + \"'. Either both source and target must be in '\" +\n          RESERVED_RAW + \"' or neither.\";\n      throw new PathOperationException(\"'\" + dst.toString() + s);\n    } else if (srcIsRR && dstIsRR) {\n      preserveRawXattrs = true;\n    }\n    return preserveRawXattrs;\n  }\n\n  /**\n   * If direct write is disabled ,copies the stream contents to a temporary\n   * file \"target._COPYING_\". If the copy is successful, the temporary file\n   * will be renamed to the real path, else the temporary file will be deleted.\n   * if direct write is enabled , then creation temporary file is skipped.\n   *\n   * @param in     the input stream for the copy","sourceCodeStart":378,"sourceCodeEnd":414,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java#L378-L414","documentation":"Thrown by CommandWithDestination.checkPathsForReservedRaw() (called from recursePath before copying): the SOURCE path starts with /.reserved/raw but the TARGET does not (or vice versa). /.reserved/raw is HDFS's virtual namespace exposing raw bytes plus extended attributes (notably raw.* encryption-zone xattrs); mixing namespaces in one copy would silently drop or leak raw xattrs, so PathOperationException is thrown with the message \"'<src>' copy from '/.reserved/raw' to non '/.reserved/raw'. Either both source and target must be in '/.reserved/raw' or neither.\"","triggerScenarios":"'hdfs dfs -cp /.reserved/raw/encrypted/file /backup/file' (target outside the raw namespace); distcp/cp-style backup of an encryption zone forgetting the /.reserved/raw prefix on BOTH sides.","commonSituations":"Backing up HDFS encryption zones: to preserve encrypted data and raw.* xattrs the copy must stay in /.reserved/raw on both ends; users add the prefix only to the source after reading partial docs.","solutions":["Keep both sides in the same namespace: 'hdfs dfs -cp /.reserved/raw/src /.reserved/raw/dst' for raw copies","Or drop /.reserved/raw entirely from BOTH paths to copy decrypted bytes (raw xattrs will not be preserved)","For encryption-zone backups, copy to /.reserved/raw on the destination cluster and re-encrypt with the target zone's key"],"exampleFix":"# before\nhdfs dfs -cp /.reserved/raw/zone/file /backup/file\n# after\nhdfs dfs -cp /.reserved/raw/zone/file /.reserved/raw/backup/file","handlingStrategy":"validation","validationCode":"// keep /.reserved/raw usage symmetric before copying\nstatic boolean bothOrNeitherRaw(Path src, Path dst) {\n  boolean s = src.toUri().getPath().startsWith(\"/.reserved/raw\");\n  boolean d = dst.toUri().getPath().startsWith(\"/.reserved/raw\");\n  return s == d;\n}\nif (!bothOrNeitherRaw(src, dst)) throw new IOException(\"raw namespace mismatch between \" + src + \" and \" + dst);","typeGuard":null,"tryCatchPattern":"try {\n  shellRun(\"-cp\", src, dst);\n} catch (PathOperationException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"/.reserved/raw\")) {\n    // re-issue with matching namespaces based on whether raw xattrs must survive\n    shellRun(\"-cp\", rawSrc.toString(), rawDst.toString());\n  } else throw e;\n}","preventionTips":["For encryption-zone backups always prefix BOTH source and destination with /.reserved/raw","Encode the raw-mode decision as one boolean in the job config and build both paths from it","Grep copy commands for '/.reserved/raw' appearing on exactly one side before deploy"],"tags":["hadoop","hdfs","encryption-zone","reserved-raw","xattr","copy","shell"],"backgroundTag":"reserved-raw-path-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}