{"record":{"id":"b3e075643cd3c729","repo":"apache/hadoop","slug":"copy-from-non-reserved-raw-to-reserved","errorCode":null,"errorMessage":"'{}' copy from non '/.reserved/raw' to '/.reserved/raw'. Either both source and target must be in '/.reserved/raw' or neither.","messagePattern":"'(.+?)' copy from non '/\\.reserved/raw' to '/\\.reserved/raw'\\. Either both source and target must be in '/\\.reserved/raw' or neither\\.","errorType":"exception","errorClass":"PathOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java","lineNumber":401,"sourceCode":"   * /.reserved/raw.\n   */\n  private boolean checkPathsForReservedRaw(Path src, Path target)\n      throws PathOperationException {\n    final boolean srcIsRR = Path.getPathWithoutSchemeAndAuthority(src).\n        toString().startsWith(RESERVED_RAW);\n    final boolean dstIsRR = Path.getPathWithoutSchemeAndAuthority(target).\n        toString().startsWith(RESERVED_RAW);\n    boolean preserveRawXattrs = false;\n    if (srcIsRR && !dstIsRR) {\n      final String s = \"' copy from '\" + RESERVED_RAW + \"' to non '\" +\n          RESERVED_RAW + \"'. Either both source and target must be in '\" +\n          RESERVED_RAW + \"' or neither.\";\n      throw new PathOperationException(\"'\" + src.toString() + s);\n    } else if (!srcIsRR && dstIsRR) {\n      final String s = \"' copy from non '\" + RESERVED_RAW +\"' to '\" +\n          RESERVED_RAW + \"'. Either both source and target must be in '\" +\n          RESERVED_RAW + \"' or neither.\";\n      throw new PathOperationException(\"'\" + dst.toString() + s);\n    } else if (srcIsRR && dstIsRR) {\n      preserveRawXattrs = true;\n    }\n    return preserveRawXattrs;\n  }\n\n  /**\n   * If direct write is disabled ,copies the stream contents to a temporary\n   * file \"target._COPYING_\". If the copy is successful, the temporary file\n   * will be renamed to the real path, else the temporary file will be deleted.\n   * if direct write is enabled , then creation temporary file is skipped.\n   *\n   * @param in     the input stream for the copy\n   * @param target where to store the contents of the stream\n   * @throws IOException if copy fails\n   */ \n  protected void copyStreamToTarget(InputStream in, PathData target)\n  throws IOException {","sourceCodeStart":383,"sourceCodeEnd":419,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java#L383-L419","documentation":"The mirror case of the /.reserved/raw guard in CommandWithDestination.checkPathsForReservedRaw(): the TARGET path starts with /.reserved/raw but the SOURCE does not. Writing ordinary bytes plus synthesized raw xattrs into the raw namespace is rejected with \"'<dst>' copy from non '/.reserved/raw' to '/.reserved/raw'. Either both source and target must be in '/.reserved/raw' or neither.\" When BOTH sides are raw, the method returns preserveRawXattrs=true so raw.* xattrs are carried over.","triggerScenarios":"'hdfs dfs -cp /plain/file /.reserved/raw/zone/file' (only the destination uses the raw namespace); restore scripts written from the backup doc but applied to a normal source path.","commonSituations":"Restoring encryption-zone backups: source chosen as a plain path while the destination keeps the /.reserved/raw prefix; mixing prefixed and unprefixed paths within one script.","solutions":["Use /.reserved/raw on both source and destination to stay in raw mode","Or remove it from both to do a normal (decrypted) copy","Standardize one convention per script and lint for the prefix appearing on exactly one side"],"exampleFix":"# before\nhdfs dfs -cp /plain/file /.reserved/raw/zone/file\n# after\nhdfs dfs -cp /plain/file /zone/file","handlingStrategy":"validation","validationCode":"// same symmetric check, phrased for restore flows\nboolean dstIsRaw = dst.toUri().getPath().startsWith(\"/.reserved/raw\");\nboolean srcIsRaw = src.toUri().getPath().startsWith(\"/.reserved/raw\");\nif (dstIsRaw != srcIsRaw) {\n  dst = dstIsRaw ? stripRawPrefix(dst) : addRawPrefix(dst); // align both sides\n}","typeGuard":null,"tryCatchPattern":"try {\n  shellRun(\"-cp\", src, dst);\n} catch (PathOperationException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"copy from non '/.reserved/raw'\")) {\n    // destination alone was raw; drop the prefix for a plain copy\n    shellRun(\"-cp\", src.toString(), stripRawPrefix(dst).toString());\n  } else throw e;\n}","preventionTips":["Build restore commands from the same path builder used for backup so prefixes stay paired","Document per cluster whether raw-mode copies are the norm; mixed scripts are the failure mode","Validate both paths with a startsWith('/.reserved/raw') equality assertion in wrappers"],"tags":["hadoop","hdfs","encryption-zone","reserved-raw","xattr","copy","shell"],"backgroundTag":"reserved-raw-path-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}