apache/hadoop · error · PathOperationException

'{}' copy from '/.reserved/raw' to non '/.reserved/raw'. Eit

Error message

'{}' copy from '/.reserved/raw' to non '/.reserved/raw'. Either both source and target must be in '/.reserved/raw' or neither.

What it means

Thrown by CommandWithDestination.checkPathsForReservedRaw() (called from recursePath before copying): the SOURCE path starts with /.reserved/raw but the TARGET does not (or vice versa). /.reserved/raw is HDFS's virtual namespace exposing raw bytes plus extended attributes (notably raw.* encryption-zone xattrs); mixing namespaces in one copy would silently drop or leak raw xattrs, so PathOperationException is thrown with the message "'<src>' copy from '/.reserved/raw' to non '/.reserved/raw'. Either both source and target must be in '/.reserved/raw' or neither."

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java:396

   *            path, not relative.
   * @param target The target path to check. This should be a fully-qualified
   *               path, not relative.
   * @return true if raw.* xattrs should be preserved.
   * @throws PathOperationException is only one of src/target are in
   * /.reserved/raw.
   */
  private boolean checkPathsForReservedRaw(Path src, Path target)
      throws PathOperationException {
    final boolean srcIsRR = Path.getPathWithoutSchemeAndAuthority(src).
        toString().startsWith(RESERVED_RAW);
    final boolean dstIsRR = Path.getPathWithoutSchemeAndAuthority(target).
        toString().startsWith(RESERVED_RAW);
    boolean preserveRawXattrs = false;
    if (srcIsRR && !dstIsRR) {
      final String s = "' copy from '" + RESERVED_RAW + "' to non '" +
          RESERVED_RAW + "'. Either both source and target must be in '" +
          RESERVED_RAW + "' or neither.";
      throw new PathOperationException("'" + src.toString() + s);
    } else if (!srcIsRR && dstIsRR) {
      final String s = "' copy from non '" + RESERVED_RAW +"' to '" +
          RESERVED_RAW + "'. Either both source and target must be in '" +
          RESERVED_RAW + "' or neither.";
      throw new PathOperationException("'" + dst.toString() + s);
    } else if (srcIsRR && dstIsRR) {
      preserveRawXattrs = true;
    }
    return preserveRawXattrs;
  }

  /**
   * If direct write is disabled ,copies the stream contents to a temporary
   * file "target._COPYING_". If the copy is successful, the temporary file
   * will be renamed to the real path, else the temporary file will be deleted.
   * if direct write is enabled , then creation temporary file is skipped.
   *
   * @param in     the input stream for the copy

View on GitHub (pinned to 2add963021)

Solutions

1. Keep both sides in the same namespace: 'hdfs dfs -cp /.reserved/raw/src /.reserved/raw/dst' for raw copies
2. Or drop /.reserved/raw entirely from BOTH paths to copy decrypted bytes (raw xattrs will not be preserved)
3. For encryption-zone backups, copy to /.reserved/raw on the destination cluster and re-encrypt with the target zone's key

Example fix

# before
hdfs dfs -cp /.reserved/raw/zone/file /backup/file
# after
hdfs dfs -cp /.reserved/raw/zone/file /.reserved/raw/backup/file

// keep /.reserved/raw usage symmetric before copying
static boolean bothOrNeitherRaw(Path src, Path dst) {
  boolean s = src.toUri().getPath().startsWith("/.reserved/raw");
  boolean d = dst.toUri().getPath().startsWith("/.reserved/raw");
  return s == d;
}
if (!bothOrNeitherRaw(src, dst)) throw new IOException("raw namespace mismatch between " + src + " and " + dst);

try {
  shellRun("-cp", src, dst);
} catch (PathOperationException e) {
  if (e.getMessage() != null && e.getMessage().contains("/.reserved/raw")) {
    // re-issue with matching namespaces based on whether raw xattrs must survive
    shellRun("-cp", rawSrc.toString(), rawDst.toString());
  } else throw e;
}

When it happens

Trigger: 'hdfs dfs -cp /.reserved/raw/encrypted/file /backup/file' (target outside the raw namespace); distcp/cp-style backup of an encryption zone forgetting the /.reserved/raw prefix on BOTH sides.

Common situations: Backing up HDFS encryption zones: to preserve encrypted data and raw.* xattrs the copy must stay in /.reserved/raw on both ends; users add the prefix only to the source after reading partial docs.

Related errors

• '{}' copy from non '/.reserved/raw' to '/.reserved/raw'. Eit
• No such file or directory
• Too many matches
• Is not a directory
• File exists

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7959b6e624179615. Report an issue: GitHub.