apache/hadoop · error · PathOperationException

'{}' copy from non '/.reserved/raw' to '/.reserved/raw'. Eit

Error message

'{}' copy from non '/.reserved/raw' to '/.reserved/raw'. Either both source and target must be in '/.reserved/raw' or neither.

What it means

The mirror case of the /.reserved/raw guard in CommandWithDestination.checkPathsForReservedRaw(): the TARGET path starts with /.reserved/raw but the SOURCE does not. Writing ordinary bytes plus synthesized raw xattrs into the raw namespace is rejected with "'<dst>' copy from non '/.reserved/raw' to '/.reserved/raw'. Either both source and target must be in '/.reserved/raw' or neither." When BOTH sides are raw, the method returns preserveRawXattrs=true so raw.* xattrs are carried over.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java:401

   * /.reserved/raw.
   */
  private boolean checkPathsForReservedRaw(Path src, Path target)
      throws PathOperationException {
    final boolean srcIsRR = Path.getPathWithoutSchemeAndAuthority(src).
        toString().startsWith(RESERVED_RAW);
    final boolean dstIsRR = Path.getPathWithoutSchemeAndAuthority(target).
        toString().startsWith(RESERVED_RAW);
    boolean preserveRawXattrs = false;
    if (srcIsRR && !dstIsRR) {
      final String s = "' copy from '" + RESERVED_RAW + "' to non '" +
          RESERVED_RAW + "'. Either both source and target must be in '" +
          RESERVED_RAW + "' or neither.";
      throw new PathOperationException("'" + src.toString() + s);
    } else if (!srcIsRR && dstIsRR) {
      final String s = "' copy from non '" + RESERVED_RAW +"' to '" +
          RESERVED_RAW + "'. Either both source and target must be in '" +
          RESERVED_RAW + "' or neither.";
      throw new PathOperationException("'" + dst.toString() + s);
    } else if (srcIsRR && dstIsRR) {
      preserveRawXattrs = true;
    }
    return preserveRawXattrs;
  }

  /**
   * If direct write is disabled ,copies the stream contents to a temporary
   * file "target._COPYING_". If the copy is successful, the temporary file
   * will be renamed to the real path, else the temporary file will be deleted.
   * if direct write is enabled , then creation temporary file is skipped.
   *
   * @param in     the input stream for the copy
   * @param target where to store the contents of the stream
   * @throws IOException if copy fails
   */ 
  protected void copyStreamToTarget(InputStream in, PathData target)
  throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

Use /.reserved/raw on both source and destination to stay in raw mode
Or remove it from both to do a normal (decrypted) copy
Standardize one convention per script and lint for the prefix appearing on exactly one side

Example fix

# before
hdfs dfs -cp /plain/file /.reserved/raw/zone/file
# after
hdfs dfs -cp /plain/file /zone/file

Defensive patterns

Strategy: validation

Validate before calling

// same symmetric check, phrased for restore flows
boolean dstIsRaw = dst.toUri().getPath().startsWith("/.reserved/raw");
boolean srcIsRaw = src.toUri().getPath().startsWith("/.reserved/raw");
if (dstIsRaw != srcIsRaw) {
  dst = dstIsRaw ? stripRawPrefix(dst) : addRawPrefix(dst); // align both sides
}

Try / catch

try {
  shellRun("-cp", src, dst);
} catch (PathOperationException e) {
  if (e.getMessage() != null && e.getMessage().contains("copy from non '/.reserved/raw'")) {
    // destination alone was raw; drop the prefix for a plain copy
    shellRun("-cp", src.toString(), stripRawPrefix(dst).toString());
  } else throw e;
}

Prevention

Build restore commands from the same path builder used for backup so prefixes stay paired
Document per cluster whether raw-mode copies are the norm; mixed scripts are the failure mode
Validate both paths with a startsWith('/.reserved/raw') equality assertion in wrappers

When it happens

Trigger: 'hdfs dfs -cp /plain/file /.reserved/raw/zone/file' (only the destination uses the raw namespace); restore scripts written from the backup doc but applied to a normal source path.

Common situations: Restoring encryption-zone backups: source chosen as a plain path while the destination keeps the /.reserved/raw prefix; mixing prefixed and unprefixed paths within one script.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b3e075643cd3c729. Report an issue: GitHub.