apache/hadoop · error · InvalidPathException

{}

Error message

{}

What it means

startFile validates the target path before locking: it must pass DFSUtil.isValidName (absolute, no illegal components), must not equal the reserved root exactly, and any /.reserved usage is only allowed for the raw (.reserved/raw) or inodes (.reserved/inodes) virtual namespaces. Anything else throws InvalidPathException echoing the offending src.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:2807

      builder.append("DIR* NameSystem.startFile: src=").append(src)
          .append(", holder=").append(holder)
          .append(", clientMachine=").append(clientMachine)
          .append(", createParent=").append(createParent)
          .append(", replication=").append(replication)
          .append(", createFlag=").append(flag)
          .append(", blockSize=").append(blockSize)
          .append(", supportedVersions=")
          .append(Arrays.toString(supportedVersions))
          .append(", ecPolicyName=").append(ecPolicyName)
          .append(", storagePolicy=").append(storagePolicy);
      NameNode.stateChangeLog.debug(builder.toString());
    }
    if (!DFSUtil.isValidName(src) ||
        FSDirectory.isExactReservedName(src) ||
        (FSDirectory.isReservedName(src)
            && !FSDirectory.isReservedRawName(src)
            && !FSDirectory.isReservedInodesName(src))) {
      throw new InvalidPathException(src);
    }

    boolean shouldReplicate = flag.contains(CreateFlag.SHOULD_REPLICATE);
    if (shouldReplicate &&
        (!org.apache.commons.lang3.StringUtils.isEmpty(ecPolicyName))) {
      throw new HadoopIllegalArgumentException("SHOULD_REPLICATE flag and " +
          "ecPolicyName are exclusive parameters. Set both is not allowed!");
    }

    INodesInPath iip = null;
    boolean skipSync = true; // until we do something that might create edits
    HdfsFileStatus stat = null;
    BlocksMapUpdateInfo toRemoveBlocks = null;

    checkOperation(OperationCategory.WRITE);
    final FSPermissionChecker pc = getPermissionChecker();
    writeLock(RwLockMode.FS);
    try {

View on GitHub (pinned to 2add963021)

Solutions

1. Qualify and normalize client-side: Path p = new Path(raw).makeQualified(fs.getUri(), fs.getWorkingDir()); pass p.toString()
2. Reject '.'/'..' components at the input boundary — HDFS paths never contain them
3. For raw block-level access use /.reserved/raw/<realPath>; for inode-id access /.reserved/inodes/<id>; never create under /.reserved itself

Example fix

// before
fs.create(new Path(userInput));   // 'file.txt' or 'a/./b' -> InvalidPathException
// after
Path p = new Path(userInput).makeQualified(fs.getUri(), fs.getWorkingDir());
fs.create(p);

Path q = new Path(raw).makeQualified(fs.getUri(), fs.getWorkingDir());
for (String comp : q.toUri().getPath().split("/")) {
  if (comp.equals(".") || comp.equals("..")) throw new IllegalArgumentException("bad path component: " + raw);
}

static boolean isWritableHdfsName(String src) {
  if (src == null || !src.startsWith("/")) return false;
  if (src.equals("/.reserved")) return false;
  if (src.startsWith("/.reserved")
      && !(src.startsWith("/.reserved/raw/") || src.startsWith("/.reserved/inodes/"))) return false;
  for (String c : src.split("/")) { if (c.equals(".") || c.equals("..")) return false; }
  return true;
}

catch (InvalidPathException e) { re-prompt for a valid absolute HDFS path; qualify via new Path(input, defaultParent).makeQualified(uri, workingDir) }

When it happens

Trigger: DFSClient.create with a relative path (no scheme/authority qualification), a path containing '.' or '..' components, an empty path, or attempts to create directly under /.reserved or an unrecognized /.reserved subtree.

Common situations: Apps building paths from raw user input without normalization; forgetting Path.makeQualified so the working-directory-relative string reaches the NN as-is; tooling poking /.reserved virtual namespaces incorrectly.

Related errors

• Invalid path string ${pathString}
• Invalid path string
• Requested replication factor of {replication}{err} for {src}
• Source '{srcFile}' exists but is a directory
• Source '{srcFile}' and destination '{destFile}' are the same

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/fa48f9d7dab53e7e. Report an issue: GitHub.