apache/hadoop · error · FileNotFoundException

Can't overwrite non-existent

Error message

Can't overwrite non-existent <src>

What it means

resolvePathForStartFile throws FileNotFoundException when CreateFlag.OVERWRITE is passed without CreateFlag.CREATE and the path does not exist: overwrite-only cannot bring a file into being, so the create fails. The public convenience API FileSystem.create(path, true) always sets both flags, so this is usually seen by code assembling the EnumSet itself.

Solutions

1. Always pair the flags: EnumSet.of(CreateFlag.CREATE, CreateFlag.OVERWRITE), or simply use fs.create(path, true)
2. If overwrite-only is intended, pre-check existence and fail with your own clear error
3. Handle deletion races by retrying the create with both flags

Example fix

// before
fs.create(path, FsPermission.getFileDefault(),
    EnumSet.of(CreateFlag.OVERWRITE), bufSize, replication, blockSize, null);

// after
fs.create(path, FsPermission.getFileDefault(),
    EnumSet.of(CreateFlag.CREATE, CreateFlag.OVERWRITE), bufSize, replication, blockSize, null);

EnumSet<CreateFlag> flags = fs.exists(path)
    ? EnumSet.of(CreateFlag.CREATE, CreateFlag.OVERWRITE)
    : EnumSet.of(CreateFlag.CREATE);

try {
  out = fs.create(path, perm, EnumSet.of(CreateFlag.OVERWRITE), bufSize, repl, bs, null);
} catch (FileNotFoundException e) {
  // OVERWRITE alone cannot create: retry with CREATE included
  out = fs.create(path, perm, EnumSet.of(CreateFlag.CREATE, CreateFlag.OVERWRITE), bufSize, repl, bs, null);
}

When it happens

Trigger: Direct ClientProtocol.create callers or wrappers that build EnumSet<CreateFlag> manually and omit CREATE (for example EnumSet.of(OVERWRITE)); also a race where the file is deleted between an exists() check and a create flagged only OVERWRITE.

Common situations: Custom RPC clients and migration tools copying CreateFlag sets around; older library code assembling flags by hand.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Block data not found, r=
• {blockURI}
• Can not create ' ' file, because parent folder does not…
• Cannot create file because parent folder does not exist.
• Cannot find a block policy with the name

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d61fe075c8703274. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java:347

    }
    INode inode = iip.getLastINode();
    if (inode != null) {
      // Verify that the destination does not exist as a directory already.
      if (inode.isDirectory()) {
        throw new FileAlreadyExistsException(iip.getPath() +
            " already exists as a directory");
      }
      // Verifies it's indeed a file and perms allow overwrite
      INodeFile.valueOf(inode, src);
      if (dir.isPermissionEnabled() && flag.contains(CreateFlag.OVERWRITE)) {
        dir.checkPathAccess(pc, iip, FsAction.WRITE);
      }
    } else {
      if (!createParent) {
        dir.verifyParentDir(iip);
      }
      if (!flag.contains(CreateFlag.CREATE)) {
        throw new FileNotFoundException("Can't overwrite non-existent " + src);
      }
    }
    return iip;
  }


  /**
   * Create a new file or overwrite an existing file<br>
   *
   * Once the file is create the client then allocates a new block with the next
   * call using {@link ClientProtocol#addBlock}.
   * <p>
   * For description of parameters and exceptions thrown see
   * {@link ClientProtocol#create}
   */
  static HdfsFileStatus startFile(
      FSNamesystem fsn, INodesInPath iip,
      PermissionStatus permissions, String holder, String clientMachine,

View on GitHub (pinned to 2add963021)