apache/hadoop · error · FileNotFoundException

"Failed to append to non-existent file " + path + " for clie

Error message

"Failed to append to non-existent file " + path + " for client " + clientMachine

What it means

FSDirAppendOp.appendFile throws FileNotFoundException when the resolved last INode is null: the path does not exist. Since append never creates files, this fires for never-created, deleted, or renamed-away paths, after the isDirectory check and after the WRITE permission check.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java:106

    final LocatedBlock lb;
    final FSDirectory fsd = fsn.getFSDirectory();
    final INodesInPath iip;
    fsd.writeLock();
    try {
      iip = fsd.resolvePath(pc, srcArg, DirOp.WRITE);
      // Verify that the destination does not exist as a directory already
      final INode inode = iip.getLastINode();
      final String path = iip.getPath();
      if (inode != null && inode.isDirectory()) {
        throw new FileAlreadyExistsException("Cannot append to directory "
            + path + "; already exists as a directory.");
      }
      if (fsd.isPermissionEnabled()) {
        fsd.checkPathAccess(pc, iip, FsAction.WRITE);
      }

      if (inode == null) {
        throw new FileNotFoundException(
            "Failed to append to non-existent file " + path + " for client "
                + clientMachine);
      }
      final INodeFile file = INodeFile.valueOf(inode, path, true);

      if (file.isStriped() && !newBlock) {
        throw new UnsupportedOperationException(
            "Append on EC file without new block is not supported. Use "
                + CreateFlag.NEW_BLOCK + " create flag while appending file.");
      }

      BlockManager blockManager = fsd.getBlockManager();
      final BlockStoragePolicy lpPolicy = blockManager
          .getStoragePolicy("LAZY_PERSIST");
      if (lpPolicy != null && lpPolicy.getId() == file.getStoragePolicyID()) {
        throw new UnsupportedOperationException(
            "Cannot append to lazy persist file " + path);
      }

View on GitHub (pinned to 2add963021)

Solutions

1. Verify existence with fs.exists(path) before calling append, and in concurrent pipelines treat the result as advisory (TOCTOU) - still catch FileNotFoundException.
2. If append-or-create semantics are wanted, catch FileNotFoundException and fall back to fs.create(path, true).
3. Fix the upstream stage that should have produced the file; if it should have existed, check the NameNode log or audit log for a delete/rename of that path.

Example fix

// before
FSDataOutputStream out = fs.append(path);

// after: create-or-append
FSDataOutputStream out;
try {
  out = fs.append(path);
} catch (FileNotFoundException e) {
  out = fs.create(path, true /*overwrite*/);
}

if (!fs.exists(path)) {
  FSDataOutputStream created = fs.create(path, true);
  created.close();
}
FSDataOutputStream out = fs.append(path);

try {
  out = fs.append(path);
} catch (FileNotFoundException e) {
  out = fs.create(path, true); // append-or-create semantics
}

When it happens

Trigger: DistributedFileSystem.append(path) for a path that was never created, was deleted (retention/cleanup job, trash), or was renamed away between an existence check and the append call.

Common situations: Chained jobs assuming the previous stage produced the file; races with cleanup jobs; typos or missing date partitions in generated paths; appends attempted after a file was moved into a snapshot path.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• failed to append to non-existent file {} on client {}
• File {} not found.
• Non existing file: ${path}. Create option is not specified i
• %s does not exist or is not file.
• ${message}: ${currentFilePath} [${t}]

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/99b1db04346a96d0. Report an issue: GitHub.