{"record":{"id":"99b1db04346a96d0","repo":"apache/hadoop","slug":"failed-to-append-to-non-existent-file-path","errorCode":null,"errorMessage":"\"Failed to append to non-existent file \" + path + \" for client \" + clientMachine","messagePattern":"\"Failed to append to non-existent file \" \\+ path \\+ \" for client \" \\+ clientMachine","errorType":"exception","errorClass":"FileNotFoundException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java","lineNumber":106,"sourceCode":"    final LocatedBlock lb;\n    final FSDirectory fsd = fsn.getFSDirectory();\n    final INodesInPath iip;\n    fsd.writeLock();\n    try {\n      iip = fsd.resolvePath(pc, srcArg, DirOp.WRITE);\n      // Verify that the destination does not exist as a directory already\n      final INode inode = iip.getLastINode();\n      final String path = iip.getPath();\n      if (inode != null && inode.isDirectory()) {\n        throw new FileAlreadyExistsException(\"Cannot append to directory \"\n            + path + \"; already exists as a directory.\");\n      }\n      if (fsd.isPermissionEnabled()) {\n        fsd.checkPathAccess(pc, iip, FsAction.WRITE);\n      }\n\n      if (inode == null) {\n        throw new FileNotFoundException(\n            \"Failed to append to non-existent file \" + path + \" for client \"\n                + clientMachine);\n      }\n      final INodeFile file = INodeFile.valueOf(inode, path, true);\n\n      if (file.isStriped() && !newBlock) {\n        throw new UnsupportedOperationException(\n            \"Append on EC file without new block is not supported. Use \"\n                + CreateFlag.NEW_BLOCK + \" create flag while appending file.\");\n      }\n\n      BlockManager blockManager = fsd.getBlockManager();\n      final BlockStoragePolicy lpPolicy = blockManager\n          .getStoragePolicy(\"LAZY_PERSIST\");\n      if (lpPolicy != null && lpPolicy.getId() == file.getStoragePolicyID()) {\n        throw new UnsupportedOperationException(\n            \"Cannot append to lazy persist file \" + path);\n      }","sourceCodeStart":88,"sourceCodeEnd":124,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java#L88-L124","documentation":"FSDirAppendOp.appendFile throws FileNotFoundException when the resolved last INode is null: the path does not exist. Since append never creates files, this fires for never-created, deleted, or renamed-away paths, after the isDirectory check and after the WRITE permission check.","triggerScenarios":"DistributedFileSystem.append(path) for a path that was never created, was deleted (retention/cleanup job, trash), or was renamed away between an existence check and the append call.","commonSituations":"Chained jobs assuming the previous stage produced the file; races with cleanup jobs; typos or missing date partitions in generated paths; appends attempted after a file was moved into a snapshot path.","solutions":["Verify existence with fs.exists(path) before calling append, and in concurrent pipelines treat the result as advisory (TOCTOU) - still catch FileNotFoundException.","If append-or-create semantics are wanted, catch FileNotFoundException and fall back to fs.create(path, true).","Fix the upstream stage that should have produced the file; if it should have existed, check the NameNode log or audit log for a delete/rename of that path."],"exampleFix":"// before\nFSDataOutputStream out = fs.append(path);\n\n// after: create-or-append\nFSDataOutputStream out;\ntry {\n  out = fs.append(path);\n} catch (FileNotFoundException e) {\n  out = fs.create(path, true /*overwrite*/);\n}","handlingStrategy":"validation","validationCode":"if (!fs.exists(path)) {\n  FSDataOutputStream created = fs.create(path, true);\n  created.close();\n}\nFSDataOutputStream out = fs.append(path);","typeGuard":null,"tryCatchPattern":"try {\n  out = fs.append(path);\n} catch (FileNotFoundException e) {\n  out = fs.create(path, true); // append-or-create semantics\n}","preventionTips":["Make append-based jobs verify upstream output exists as part of preflight checks.","Audit cleanup/retention jobs so they do not race appends on the same paths."],"tags":["hdfs","append","file-not-found"],"backgroundTag":"file-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}