{"record":{"id":"b89f5011489b62a2","repo":"apache/hadoop","slug":"src-for-client-clientmachine-already-exists","errorCode":null,"errorMessage":"<src> for client <clientMachine> already exists","messagePattern":"<src> for client <clientMachine> already exists","errorType":"exception","errorClass":"FileAlreadyExistsException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java","lineNumber":396,"sourceCode":"\n    if (iip.getLastINode() != null) {\n      fsn.writeLock(RwLockMode.BM);\n      try {\n        if (overwrite) {\n          List<INode> toRemoveINodes = new ChunkedArrayList<>();\n          List<Long> toRemoveUCFiles = new ChunkedArrayList<>();\n          long ret = FSDirDeleteOp.delete(fsd, iip, toRemoveBlocks,\n              toRemoveINodes, toRemoveUCFiles, now());\n          if (ret >= 0) {\n            iip = INodesInPath.replace(iip, iip.length() - 1, null);\n            FSDirDeleteOp.incrDeletedFileCount(ret);\n            fsn.removeLeasesAndINodes(toRemoveUCFiles, toRemoveINodes, true);\n          }\n        } else {\n          // If lease soft limit time is expired, recover the lease\n          fsn.recoverLeaseInternal(FSNamesystem.RecoverLeaseOp.CREATE_FILE, iip,\n              src, holder, clientMachine, false);\n          throw new FileAlreadyExistsException(src + \" for client \" +\n              clientMachine + \" already exists\");\n        }\n      } finally {\n        fsn.writeUnlock(RwLockMode.BM, \"create\");\n      }\n    }\n    fsn.checkFsObjectLimit();\n    INodeFile newNode = null;\n    INodesInPath parent =\n        FSDirMkdirOp.createAncestorDirectories(fsd, iip, permissions);\n    if (parent != null) {\n      iip = addFile(fsd, parent, iip.getLastLocalName(), permissions,\n          replication, blockSize, holder, clientMachine, shouldReplicate,\n          ecPolicyName, storagePolicy);\n      newNode = iip != null ? iip.getLastINode().asFile() : null;\n    }\n    if (newNode == null) {\n      throw new IOException(\"Unable to add \" + src +  \" to namespace\");","sourceCodeStart":378,"sourceCodeEnd":414,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java#L378-L414","documentation":"startFile without OVERWRITE on a file that is open for write goes through lease handling: recoverLeaseInternal throws AlreadyBeingCreatedException when another client holds the lease within the soft limit, and when the soft limit has passed it recovers the lease and then throws FileAlreadyExistsException so the client can retry. Either way the path exists and is under construction by a live or recently-live writer.","triggerScenarios":"fs.create(path) or fs.append(path) without overwrite while the same file is open for write by another client, or by a crashed client whose lease is younger than dfs.namenode.lease-soft-limit-sec (default 60 s).","commonSituations":"Application restart reopening the same output file within a minute of the previous instance dying; speculative or duplicated task attempts writing the same HDFS path; two services configured with the same output file.","solutions":["Wait for the lease to clear (soft limit 60 s, hard limit 1 h) or force it with 'hdfs debug recoverLease -path <src>' or DistributedFileSystem.recoverLease(), then retry","Give each writer attempt a unique file name so writers never collide","If replacing the file is intended, create with overwrite=true"],"exampleFix":"// before\nFSDataOutputStream out = fs.create(path, false); // throws while old lease lives\n\n// after: recover a stale lease from a dead writer, then create\nDistributedFileSystem dfs = (DistributedFileSystem) fs;\nwhile (!dfs.isFileClosed(path)) {\n  dfs.recoverLease(path);\n  Thread.sleep(1_000L);\n}\nFSDataOutputStream out = dfs.create(path, false);","handlingStrategy":"retry","validationCode":"DistributedFileSystem dfs = (DistributedFileSystem) fs;\nif (!dfs.isFileClosed(path)) {\n  dfs.recoverLease(path);\n  // poll until closed, bounded by lease soft/hard limits\n}","typeGuard":null,"tryCatchPattern":"try {\n  out = fs.create(path, false);\n} catch (FileAlreadyExistsException | AlreadyBeingCreatedException e) {\n  Thread.sleep(5_000L);\n  out = fs.create(path, false); // retry after the lease clears\n}","preventionTips":["Give each writer attempt a unique file name","Wait for or force lease recovery before reopening a crashed writer's file","Check isFileClosed() before create/append on shared paths"],"tags":["hdfs","lease","concurrency","create","file-exists"],"backgroundTag":"lease-conflict","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}