{"record":{"id":"1b7ef9b17f608ebf","repo":"apache/hadoop","slug":"unable-to-add-src-to-namespace","errorCode":null,"errorMessage":"Unable to add <src> to namespace","messagePattern":"Unable to add <src> to namespace","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java","lineNumber":414,"sourceCode":"          throw new FileAlreadyExistsException(src + \" for client \" +\n              clientMachine + \" already exists\");\n        }\n      } finally {\n        fsn.writeUnlock(RwLockMode.BM, \"create\");\n      }\n    }\n    fsn.checkFsObjectLimit();\n    INodeFile newNode = null;\n    INodesInPath parent =\n        FSDirMkdirOp.createAncestorDirectories(fsd, iip, permissions);\n    if (parent != null) {\n      iip = addFile(fsd, parent, iip.getLastLocalName(), permissions,\n          replication, blockSize, holder, clientMachine, shouldReplicate,\n          ecPolicyName, storagePolicy);\n      newNode = iip != null ? iip.getLastINode().asFile() : null;\n    }\n    if (newNode == null) {\n      throw new IOException(\"Unable to add \" + src +  \" to namespace\");\n    }\n    fsn.leaseManager.addLease(\n        newNode.getFileUnderConstructionFeature().getClientName(),\n        newNode.getId());\n    if (feInfo != null) {\n      FSDirEncryptionZoneOp.setFileEncryptionInfo(fsd, iip, feInfo,\n          XAttrSetFlag.CREATE);\n    }\n    setNewINodeStoragePolicy(fsd.getBlockManager(), iip, isLazyPersist);\n    fsd.getEditLog().logOpenFile(src, newNode, overwrite, logRetryEntry);\n    if (NameNode.stateChangeLog.isDebugEnabled()) {\n      NameNode.stateChangeLog.debug(\"DIR* NameSystem.startFile: added \" +\n          src + \" inode \" + newNode.getId() + \" \" + holder);\n    }\n    return FSDirStatAndListingOp.getFileInfo(fsd, iip, false, false);\n  }\n\n  static INodeFile addFileForEditLog(","sourceCodeStart":396,"sourceCodeEnd":432,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java#L396-L432","documentation":"After the earlier existence and lease checks pass, startFile performs the namespace insertion: createAncestorDirectories then addFile. Both signal failure by returning null, and if the new INodeFile cannot be established the operation aborts with IOException('Unable to add ... to namespace'). The classic cause is a concurrent create of the same path landing between the read-lock checks and the write, so the namespace changed under the writer.","triggerScenarios":"Two clients creating the exact same file path concurrently, the loser's addFile finding the inode already present; ancestor-directory creation failing during a concurrent rename/delete of parent directories; rare edit-log or replay anomalies.","commonSituations":"Duplicate or speculative tasks creating the same output file; job drivers touching output paths from multiple threads; concurrent renames around the target directory.","solutions":["Retry the create: by then the winner exists, so decide overwrite versus a new name","Guarantee a single creator per path (driver creates, tasks write partitions; or per-attempt file names)","After the failure, re-stat the path and branch on existence"],"exampleFix":"// before: N tasks race to create the same file\nFSDataOutputStream out = fs.create(sharedPath, false);\n\n// after: unique file per attempt\nPath p = new Path(outDir, \"attempt-\" + attemptId);\ntry (FSDataOutputStream out = fs.create(p, false)) { ... }","handlingStrategy":"retry","validationCode":"if (fs.exists(path)) {\n  // someone already created it: decide overwrite vs. new name before calling create\n  return existingFilePlan(path);\n}\n// proceed with create; races still possible, so keep the catch path","typeGuard":null,"tryCatchPattern":"try {\n  out = fs.create(path, false);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Unable to add\")\n      && fs.exists(path)) {\n    out = fs.create(path, true); // loser of the race: overwrite if that is safe\n  } else {\n    throw e;\n  }\n}","preventionTips":["Single creator per path: driver creates, workers write partitions","Use deterministic per-attempt file names","Treat create as fallible under concurrency and re-stat on failure"],"tags":["hdfs","create","race-condition","namespace"],"backgroundTag":"concurrent-create-race","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}