apache/hadoop · error · IOException

Unable to add <src> to namespace

Error message

Unable to add <src> to namespace

What it means

After the earlier existence and lease checks pass, startFile performs the namespace insertion: createAncestorDirectories then addFile. Both signal failure by returning null, and if the new INodeFile cannot be established the operation aborts with IOException('Unable to add ... to namespace'). The classic cause is a concurrent create of the same path landing between the read-lock checks and the write, so the namespace changed under the writer.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java:414

          throw new FileAlreadyExistsException(src + " for client " +
              clientMachine + " already exists");
        }
      } finally {
        fsn.writeUnlock(RwLockMode.BM, "create");
      }
    }
    fsn.checkFsObjectLimit();
    INodeFile newNode = null;
    INodesInPath parent =
        FSDirMkdirOp.createAncestorDirectories(fsd, iip, permissions);
    if (parent != null) {
      iip = addFile(fsd, parent, iip.getLastLocalName(), permissions,
          replication, blockSize, holder, clientMachine, shouldReplicate,
          ecPolicyName, storagePolicy);
      newNode = iip != null ? iip.getLastINode().asFile() : null;
    }
    if (newNode == null) {
      throw new IOException("Unable to add " + src +  " to namespace");
    }
    fsn.leaseManager.addLease(
        newNode.getFileUnderConstructionFeature().getClientName(),
        newNode.getId());
    if (feInfo != null) {
      FSDirEncryptionZoneOp.setFileEncryptionInfo(fsd, iip, feInfo,
          XAttrSetFlag.CREATE);
    }
    setNewINodeStoragePolicy(fsd.getBlockManager(), iip, isLazyPersist);
    fsd.getEditLog().logOpenFile(src, newNode, overwrite, logRetryEntry);
    if (NameNode.stateChangeLog.isDebugEnabled()) {
      NameNode.stateChangeLog.debug("DIR* NameSystem.startFile: added " +
          src + " inode " + newNode.getId() + " " + holder);
    }
    return FSDirStatAndListingOp.getFileInfo(fsd, iip, false, false);
  }

  static INodeFile addFileForEditLog(

View on GitHub (pinned to 2add963021)

Solutions

  1. Retry the create: by then the winner exists, so decide overwrite versus a new name
  2. Guarantee a single creator per path (driver creates, tasks write partitions; or per-attempt file names)
  3. After the failure, re-stat the path and branch on existence

Example fix

// before: N tasks race to create the same file
FSDataOutputStream out = fs.create(sharedPath, false);

// after: unique file per attempt
Path p = new Path(outDir, "attempt-" + attemptId);
try (FSDataOutputStream out = fs.create(p, false)) { ... }
Defensive patterns

Strategy: retry

Validate before calling

if (fs.exists(path)) {
  // someone already created it: decide overwrite vs. new name before calling create
  return existingFilePlan(path);
}
// proceed with create; races still possible, so keep the catch path

Try / catch

try {
  out = fs.create(path, false);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("Unable to add")
      && fs.exists(path)) {
    out = fs.create(path, true); // loser of the race: overwrite if that is safe
  } else {
    throw e;
  }
}

Prevention

When it happens

Trigger: Two clients creating the exact same file path concurrently, the loser's addFile finding the inode already present; ancestor-directory creation failing during a concurrent rename/delete of parent directories; rare edit-log or replay anomalies.

Common situations: Duplicate or speculative tasks creating the same output file; job drivers touching output paths from multiple threads; concurrent renames around the target directory.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1b7ef9b17f608ebf. Report an issue: GitHub.