{"record":{"id":"feffa08767015b22","repo":"apache/hadoop","slug":"datanode-datanode-not-found","errorCode":null,"errorMessage":"Datanode {datanode} not found.","messagePattern":"Datanode (.+?) not found\\.","errorType":"exception","errorClass":"HadoopIllegalArgumentException","httpStatus":null,"severity":"warning","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java","lineNumber":1760,"sourceCode":"    }\n    if(iFile.isUnderConstruction()) {\n      return true;\n    }\n    if (iFile.getAccessTime() > time || iFile.getModificationTime() > time) {\n      return true;\n    }\n    return false;\n  }\n\n  /** Get all blocks with location information from a datanode. */\n  public BlocksWithLocations getBlocksWithLocations(final DatanodeID datanode,\n      final long size, final long minBlockSize, final long timeInterval,\n      final StorageType storageType) throws UnregisteredNodeException {\n    final DatanodeDescriptor node = getDatanodeManager().getDatanode(datanode);\n    if (node == null) {\n      blockLog.warn(\"BLOCK* getBlocks: Asking for blocks from an\" +\n          \" unrecorded node {}\", datanode);\n      throw new HadoopIllegalArgumentException(\n          \"Datanode \" + datanode + \" not found.\");\n    }\n\n    int numBlocks = node.numBlocks();\n    if(numBlocks == 0) {\n      return new BlocksWithLocations(new BlockWithLocations[0]);\n    }\n\n    // skip stale storage, then choose specific storage type.\n    DatanodeStorageInfo[] storageInfos = Arrays\n        .stream(node.getStorageInfos())\n        .filter(s -> !s.areBlockContentsStale())\n        .filter(s -> storageType == null || s.getStorageType().equals(storageType))\n        .toArray(DatanodeStorageInfo[]::new);\n\n    // starting from a random block\n    int startBlock = ThreadLocalRandom.current().nextInt(numBlocks);\n    Iterator<BlockInfo> iter = node.getBlockIterator(startBlock, storageInfos);","sourceCodeStart":1742,"sourceCodeEnd":1778,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java#L1742-L1778","documentation":"Thrown as HadoopIllegalArgumentException from BlockManager.getBlocksWithLocations when a datanode (typically the Balancer's datanode-side Mover/Balancer agent) requests the list of that node's blocks but the NameNode has no DatanodeDescriptor registered for the given DatanodeID. It means the node is unknown to the NN — it never registered, was decommissioned and removed, or its registration was evicted.","triggerScenarios":"Balancer/Datanode calls DataNodeProtocol getBlocks(datanodeID, size, ...) after the NN restarted and lost (or reordered) registration, or for a node that deregistered/decommissioned; NN failover to a standby that has not seen this DN register yet; stale balancer instance pointing at a different cluster/namespace.","commonSituations":"Balancer started while NameNode is in startup/safe mode before datanodes re-registered; HA failover with running balancer; running the balancer against the wrong NameNode/namespace (cluster mismatch); node decommissioned mid-balancer-run.","solutions":["Restart the balancer after datanodes have (re)registered with the active NameNode — verify with `hdfs dfsadmin -report | grep -A2 Datanode`","Point the balancer at the active NN of the correct namespace (-fs hdfs://correct-nameservice)","If NN just restarted, wait for block reports/registrations to complete and exit safe mode, then rerun","Remove/retire the stale datanode reference if the node was decommissioned intentionally"],"exampleFix":"# before: balancer runs through NN restart / failover\nhdfs --daemon start balancer  # getBlocks -> Datanode ... not found\n\n# after: confirm registrations, then start balancer against active NN\nhdfs dfsadmin -report | grep -c '^Datanode'\nhdfs --daemon start balancer -fs hdfs://mycluster   # nameservice of active NN","handlingStrategy":"retry","validationCode":"// Balancer/tool side: confirm the datanode is known to the NN before getBlocks\nDatanodeInfo[] live = ((DistributedFileSystem) fs).getDataNodeStats(DatanodeReportTypes.LIVE);\nboolean known = Arrays.stream(live).anyMatch(d -> d.getXferAddr().equals(node.getXferAddr()));\nif (known) { nn.getBlockLocations(node, size, minSize, interval, type); }","typeGuard":null,"tryCatchPattern":"try {\n  blocks = nn.getBlockLocations(node, size, minSize, interval, type);\n} catch (HadoopIllegalArgumentException e) {\n  if (e.getMessage().contains(\"not found\")) { reRegisterOrSkip(node); /* restart DN or skip node */ }\n  else { throw e; }\n}","preventionTips":["Start balancer/mover only after `hdfs dfsadmin -report` shows all DNs registered","Target the active NameNode's nameservice explicitly","Restart the balancer after NN restart/failover","Treat unknown-node errors as cluster-state signals, not balancer bugs"],"tags":["hdfs","balancer","datanode-registration","namenode","rpc"],"backgroundTag":"unregistered-datanode","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}