{"record":{"id":"1bf26c8b1f15613b","repo":"apache/hadoop","slug":"processreport-from-dead-or-unregistered-node-nod","errorCode":null,"errorMessage":"ProcessReport from dead or unregistered node: {nodeID}","messagePattern":"ProcessReport from dead or unregistered node: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"warning","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java","lineNumber":2931,"sourceCode":"   */\n  public boolean processReport(final DatanodeID nodeID,\n      final DatanodeStorage storage,\n      final BlockListAsLongs newReport,\n      BlockReportContext context) throws IOException {\n    namesystem.writeLock(RwLockMode.GLOBAL);\n    final long startTime = Time.monotonicNow(); //after acquiring write lock\n    final long endTime;\n    DatanodeDescriptor node;\n    Collection<Block> invalidatedBlocks = Collections.emptyList();\n    String strBlockReportId =\n        context != null ? Long.toHexString(context.getReportId()) : \"\";\n    String fullBrLeaseId =\n        context != null ? Long.toHexString(context.getLeaseId()) : \"\";\n\n    try {\n      node = datanodeManager.getDatanode(nodeID);\n      if (node == null || !node.isRegistered()) {\n        throw new IOException(\n            \"ProcessReport from dead or unregistered node: \" + nodeID);\n      }\n\n      // To minimize startup time, we discard any second (or later) block reports\n      // that we receive while still in startup phase.\n      // Register DN with provided storage, not with storage owned by DN\n      // DN should still have a ref to the DNStorageInfo.\n      DatanodeStorageInfo storageInfo =\n          providedStorageMap.getStorage(node, storage);\n\n      if (storageInfo == null) {\n        // We handle this for backwards compatibility.\n        storageInfo = node.updateStorage(storage);\n      }\n      if (namesystem.isInStartupSafeMode()\n          && !StorageType.PROVIDED.equals(storageInfo.getStorageType())\n          && storageInfo.getBlockReportCount() > 0) {\n        blockLog.info(\"BLOCK* processReport 0x{} with lease ID 0x{}: \"","sourceCodeStart":2913,"sourceCodeEnd":2949,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java#L2913-L2949","documentation":"Thrown by BlockManager.processReport (full block report handling) when the reporting nodeID resolves to null or a DatanodeDescriptor with isRegistered()==false. The NameNode only accepts full block reports from registered, live datanodes; anything else is rejected outright because block --> storage maps can only be updated for known storage identities.","triggerScenarios":"Datanode sends its periodic full block report before (re-)registering — typical after NN restart/failover, when a DN's registration was evicted, or when the node was decommissioned but still running.","commonSituations":"NN restart in progress (safe mode) while DNs race to report; HA failover; datanode with stale registration talking to a new active; include/exclude list changes removing the node.","solutions":["Confirm registration: `hdfs dfsadmin -report`; the DN re-registers on its next heartbeat so this is usually self-healing — watch that subsequent BRs succeed","Restart the datanode to force immediate re-registration: `hdfs --daemon restart datanode`","Check dfs.hosts / dfs.hosts.exclude and refresh: `hdfs dfsadmin -refreshNodes` if the node was accidentally excluded","During NN restarts, expect these transiently until all DNs re-register; no data risk — reports are retried"],"exampleFix":"# before: full BR rejected during NN restart\n# NN log: ProcessReport from dead or unregistered node: 10.0.0.5:9866\n\n# after: verify/force registration, confirm next BR accepted\nhdfs dfsadmin -refreshNodes\nhdfs --daemon restart datanode\ngrep 'ProcessReport from dead' /var/log/hadoop-hdfs/*.log | tail  # should stop","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  nn.blockReport(bpRegistration, poolId, storage, reports, context);\n} catch (IOException e) {\n  if (e.getMessage().contains(\"dead or unregistered node\")) {\n    reRegisterWith Namenode(); // DN: handshake again, retry report on next cycle\n  } else { throw e; }\n}","preventionTips":["Expect and tolerate transient rejections during NN restart; DN retries automatically","Restart datanodes after persistent registration failures","Use `hdfs dfsadmin -refreshNodes` after editing host files","Do not point DNs at more than one active NN"],"tags":["hdfs","block-report","datanode-registration","namenode","safe-mode"],"backgroundTag":"unregistered-datanode","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}