{"record":{"id":"8a553334cbdb3b76","repo":"apache/hadoop","slug":"name-node-is-in-safe-mode-namenodehostname","errorCode":null,"errorMessage":"{}. Name node is in safe mode.\n{} NamenodeHostName:{}","messagePattern":"(.+?)\\. Name node is in safe mode\\.\n(.+?) NamenodeHostName:(.+?)","errorType":"exception","errorClass":"RetriableException","httpStatus":null,"severity":"warning","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java","lineNumber":1703,"sourceCode":"      getSnapshotManager().initThreadLocals();\n    }\n  }\n  \n  /**\n   * @throws RetriableException\n   *           If 1) The NameNode is in SafeMode, 2) HA is enabled, and 3)\n   *           NameNode is in active state\n   * @throws SafeModeException\n   *           Otherwise if NameNode is in SafeMode.\n   */\n  void checkNameNodeSafeMode(String errorMsg)\n      throws RetriableException, SafeModeException {\n    if (isInSafeMode()) {\n      SafeModeException se = newSafemodeException(errorMsg);\n      if (haEnabled && haContext != null\n          && haContext.getState().getServiceState() == HAServiceState.ACTIVE\n          && isInStartupSafeMode()) {\n        throw new RetriableException(se);\n      } else {\n        throw se;\n      }\n    }\n  }\n\n  private SafeModeException newSafemodeException(String errorMsg) {\n    return new SafeModeException(errorMsg + \". Name node is in safe \" +\n        \"mode.\\n\" + getSafeModeTip() + \" NamenodeHostName:\" + nameNodeHostName);\n  }\n\n  boolean isPermissionEnabled() {\n    return isPermissionEnabled;\n  }\n\n  public static Collection<URI> getNamespaceDirs(Configuration conf) {\n    return getStorageDirs(conf, DFS_NAMENODE_NAME_DIR_KEY);\n  }","sourceCodeStart":1685,"sourceCodeEnd":1721,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java#L1685-L1721","documentation":"checkNameNodeSafeMode guards every namespace-mutating operation. While the NameNode is in (startup) safe mode it rejects writes with SafeModeException, appending the current safe-mode tip and host name to the caller's message. When HA is enabled, the NN is active, and it is still in startup safe mode, the exception is wrapped in RetriableException to tell clients to retry rather than fail. The condition is transient: it clears once DataNode block reports reach the safe-mode thresholds.","triggerScenarios":"Any write or namespace op (mkdir, delete, setReplication, setPermission, cache directive ops, saveNamespace...) invoked while isInSafeMode() is true — typically during NN startup before dfs.namenode.safemode.threshold-pct of blocks is reported, or right after a failover while the new active re-enters startup safe mode.","commonSituations":"Clients or startup scripts racing NN boot; slow or dead DataNodes keeping thresholds unmet; safe-mode threshold/extension misconfigured; monitoring jobs writing HDFS the moment the NN process appears up.","solutions":["Wait for automatic exit: 'hdfs dfsadmin -safemode wait' or poll 'hdfs dfsadmin -safemode get'","Make clients retry: catch SafeModeException (and RetriableException) with back-off — the state self-heals","Fix slow/absent DataNodes first; their block reports are what ends safe mode","In an emergency 'hdfs dfsadmin -safemode leave' (understand blocks may be under-replicated); before tuning, review dfs.namenode.safemode.threshold-pct and dfs.namenode.safemode.extension"],"exampleFix":"// before: single-shot call fails during NN startup\nfs.mkdirs(new Path(\"/ingest\"));   // SafeModeException\n\n// after: wait out safe mode, then write\n((DistributedFileSystem) fs).setSafeMode(\n    HdfsConstants.SafeModeAction.SAFEMODE_WAIT);\nfs.mkdirs(new Path(\"/ingest\"));","handlingStrategy":"retry","validationCode":"// client-side pre-check before writes\nDistributedFileSystem dfs = (DistributedFileSystem) FileSystem.get(conf);\nwhile (dfs.getInSafeMode()) {\n  LOG.info(\"NameNode in safe mode, waiting for block reports\");\n  Thread.sleep(5_000);\n}","typeGuard":null,"tryCatchPattern":"try {\n  dfs.mkdirs(path);\n} catch (SafeModeException e) {\n  // startup safe mode: back off and retry — state clears itself\n  Thread.sleep(5_000); /* then retry */\n} catch (RetriableException e) {\n  // NN told us to retry (HA active + startup safe mode)\n  Thread.sleep(5_000); /* then retry */\n}","preventionTips":["Gate startup scripts and ingest jobs on 'hdfs dfsadmin -safemode wait'","Build retry-with-backoff around SafeModeException into HDFS-writing clients","Keep DataNodes healthy — block reports end safe mode; monitor dfs.namenode.safemode metrics","Set realistic dfs.namenode.safemode.threshold-pct and extension for your cluster size"],"tags":["hadoop","hdfs","namenode","safe-mode","transient","retry"],"backgroundTag":"service-temporarily-unavailable","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}