{"record":{"id":"ff658307aa3639e6","repo":"apache/hadoop","slug":"is-not-currently-healthy-cannot-be-failover-ta","errorCode":null,"errorMessage":"{} is not currently healthy. Cannot be failover target","messagePattern":"(.+?) is not currently healthy\\. Cannot be failover target","errorType":"exception","errorClass":"ServiceFailedException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java","lineNumber":770,"sourceCode":"  private ZKFCProtocol cedeRemoteActive(HAServiceTarget remote, int timeout)\n    throws IOException {\n    LOG.info(\"Asking \" + remote + \" to cede its active state for \"\n               + timeout + \"ms\");\n    ZKFCProtocol oldZkfc = remote.getZKFCProxy(conf, timeout);\n    oldZkfc.cedeActive(timeout);\n    return oldZkfc;\n  }\n\n  /**\n   * If the local node is an observer or is unhealthy it\n   * is not eligible for graceful failover.\n   * @throws ServiceFailedException if the node is an observer or unhealthy\n   */\n  private synchronized void checkEligibleForFailover()\n      throws ServiceFailedException {\n    // Check health\n    if (this.getLastHealthState() != State.SERVICE_HEALTHY) {\n      throw new ServiceFailedException(\n          localTarget + \" is not currently healthy. \" +\n          \"Cannot be failover target\");\n    }\n    if (serviceState == HAServiceState.OBSERVER) {\n      throw new ServiceFailedException(\n          localTarget + \" is in observer state. \" +\n          \"Cannot be failover target\");\n    }\n  }\n\n  /**\n   * @return an {@link HAServiceTarget} for the current active node\n   * in the cluster, or null if no node is active.\n   * @throws IOException if a ZK-related issue occurs\n   * @throws InterruptedException if thread is interrupted \n   */\n  private HAServiceTarget getCurrentActive()\n      throws IOException, InterruptedException {","sourceCodeStart":752,"sourceCodeEnd":788,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ZKFailoverController.java#L752-L788","documentation":"checkEligibleForFailover() throws ServiceFailedException('<target> is not currently healthy. Cannot be failover target') when graceful failover is requested and the local node's last health-monitor state is not SERVICE_HEALTHY (a sibling check rejects observer-state nodes). The pre-flight gate stops failover onto a node that cannot safely take over.","triggerScenarios":"Running 'hdfs haadmin -failover' (gracefulFailover) where the intended new active's health monitor last reported anything other than healthy: a failed checkHealth, an unreachable local service, or the monitor not having completed its first check yet after ZKFC startup.","commonSituations":"Standby NameNode down or unhealthy (shared edits dir missing, disk issues, OOM); failover issued seconds after ZKFC start before the first health check completes; target actually an observer node (sibling 'observer state' message); transient health-check RPC failures.","solutions":["Check the target's health: 'hdfs haadmin -checkHealth <nnId>' and the target NameNode's logs; fix whatever the check reports.","Wait for the ZKFC health monitor to report SERVICE_HEALTHY (first check completes shortly after start) before issuing failover.","Make sure the failover target is a standby, not an observer node.","Retry the failover once health is green."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Same gate the ZKFC applies, run before issuing the failover\nzkfc.checkEligibleForFailover(); // ServiceFailedException if not SERVICE_HEALTHY or observer\n// external equivalent: hdfs haadmin -checkHealth <nnId>","typeGuard":null,"tryCatchPattern":"try {\n  zkfc.gracefulFailover();\n} catch (ServiceFailedException e) {\n  if (e.getMessage().endsWith(\"Cannot be failover target\")) {\n    // target unhealthy or observer: check NN logs/health, wait for\n    // SERVICE_HEALTHY (first health check after ZKFC start), then retry\n  }\n}","preventionTips":["Always run 'hdfs haadmin -checkHealth' on the intended target before manual failover.","Don't issue failover immediately after ZKFC restart — wait for the first health check to complete.","Track each node's role (standby vs observer) in ops runbooks; observers are rejected targets by design."],"tags":["hadoop","high-availability","graceful-failover","health-check"],"backgroundTag":"service-health-check-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}