apache/hadoop · error · ServiceFailedException

Cannot transition from 'active' to 'observer'

Error message

Cannot transition from 'active' to 'observer'

What it means

transitionToObserver forbids ACTIVE -> OBSERVER directly (the code comment states 'Transition from ACTIVE to OBSERVER is forbidden'). An active NameNode owns the edit log; an observer merely tails it, so the demotion must pass through standby where edit ownership is relinquished first.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java:2048

    namesystem.checkSuperuserPrivilege(operationName);
    if (!haEnabled) {
      throw new ServiceFailedException("HA for namenode is not enabled");
    }
    state.setState(haContext, STANDBY_STATE);
  }

  synchronized void transitionToObserver() throws IOException {
    String operationName = "transitionToObserver";
    namesystem.checkSuperuserPrivilege(operationName);
    if (notBecomeActiveInSafemode && isInSafeMode()) {
      throw new ServiceFailedException(getRole() + " still not leave safemode");
    }
    if (!haEnabled) {
      throw new ServiceFailedException("HA for namenode is not enabled");
    }
    // Transition from ACTIVE to OBSERVER is forbidden.
    if (state == ACTIVE_STATE) {
      throw new ServiceFailedException(
          "Cannot transition from '" + ACTIVE_STATE + "' to '" +
              OBSERVER_STATE + "'");
    }
    state.setState(haContext, OBSERVER_STATE);
  }

  synchronized HAServiceStatus getServiceStatus()
      throws ServiceFailedException, AccessControlException {
    if (!haEnabled) {
      throw new ServiceFailedException("HA for namenode is not enabled");
    }
    if (state == null) {
      return new HAServiceStatus(HAServiceState.INITIALIZING);
    }
    HAServiceState retState = state.getServiceState();
    HAServiceStatus ret = new HAServiceStatus(retState);
    if (retState == HAServiceState.STANDBY) {
      if (namesystem.isInSafeMode()) {

View on GitHub (pinned to 2add963021)

Solutions

1. Go active -> standby -> observer: 'hdfs haadmin -transitionToStandby <nnId>' (after ensuring/failing over another NN to active), then 'hdfs haadmin -transitionToObserver <nnId>'.
2. In automatic-failover deployments, trigger failover so a standby becomes active, then demote the old active to observer.
3. Check current role first with 'hdfs haadmin -getServiceStatus <nnId>' and never issue observer transitions to an ACTIVE node.

Example fix

# before
hdfs haadmin -transitionToObserver nn1   # nn1 is Active -> rejected

# after
hdfs haadmin -transitionToStandby nn1
hdfs haadmin -transitionToObserver nn1   # active -> standby -> observer

// never send observer transitions to an ACTIVE nn
HAServiceStatus st = haProxy.getServiceStatus(nnId);
if ("ACTIVE".equals(String.valueOf(st.getState())))
  throw new IllegalStateException("demote to standby before transitionToObserver");
haProxy.transitionToObserver(nnId);

boolean isActiveToObserverDenied(Throwable t) {
  return t instanceof ServiceFailedException
      && String.valueOf(t.getMessage()).contains("'active' to 'observer'");
}

When it happens

Trigger: 'hdfs haadmin -transitionToObserver <nnId>' while that NameNode is the active node of the nameservice - e.g., trying to convert the current primary into a read node in one step.

Common situations: Reshaping a cluster's roles (want fewer actives, more observers) and issuing observer transitions against the still-active NN; manual drills that skip the standby step; automation assuming any-to-any transitions.

Related errors

• Cannot transition from 'observer' to 'active'
• {getRole()} still not leave safemode
• Failed to start active services
• {} is in observer state. Cannot be failover target
• Block pool {bpid} has not recognized an active NN

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a04ecdff15489641. Report an issue: GitHub.