What causes "Transition from state {} to {} is not allowed." in apache/hadoop?

Route multi-step changes through standby: run -transitionToStandby first, confirm with 'hdfs haadmin -getServiceState', then -transitionToObserver (or -transitionToActive)

apache/hadoop · error · ServiceFailedException

Transition from state {} to {} is not allowed.

Error message

Transition from state {} to {} is not allowed.

What it means

HAState is the base of the NameNode HA state machine. Only specific transitions are legal: ActiveState permits only Active->Standby; StandbyState permits Standby->Active and Standby->Observer; an observer (StandbyState with isObserver=true) permits Observer->Standby. The base HAState.setState() throws ServiceFailedException for every other from/to pair, so any transition not modelled by a subclass override is rejected.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/HAState.java:140

   * Method to be overridden by subclasses to perform steps necessary for
   * exiting a state.
   * @param context HA context
   * @throws ServiceFailedException on failure to enter the state.
   */
  public abstract void exitState(final HAContext context)
      throws ServiceFailedException;

  /**
   * Move from the existing state to a new state
   * @param context HA context
   * @param s new state
   * @throws ServiceFailedException on failure to transition to new state.
   */
  public void setState(HAContext context, HAState s) throws ServiceFailedException {
    if (this == s) { // Already in the new state
      return;
    }
    throw new ServiceFailedException("Transition from state " + this + " to "
        + s + " is not allowed.");
  }
  
  /**
   * Check if an operation is supported in a given state.
   * @param context HA context
   * @param op Type of the operation.
   * @throws StandbyException if a given type of operation is not
   *           supported in standby state
   */
  public abstract void checkOperation(final HAContext context, final OperationCategory op)
      throws StandbyException;

  public abstract boolean shouldPopulateReplQueues();

  /**
   * @return String representation of the service state.
   */

View on GitHub (pinned to 2add963021)

Solutions

Route multi-step changes through standby: run -transitionToStandby first, confirm with 'hdfs haadmin -getServiceState', then -transitionToObserver (or -transitionToActive)
Always read the current state before requesting a transition instead of assuming it
If automatic failover is enabled, stop the target NN's ZKFC before manual transitions to avoid racing state changes
In code, call transitionTo* on the target NN and on ServiceFailedException re-read HAServiceState before re-planning the path

Example fix

# before: active -> observer in one step (rejected)
hdfs haadmin -transitionToObserver nn2
# after: two legal steps
hdfs haadmin -transitionToStandby nn2
hdfs haadmin -getServiceState nn2   # must print standby
hdfs haadmin -transitionToObserver nn2

Defensive patterns

Strategy: try-catch

Validate before calling

// Read actual state before planning a transition
HAServiceProtocol proxy = HAUtil.getProxiesForAllNNs... // or your resolved proxy
HAServiceState cur = proxy.getServiceStatus().getState();
if (cur == HAServiceState.ACTIVE && target == HAServiceState.OBSERVER) {
  // plan two steps: ACTIVE -> STANDBY -> OBSERVER
}

Try / catch

try {
  nameNode.setState(targetState);
} catch (ServiceFailedException e) {
  // re-read state, re-plan a legal path (via STANDBY), then retry once
  HAServiceState cur = readServiceState();
  LOG.warn("transition to " + targetState + " rejected from " + cur, e);
  transitionViaStandby(cur, targetState);
}

Prevention

Model transitions as a graph (only the legal edges) in failover tooling instead of free-form commands
Script 'check state, then transition, then verify state' as one unit
Prefer automatic failover (ZKFC) over chained manual haadmin commands
Treat OBSERVER as reachable only from STANDBY in runbooks

When it happens

Trigger: Requesting a transition the current state cannot perform in one step, typically ACTIVE -> OBSERVER directly ('hdfs haadmin -transitionToObserver' on an active NN) instead of via STANDBY; also custom tooling calling HAServiceProtocol.transitionToStandby/Active/Observer in a sequence the state machine does not allow.

Common situations: Manual failover drills whose haadmin commands are chained in the wrong order; runbooks written before observer support that now target observers; ZKFC racing a manual transition; admin scripts assuming any-to-any transitions are legal.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/55d73f34538cf0fd. Report an issue: GitHub.