apache/hadoop · error · ServiceFailedException

Failed to stop active services

Error message

Failed to stop active services

What it means

ActiveState.exitState() delegates to HAContext.stopActiveServices() and wraps any IOException as ServiceFailedException('Failed to stop active services'). It occurs while deactivating an ACTIVE NameNode (transition to standby or shutdown); the underlying cause in the exception identifies which service failed to stop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/ActiveState.java:72

    }
    super.setState(context, s);
  }

  @Override
  public void enterState(HAContext context) throws ServiceFailedException {
    try {
      context.startActiveServices();
    } catch (IOException e) {
      throw new ServiceFailedException("Failed to start active services", e);
    }
  }

  @Override
  public void exitState(HAContext context) throws ServiceFailedException {
    try {
      context.stopActiveServices();
    } catch (IOException e) {
      throw new ServiceFailedException("Failed to stop active services", e);
    }
  }

}

View on GitHub (pinned to 2add963021)

Solutions

1. Read the wrapped cause in the NameNode log to identify the failing subsystem
2. Verify shared journal (QJM) health and retry the transition after fixing it
3. If the node is wedged between states, restart the NameNode so HA state is rebuilt cleanly
4. Avoid issuing concurrent manual transitions and automatic failover to the same node

try {
  haAdmin.transitionToStandby(nn);
} catch (ServiceFailedException e) { // "Failed to stop active services"
  Throwable root = e.getCause();
  LOG.error("Standby transition failed, node may be wedged: {}", root, root);
  // safest recovery: restart this NameNode so HA state is rebuilt
}

When it happens

Trigger: 'hdfs haadmin -transitionToStandby', failover, or shutdown calls stopActiveServices() and an IOException escapes — e.g., journal manager close failure, checkpointer/upload thread failing to terminate cleanly, storage flush errors on the way down.

Common situations: Storage or QJM trouble already present surfaces during teardown; failover under load where threads race shutdown; double transitions triggered by both ZKFC and admin CLI.

Related errors

• Failed to start active services
• HA for namenode is not enabled
• Cannot transition from 'observer' to 'active'
• {getRole()} still not leave safemode
• Cannot transition from 'active' to 'observer'

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f3620ff7114c9c0d. Report an issue: GitHub.