What causes "Service {} is in wrong state: {}" in apache/hadoop?

Create clients via RegistryOperationsFactory.createAnonymousInstance/createKerberosInstance/createAuthenticatedInstance, which start the service before returning it.

apache/hadoop · error · ServiceStateException

Service {} is in wrong state: {}

Error message

Service {} is in wrong state: {}

What it means

Every zk*() operation on CuratorService begins with checkServiceLive(), which requires the Hadoop CompositeService to be in STATE.STARTED; any other state (NOTINITED, INITED, STOPPED) throws ServiceStateException('Service <name> is in wrong state: <state>'). It is a lifecycle precondition failure: the Curator/ZooKeeper client does not exist yet, or has been torn down, so the operation is refused before any ZK call is made.

Source

Thrown at hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/client/impl/zk/CuratorService.java:206

   */
  @Override
  protected void serviceStop() throws Exception {
    IOUtils.closeStream(curator);

    if (curatorCacheBridge != null) {
      curatorCacheBridge.close();
    }
    super.serviceStop();
  }

  /**
   * Internal check that a service is in the live state.
   *
   * @throws ServiceStateException if not
   */
  private void checkServiceLive() throws ServiceStateException {
    if (!isInState(STATE.STARTED)) {
      throw new ServiceStateException(
          "Service " + getName() + " is in wrong state: "
              + getServiceState());
    }
  }

  /**
   * Flag to indicate whether or not the registry is secure.
   * Valid once the service is inited.
   *
   * @return service security policy
   */
  public boolean isSecure() {
    return registrySecurity.isSecureRegistry();
  }

  /**
   * Get the registry security helper.
   *

View on GitHub (pinned to 2add963021)

Solutions

Create clients via RegistryOperationsFactory.createAnonymousInstance/createKerberosInstance/createAuthenticatedInstance, which start the service before returning it.
If constructing manually, call init(conf) then start() and wait for completion before issuing any operation.
Never reuse an instance after stop(); build and start a fresh one.

Example fix

// before
CuratorService curator = new CuratorService("registry");
curator.init(conf);
curator.zStat("/services/api"); // ServiceStateException: wrong state INITED

// after: init + start, or simply
RegistryOperations ops = RegistryOperationsFactory.createAnonymousInstance(conf);

Defensive patterns

Strategy: validation

Validate before calling

if (!registryOperations.isInState(STATE.STARTED)) {
  throw new IllegalStateException("Registry client not started; call init(conf) + start() first");
}
// then issue operations

Type guard

// narrow to a started service before issuing zk operations
private static boolean isLive(Service service) {
  return service.isInState(STATE.STARTED);
}

Try / catch

try {
  curatorService.zStat(path);
} catch (ServiceStateException e) {
  // lifecycle bug: start the service (or rebuild via RegistryOperationsFactory) before retrying
}

Prevention

Always build registry clients with RegistryOperationsFactory — it creates, inits and starts in one call.
Do not issue registry operations from your own serviceInit(); wait until serviceStart has completed.
Never reuse a stopped client; construct and start a new instance instead.

When it happens

Trigger: Instantiating CuratorService/RegistryOperationsService with new, calling init(conf), then issuing zkStat/zkCreate/etc. without start(); invoking operations after serviceStop(); issuing calls from another thread while startup is still in progress; calling registry methods from your own service's init phase.

Common situations: Not using RegistryOperationsFactory (whose create* methods create, init and start the instance); ordering bugs where a dependent service uses the registry before it starts; reusing a stopped client after shutdown.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0a780c4824a0cfeb. Report an issue: GitHub.