apache/druid · warning

Watcher for node role

Error message

Watcher for node role [%s] returned an empty collection.

What it means

BaseNodeRoleWatcher.getAllNodes() waits for cache initialization (awaitInitialization) and, if the resulting node collection is empty, logs a warning before returning it. An empty result means no ZK ephemeral nodes exist for that role — either genuinely no nodes of the role are online, or the cache never populated.

Solutions

1. Confirm the target role's services are running and announced under the expected ZK discovery path (check /druid/discovery/... in zkCli).
2. Check for a preceding InterruptedException or initialization timeout log indicating the cache never fully initialized.
3. Verify druid.discovery.zk.paths base configuration matches the cluster the nodes announce to.
4. If nodes were just started, retry after discovery propagation; the empty result may be a startup race.

Example fix

// before
Collection<DruidNodeDiscovery.Node> nodes = discovery.getForNodeRole(NodeRole.COORDINATOR).getAllNodes();
// use nodes assuming non-empty
// after
if (nodes.isEmpty()) {
  throw new ISE("No coordinator nodes discovered; check ZK discovery path and coordinator availability");
}

DruidNodeDiscovery disco = discovery.getForNodeRole(nodeRole);
Collection<Node> nodes = disco.getAllNodes();
if (nodes.isEmpty()) {
  throw new ISE("No nodes of role [%s] discovered; check ZK path and service availability", nodeRole.getJsonName());
}

When it happens

Trigger: Calling getAllNodes() (e.g. via DruidNodeDiscovery.presentNodes) for a role with no announced nodes, or when initialization timed out/interrupted (see the swallowed InterruptedException) leaving the cache empty.

Common situations: Querying for a service role that isn't deployed (e.g. asking for overlord nodes in a cluster without one), ZK path misconfiguration, connectivity issues preventing cache population, or calling presentNodes immediately at startup before peers announce.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

• Node [ ] of role [ ] went offline.
• Cache for node role [ ] could not be initialized before…
• Cache initialization for node role
• can't start.
• can't stop.

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/e8dc0975df2b9237. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/discovery/BaseNodeRoleWatcher.java:102

      ScheduledExecutorService listenerExecutor,
      NodeRole nodeRole
  )
  {
    BaseNodeRoleWatcher nodeRoleWatcher = new BaseNodeRoleWatcher(listenerExecutor, nodeRole);
    nodeRoleWatcher.scheduleTimeout(DEFAULT_TIMEOUT_SECONDS);
    return nodeRoleWatcher;
  }

  public Collection<DiscoveryDruidNode> getAllNodes()
  {
    try {
      awaitInitialization();
    }
    catch (InterruptedException ex) {
      Thread.currentThread().interrupt();
    }
    if (unmodifiableNodes.isEmpty()) {
      LOGGER.warn("Watcher for node role [%s] returned an empty collection.", nodeRole.getJsonName());
    }
    return unmodifiableNodes;
  }

  public void registerListener(DruidNodeDiscovery.Listener listener)
  {
    synchronized (lock) {
      // No need to wait on CountDownLatch, because we are holding the lock under which it could only be counted down.
      if (cacheInitialized.getCount() == 0) {
        // It is important to take a snapshot here as list of nodes might change by the time listeners process
        // the changes.
        List<DiscoveryDruidNode> currNodes = Lists.newArrayList(nodes.values());
        safeSchedule(
            () -> {
              listener.nodesAdded(currNodes);
              if (cacheInitializationTimedOut) {
                listener.nodeViewInitializedTimedOut();
              } else {

View on GitHub (pinned to 9b90983fd2)