{"record":{"id":"3a1219329733ee1a","repo":"apache/druid","slug":"cache-for-node-role-s-could-not-be-initialized","errorCode":null,"errorMessage":"Cache for node role [%s] could not be initialized before timeout. This service may not have full information about other nodes of type [%s].","messagePattern":"Cache for node role \\[(.+?)\\] could not be initialized before timeout\\. This service may not have full information about other nodes of type \\[(.+?)\\]\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"server/src/main/java/org/apache/druid/discovery/BaseNodeRoleWatcher.java","lineNumber":284,"sourceCode":"  }\n\n  private void cacheInitializedTimedOut()\n  {\n    synchronized (lock) {\n      // No need to wait on CountDownLatch, because we are holding the lock under which it could only be\n      // counted down.\n      if (cacheInitialized.getCount() != 0) {\n        cacheInitialized(true);\n      }\n    }\n  }\n\n  // This method is called only once with either timedOut = true or false, but not both.\n  @GuardedBy(\"lock\")\n  private void cacheInitialized(boolean timedOut)\n  {\n    if (timedOut) {\n      LOGGER.warn(\n          \"Cache for node role [%s] could not be initialized before timeout. \"\n          + \"This service may not have full information about other nodes of type [%s].\",\n          nodeRole.getJsonName(),\n          nodeRole.getJsonName()\n      );\n      cacheInitializationTimedOut = true;\n    }\n\n    // It is important to take a snapshot here as list of nodes might change by the time listeners process\n    // the changes.\n    List<DiscoveryDruidNode> currNodes = Lists.newArrayList(nodes.values());\n    LOGGER.info(\n        \"Node watcher of role [%s] is now initialized with %d nodes.\",\n        nodeRole.getJsonName(),\n        currNodes.size());\n\n    for (DruidNodeDiscovery.Listener listener : nodeListeners) {\n      safeSchedule(","sourceCodeStart":266,"sourceCodeEnd":302,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/server/src/main/java/org/apache/druid/discovery/BaseNodeRoleWatcher.java#L266-L302","documentation":"Log warning from BaseNodeRoleWatcher.cacheInitialized(timedOut=true), fired when the service-discovery cache for a node role was not populated within the configured timeout. The service marks cacheInitializationTimedOut and continues operating, possibly without complete information about other nodes of the same role. Late initialization events are then ignored.","triggerScenarios":"The CountDownLatch for cache population is not counted down before the scheduled timeout task runs cacheInitialized(true); e.g. no nodes of the role announced, or the watcher's addListener/child watchers have not delivered updates within druid.discovery.cache.timeoutMillis.","commonSituations":"ZooKeeper connectivity problems or session instability at startup; configured timeout too small for cluster size; a role with zero instances so the 'at least one node' expectation never completes; heavy startup load delaying curator cache population.","solutions":["Check ZooKeeper connectivity and curator session state in the logs at startup.","Raise druid.discovery.cache.timeoutMillis (e.g. from PT10S to PT1M) to accommodate slow clusters.","Confirm nodes of the role are actually running and announcing their services via the coordinator/announcer.","Restart the service once discovery is healthy to re-attempt cache initialization."],"exampleFix":"// before\ndruid.discovery.cache.timeoutMillis=PT10S\n// after\ndruid.discovery.cache.timeoutMillis=PT1M","handlingStrategy":"validation","validationCode":"// Verify ZK reachability before/at startup and that the cache timeout is adequate\nif (!zkHealthCheck.await(30, SECONDS)) {\n  throw new IllegalStateException(\"ZooKeeper not reachable; discovery cache will time out\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Set druid.discovery.cache.timeoutMillis generously (minutes, not seconds)","Check ZK session stability metrics at startup","Ensure expected node roles are actually running and announcing","Restart services after discovery-related incidents to reinitialize caches"],"tags":["service-discovery","timeout","zookeeper"],"backgroundTag":"request-timeout","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}