{"record":{"id":"9334912720c22860","repo":"apache/druid","slug":"resetting-server-s-with-state-s-as-it-is-not-s","errorCode":null,"errorMessage":"Resetting server[%s] with state[%s] as it is not syncing properly.","messagePattern":"Resetting server\\[(.+?)\\] with state\\[(.+?)\\] as it is not syncing properly\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"server/src/main/java/org/apache/druid/client/HttpServerInventoryView.java","lineNumber":483,"sourceCode":"          e.getKey(),\n          serverHolder.syncer.getDebugInfo()\n      );\n    }\n    return result;\n  }\n\n  @VisibleForTesting\n  void checkAndResetUnhealthyServers()\n  {\n    // Ensure that the collection is not being modified during iteration. Iterate over a copy\n    final Set<Map.Entry<String, DruidServerHolder>> serverEntrySet = ImmutableSet.copyOf(servers.entrySet());\n    for (Map.Entry<String, DruidServerHolder> e : serverEntrySet) {\n      DruidServerHolder serverHolder = e.getValue();\n      if (serverHolder.syncer.needsReset()) {\n        synchronized (servers) {\n          // Reset only if the server is still present in the map\n          if (servers.containsKey(e.getKey())) {\n            log.warn(\n                \"Resetting server[%s] with state[%s] as it is not syncing properly.\",\n                serverHolder.druidServer.getName(),\n                serverHolder.syncer.getDebugInfo()\n            );\n            serverRemoved(serverHolder.druidServer);\n            serverAdded(serverHolder.druidServer.copyWithoutSegments());\n          }\n        }\n      }\n    }\n  }\n\n  private void emitServerStatusMetrics()\n  {\n    final ServiceMetricEvent.Builder eventBuilder = ServiceMetricEvent.builder();\n    try {\n      final Map<String, DruidServerHolder> serversCopy = ImmutableMap.copyOf(servers);\n      serversCopy.forEach((serverName, serverHolder) -> {","sourceCodeStart":465,"sourceCodeEnd":501,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/server/src/main/java/org/apache/druid/client/HttpServerInventoryView.java#L465-L501","documentation":"HttpServerInventoryView tracks Druid servers and their segment inventories via per-server syncers. When a server's ServerInventorySyncer reports it is not syncing properly (needsReset()), the view removes and re-adds the server with an empty segment list so the sync can restart cleanly. This log line warns that such a reset is being performed, which temporarily empties the server's inventory.","triggerScenarios":"The checkAndResetUnhealthyServers periodic task finds serverHolder.syncer.needsReset() true, typically because the server missed consecutive sync polls or returned malformed/failed sync responses (full or delta).","commonSituations":"Historical/broker processes overloaded and missing sync intervals; network partitions between coordinator-adjacent inventory consumers and data servers; data server restarts mid-sync; misconfigured sync polling intervals too aggressive for cluster size.","solutions":["Check logs on the affected data server for sync endpoint failures (full/delta sync errors) and fix the root cause (GC pauses, network, overloading).","Increase the sync poll interval or adjust unhealthy-hold/reset configuration so healthy-but-slow servers are not reset.","Verify the affected server is not overloaded; scale it or reduce segment counts to keep sync responses fast.","If resets are transient and self-heal, treat as noise; if persistent, restart the data server and investigate connectivity."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if (serverHolder.syncer.needsReset()) { log.warn(\"Server {} sync unhealthy; reset imminent\", serverHolder.druidServer.getName()); }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Monitor sync latency and failure logs on data servers","Size sync intervals for worst-case server load","Alert on repeated reset warnings for the same server"],"tags":["druid","cluster-coordination","segment-inventory","sync"],"backgroundTag":"invalid-state-transition","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}