{"record":{"id":"47eef5a36495bc1d","repo":"nats-io/nats-server","slug":"group-node-unhealthy","errorCode":null,"errorMessage":"group node unhealthy","messagePattern":"group node unhealthy","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"server/jetstream_cluster.go","lineNumber":1069,"sourceCode":"\n\tcase node != msetNode:\n\t\ts.Warnf(\"Detected stream cluster node skew '%s > %s'\", acc.GetName(), streamName)\n\t\treturn errors.New(\"cluster node skew detected\")\n\n\tcase nrgWerr != nil:\n\t\treturn fmt.Errorf(\"node write error: %v\", nrgWerr)\n\n\tcase streamWerr != nil:\n\t\treturn fmt.Errorf(\"stream write error: %v\", streamWerr)\n\n\tcase !mset.isMonitorRunning():\n\t\treturn errors.New(\"monitor goroutine not running\")\n\n\tcase mset.isCatchingUp():\n\t\treturn errors.New(\"stream catching up\")\n\n\tcase !node.Healthy():\n\t\treturn errors.New(\"group node unhealthy\")\n\n\tdefault:\n\t\treturn nil\n\t}\n}\n\n// isConsumerHealthy will determine if the consumer is up to date.\n// For R1 it will make sure the consunmer is present on this server.\nfunc (js *jetStream) isConsumerHealthy(mset *stream, consumer string, ca *consumerAssignment) error {\n\tjs.mu.RLock()\n\tif ca != nil && ca.unsupported != nil {\n\t\tjs.mu.RUnlock()\n\t\treturn nil\n\t}\n\tif mset == nil {\n\t\tjs.mu.RUnlock()\n\t\treturn errors.New(\"stream missing\")\n\t}","sourceCodeStart":1051,"sourceCodeEnd":1087,"githubUrl":"https://github.com/nats-io/nats-server/blob/3a66a489d262bf89b71a71c955c94920394532f3/server/jetstream_cluster.go#L1051-L1087","documentation":"Returned by jetStream.isStreamHealthy as the final check: the stream's Raft group node reports itself unhealthy (node.Healthy() is false), meaning the peer has not had recent enough contact (heartbeats/elections) with its raft group to be considered a healthy participant.","triggerScenarios":"HEALTHCHECK on a peer that lost quorum/leader, has network issues to other peers, is suspended, or whose raft node stopped making progress (e.g. elections timing out).","commonSituations":"Network partitions between JetStream peers; too few live replicas (no quorum); overloaded server missing raft heartbeats; firewall blocking cluster routes.","solutions":["Check network connectivity and cluster route ports between all stream peers","Confirm enough replicas are running to form quorum (R1 lacks this check; R>1 needs a majority)","Inspect raft/election warnings in server logs; restart or reconnect the unhealthy node"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err != nil && strings.Contains(err.Error(), \"group node unhealthy\") {\n    // check peer connectivity/quorum before escalating\n    // retry, and alert if consecutive failures exceed threshold\n    time.Sleep(backoff)\n    return healthcheck()\n}","preventionTips":["Maintain reliable network routes between all JetStream peers","Always run at least quorum-sized replica counts (3 for R3)","Alert on sustained raft unhealthiness rather than single health-check failures"],"tags":["jetstream","raft","healthcheck","network"],"backgroundTag":"jetstream-raft-node-unhealthy","analyzedSha":"3a66a489d262bf89b71a71c955c94920394532f3","analyzedAt":"2026-09-02T04:41:54.247Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}