nats-io/nats-server · error

group node unhealthy

Error message

group node unhealthy

What it means

Returned by jetStream.isStreamHealthy as the final check: the stream's Raft group node reports itself unhealthy (node.Healthy() is false), meaning the peer has not had recent enough contact (heartbeats/elections) with its raft group to be considered a healthy participant.

Source

Thrown at server/jetstream_cluster.go:1069

	case node != msetNode:
		s.Warnf("Detected stream cluster node skew '%s > %s'", acc.GetName(), streamName)
		return errors.New("cluster node skew detected")

	case nrgWerr != nil:
		return fmt.Errorf("node write error: %v", nrgWerr)

	case streamWerr != nil:
		return fmt.Errorf("stream write error: %v", streamWerr)

	case !mset.isMonitorRunning():
		return errors.New("monitor goroutine not running")

	case mset.isCatchingUp():
		return errors.New("stream catching up")

	case !node.Healthy():
		return errors.New("group node unhealthy")

	default:
		return nil
	}
}

// isConsumerHealthy will determine if the consumer is up to date.
// For R1 it will make sure the consunmer is present on this server.
func (js *jetStream) isConsumerHealthy(mset *stream, consumer string, ca *consumerAssignment) error {
	js.mu.RLock()
	if ca != nil && ca.unsupported != nil {
		js.mu.RUnlock()
		return nil
	}
	if mset == nil {
		js.mu.RUnlock()
		return errors.New("stream missing")
	}

View on GitHub (pinned to 3a66a489d2)

Solutions

  1. Check network connectivity and cluster route ports between all stream peers
  2. Confirm enough replicas are running to form quorum (R1 lacks this check; R>1 needs a majority)
  3. Inspect raft/election warnings in server logs; restart or reconnect the unhealthy node
Defensive patterns

Strategy: retry

Try / catch

if err != nil && strings.Contains(err.Error(), "group node unhealthy") {
    // check peer connectivity/quorum before escalating
    // retry, and alert if consecutive failures exceed threshold
    time.Sleep(backoff)
    return healthcheck()
}

Prevention

When it happens

Trigger: HEALTHCHECK on a peer that lost quorum/leader, has network issues to other peers, is suspended, or whose raft node stopped making progress (e.g. elections timing out).

Common situations: Network partitions between JetStream peers; too few live replicas (no quorum); overloaded server missing raft heartbeats; firewall blocking cluster routes.

Related errors


AI-assisted analysis of nats-io/nats-server@3a66a489d2 (2026-09-02). Data as JSON: /api/errors/47eef5a36495bc1d. Report an issue: GitHub.