dgraph-io/dgraph · critical

Unhealthy connection to %v

Error message

Unhealthy connection to %v

What it means

When starting a Zero node with an explicit --peer, initAndStartNode obtains a connection pool to that peer via conn.GetPools().Connect. If no healthy connection can be established, it returns this error before attempting to join the raft group — Zero cannot talk to the specified peer at all.

Source

Thrown at dgraph/cmd/zero/raft.go:666

			n.server.SetMembershipState(zs.State)
			for _, id := range sp.Metadata.ConfState.Voters {
				n.Connect(id, zs.State.Zeros[id].Addr)
			}
		}

		n.SetRaft(raft.RestartNode(n.Cfg))
		foundCID, err := n.checkForCIDInEntries()
		if err != nil {
			return err
		}
		if !foundCID {
			go n.proposeNewCID()
		}

	case len(opts.peer) > 0:
		p := conn.GetPools().Connect(opts.peer, opts.tlsClientConfig)
		if p == nil {
			return errors.Errorf("Unhealthy connection to %v", opts.peer)
		}

		timeout := 8 * time.Second
		for {
			c := pb.NewRaftClient(p.Get())
			ctx, cancel := context.WithTimeout(n.ctx, timeout)
			// JoinCluster can block indefinitely, raft ignores conf change proposal
			// if it has pending configuration.
			_, err := c.JoinCluster(ctx, n.RaftContext)
			if err == nil {
				cancel()
				break
			}
			if x.ShouldCrash(err) {
				cancel()
				log.Fatalf("Error while joining cluster: %v", err)
			}
			glog.Errorf("Error while joining cluster: %v\n", err)

View on GitHub (pinned to 759e242be6)

Solutions

1. Verify the peer Zero is running and reachable at the exact host:port (e.g. nc/curl the gRPC port 5080).
2. Check --tls_client_config flags match the peer's TLS setup.
3. Fix DNS/firewall so the peer address resolves and the port is open.
4. Start the peer Zero before this node, or remove --peer to bootstrap a new group.

Example fix

// before
dgraph zero --peer "zero-1.local:5080"  # zero-1 not up yet -> Unhealthy connection
// after
# ensure peer is running first, then:
dgraph zero --peer "zero-1.local:5080" --my "zero-2.local:5080"

// Go: verify peer reachability before starting zero
conn, err := net.DialTimeout("tcp", peerAddr, 5*time.Second)
if err != nil {
	return fmt.Errorf("peer %s unreachable: %w", peerAddr, err)
}
conn.Close()

func healthyPool(p *pool.Pool) bool { return p != nil }

err := startZero()
for i := 0; err != nil && i < 5; i++ {
	if strings.Contains(err.Error(), "Unhealthy connection") {
		time.Sleep(time.Duration(1<<i) * time.Second) // peer may still be booting
		err = startZero()
		continue
	}
	break
}

When it happens

Trigger: Starting dgraph zero with --peer set to an address where no Zero is listening, the peer is down, TLS client config mismatches the peer, or the network/firewall blocks the gRPC port.

Common situations: Typo in --peer host:port; peer Zero not yet started (ordering issue in scripts); wrong port (default 5080); k8s service DNS not resolving; mTLS configured on one side only.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• unable to reach quorum
• failed to connect to endpoint [%s]: %w
• failed to initiate external snapshot stream: %v
• failed to turn off drain mode: %v
• failed to disable drain mode: %v

AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/7452ad061e33ead5. Report an issue: GitHub.