vitessio/vitess · info

Connection DNS for %s has changed; old: %v new: %v

Error message

Connection DNS for %s has changed; old: %v  new: %v

What it means

DNSTracker detects that the IPs behind a tracked hostname have changed since the connection was created and returns this error to signal the caller that the connection must be re-established. It is an intentional signal, not a malfunction: DNS rebalancing or failover changed the target addresses.

Source

Thrown at go/netutil/netutil.go:150

	if dnsName != "" {
		addrs, _ = dnsLookup(dnsName)
	}

	return func() (bool, error) {
		if dnsName == "" {
			return false, nil
		}
		newaddrs, err := dnsLookup(dnsName)
		if err != nil {
			return false, err
		}
		if len(newaddrs) == 0 { // Should not happen, but just in case
			return false, fmt.Errorf("Connection DNS for %s reporting as empty, ignoring", dnsName)
		}
		if !addrEqual(addrs, newaddrs) {
			oldaddr := addrs
			addrs = newaddrs // Update the closure variable
			return true, fmt.Errorf("Connection DNS for %s has changed; old: %v  new: %v", dnsName, oldaddr, newaddrs)
		}
		return false, nil
	}
}

func addrEqual(a, b []net.IP) bool {
	if len(a) != len(b) {
		return false
	}
	for idx, v := range a {
		if !net.IP.Equal(v, b[idx]) {
			return false
		}
	}
	return true
}

// NormalizeIP normalizes loopback addresses to avoid spurious errors when

View on GitHub (pinned to 01a25a7d17)

Solutions

  1. Treat the returned error as expected: close and recreate the connection to the new address (the tracker's purpose is to trigger this)
  2. Ensure callers honor the (true, err) return and retry the dial; don't suppress it
  3. Increase DNS TTL stability or pin records if changes are unintentional
  4. Check why the record changed (failover vs. misconfigured DNS automation)

Example fix

// before (ignoring refresh signal)
changed, _ := tracker.Check(); _ = changed
// after
changed, err := tracker.Check()
if changed || err != nil {
    conn.Close()
    conn = dial(addr) // reconnect to new IPs
}
Defensive patterns

Strategy: retry

Try / catch

changed, err := tracker.Check()
if changed {
    conn.Close()
    conn, err = dial(addr) // DNS changed: reconnect to new IPs
    if err != nil {
        return retry.Do(func() error { conn, err = dial(addr); return err })
    }
}

Prevention

When it happens

Trigger: checkAndRefreshDNS finds addrEqual(addrs, newaddrs) false — e.g. a tablet was rescheduled (new pod IP), a DNS record was updated for failover, or a load-balancer endpoint changed — during a DNSTracker refresh of an existing connection.

Common situations: Kubernetes pod restarts getting new IPs; DNS-based failover switching to a standby; TTL expiry after record changes; rolling updates changing service endpoint IPs.

Related errors


AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/da21dd775f4b8814. Report an issue: GitHub.