thanos-io/thanos · error

resolution against server

Error message

resolution against server %s for %s

What it means

This is a wrapping applied to every individual error encountered while querying one DNS server during lookupFromAnyServer. Each failure to exchange a query with a server (timeout, connection error, malformed reply) is wrapped with the server address and queried name, collected, and later surfaced in the aggregated messages of errors 562/564.

Solutions

  1. Identify the failing server from the wrapped message ('resolution against server X for Y') and test it directly with dig @X Y
  2. Remove or fix unreachable nameserver entries in resolv.conf
  3. Check network policies/firewalls allow UDP/TCP 53 to the resolver
  4. Ensure the resolver IP is valid for your network (no stale VPN or cluster-DNS addresses)
Defensive patterns

Strategy: retry

Validate before calling

conn, err := net.DialTimeout("udp", net.JoinHostPort(server, "53"), 2*time.Second)
if err != nil { /* server unreachable; fix resolv.conf or routing before lookups */ } else { conn.Close() }

Try / catch

if err != nil && strings.Contains(err.Error(), "resolution against server") {
    logger.Warn("DNS server failing", "err", err) // treat per-server; retry whole lookup
}

Prevention

When it happens

Trigger: askServerForName returns an error for servAddr net.JoinHostPort(server, conf.Port) — e.g. UDP/TCP timeout, ICMP port unreachable, or connection refused — for any server iterated in conf.Servers.

Common situations: Dead or misconfigured nameserver entry in resolv.conf; DNS port blocked by network policy; IPv6 nameserver unreachable from an IPv4-only pod; transient CoreDNS restarts.

Related errors


AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/ee05254cb99e23df. Report an issue: GitHub.

Appendix: source

Thrown at pkg/discovery/dns/miekgdns/lookup.go:105

//     (RCODE==SUCCESS, ANCOUNT > 0);
//  2. "yes, I know that name, but it has no records of the requested type"
//     (RCODE==SUCCESS, ANCOUNT==0); or
//  3. "I know that name doesn't exist" (RCODE==NXDOMAIN).
//
// A non-viable answer is "anything else", which encompasses both various
// system-level problems (like network timeouts) and also
// valid-but-unexpected DNS responses (SERVFAIL, REFUSED, etc).
func lookupFromAnyServer(name string, qtype dns.Type, conf *dns.ClientConfig) (*dns.Msg, error) {
	client := &dns.Client{}

	var errs []error

	// TODO(bwplotka): Worth to do fanout and grab fastest as golang native lib?
	for _, server := range conf.Servers {
		servAddr := net.JoinHostPort(server, conf.Port)
		msg, err := askServerForName(name, qtype, client, servAddr, true)
		if err != nil {
			errs = append(errs, errors.Wrapf(err, "resolution against server %s for %s", server, name))
			continue
		}

		if msg.Rcode == dns.RcodeSuccess || msg.Rcode == dns.RcodeNameError {
			return msg, nil
		}
	}

	return nil, errors.Errorf("could not resolve %s: no servers returned a viable answer. Errs %v", name, fmtErrs(errs))
}

func fmtErrs(errs []error) string {
	b := bytes.Buffer{}
	for _, err := range errs {
		b.WriteString(";")
		b.WriteString(err.Error())
	}
	return b.String()

View on GitHub (pinned to 35b8b99117)