thanos-io/thanos · error

could not resolve : all servers responded with errors to at…

Error message

could not resolve %q: all servers responded with errors to at least one search domain. Errs %s

What it means

Returned by lookupWithSearchPath when at least one queried DNS server responded with an actual error (not NXDOMAIN) for every name permutation tried — i.e. the resolution failed for infrastructural reasons rather than 'host not found'. The message aggregates all underlying per-server errors via fmtErrs for diagnosis.

Solutions

1. Read the aggregated 'Errs ...' tail of the message to see the per-server root causes
2. Check reachability of the nameservers in resolv.conf: dig @<server> <name> for each one
3. Restore connectivity to the DNS servers (VPN, firewall rules, CoreDNS/kube-dns health)
4. Verify the search domains and the queried name are correct; retry once connectivity is restored

// pre-check the configured nameservers respond
for _, ns := range nameserversFromResolvConf() {
    if err := probeDNS(ns); err != nil { /* surface unreachable server early */ }
}

resp, err := r.LookupSRV(ctx, svc, proto, name)
if err != nil {
    if strings.Contains(err.Error(), "all servers responded with errors") {
        return retry.Do(func() error { return resolveAgain() }, retry.Attempts(3), retry.Delay(2*time.Second))
    }
    return err
}

When it happens

Trigger: All servers in resolv.conf either time out, refuse connections, or return failure rcodes for every search-path variant of the name, leaving errs non-empty in lookupWithSearchPath.

Common situations: DNS server IP in resolv.conf is stale/unreachable (VPN down, cluster DNS pod down); firewall blocking UDP/TCP 53; resolv.conf pointing at a resolver that drops queries; ndots/search-path churn amplifying failures.

Related errors

• no such host
• resolution against server
• exchange
• unsupported network
• lookup IP addresses

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/21fcf375e24ea4bb. Report an issue: GitHub.

Appendix: source

Thrown at pkg/discovery/dns/miekgdns/lookup.go:76

			// we can no longer say "this name definitely doesn't
			// exist", because we did not get that answer for
			// at least one name.
			errs = append(errs, err)
			continue
		}

		if response.Rcode == dns.RcodeSuccess {
			// Outcome 1: GOLD!
			return response, nil
		}
	}

	if len(errs) == 0 {
		// Outcome 2: everyone says NXDOMAIN.
		return &dns.Msg{}, ErrNoSuchHost
	}
	// Outcome 3: boned.
	return nil, errors.Errorf("could not resolve %q: all servers responded with errors to at least one search domain. Errs %s", name, fmtErrs(errs))
}

// lookupFromAnyServer uses all configured servers to try and resolve a specific
// name.  If a viable answer is received from a server, then it is
// immediately returned, otherwise the other servers in the config are
// tried, and if none of them return a viable answer, an error is returned.
//
// A "viable answer" is one which indicates either:
//
//  1. "yes, I know that name, and here are its records of the requested type"
//     (RCODE==SUCCESS, ANCOUNT > 0);
//  2. "yes, I know that name, but it has no records of the requested type"
//     (RCODE==SUCCESS, ANCOUNT==0); or
//  3. "I know that name doesn't exist" (RCODE==NXDOMAIN).
//
// A non-viable answer is "anything else", which encompasses both various
// system-level problems (like network timeouts) and also
// valid-but-unexpected DNS responses (SERVFAIL, REFUSED, etc).

View on GitHub (pinned to 35b8b99117)