What causes "recursively resolve %s" in thanos-io/thanos?

Inspect the nested root cause after 'recursively resolve :' in the message to pick the right fix

recursively resolve

Error message

recursively resolve %s

What it means

Wraps an error from the recursive lookupSRV call made when an answer record is a CNAME instead of a SRV: the resolver re-resolves addr.Target with empty service/proto and, if that nested lookup fails, attaches 'recursively resolve <target>' context to the underlying error (which may itself be error 567, 562, etc.).

Solutions

Inspect the nested root cause after 'recursively resolve <target>:' in the message to pick the right fix
Verify the CNAME target exists: dig SRV <target>
Remove or update the dangling/looping CNAME record in your DNS zone
Ensure the target's zone/search domain is correctly configured for your resolver

Defensive patterns

Strategy: try-catch

Validate before calling

// verify the SRV target resolves before discovery
ans, err := lookupSRVManually(target); if err != nil { /* dangling CNAME; fix DNS */ }

Try / catch

if err != nil && strings.Contains(err.Error(), "recursively resolve") {
    logger.Warn("SRV answer contained unresolvable CNAME", "err", err)
    return usePartialResultsOrSkip()
}

Prevention

Ensure CNAME targets referenced by SRV answers always exist in DNS
Clean up CNAMEs when deleting services they point to
Use fully-qualified CNAME targets resolvable in the configured search domains
Monitor for dangling CNAMEs with periodic DNS audits

When it happens

Trigger: A SRV query answer contains a *dns.CNAME whose Target fails to resolve recursively — NXDOMAIN, server errors, or iteration limit exceeded on the nested call.

Common situations: Dangling CNAMEs pointing at deleted services; CNAME chains that exceed the recursion cap; the target name only resolving in certain search domains.

Related errors

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/2ccdbb2c2a09bf74. Report an issue: GitHub.

Appendix: source

Thrown at pkg/discovery/dns/miekgdns/resolver.go:56

	response, err := r.lookupWithSearchPath(target, dns.Type(dns.TypeSRV))
	if err != nil {
		return "", nil, err
	}

	for _, record := range response.Answer {
		switch addr := record.(type) {
		case *dns.SRV:
			addrs = append(addrs, &net.SRV{
				Weight:   addr.Weight,
				Target:   addr.Target,
				Priority: addr.Priority,
				Port:     addr.Port,
			})
		case *dns.CNAME:
			// Recursively resolve it.
			_, resp, err := r.lookupSRV("", "", addr.Target, currIteration+1, maxIterations)
			if err != nil {
				return "", nil, errors.Wrapf(err, "recursively resolve %s", addr.Target)
			}
			addrs = append(addrs, resp...)
		default:
			return "", nil, errors.Errorf("invalid SRV response record %s", record)
		}
	}

	return "", addrs, nil
}

func (r *Resolver) LookupIPAddr(_ context.Context, host string) ([]net.IPAddr, error) {
	return r.lookupIPAddr(host, 1, 8)
}

func (r *Resolver) LookupIPAddrByNetwork(ctx context.Context, network, host string) ([]net.IPAddr, error) {
	var qtype dns.Type
	switch network {
	case "ip6":

View on GitHub (pinned to 35b8b99117)