What causes "got truncated message on TCP (64kiB limit exceeded?)" in thanos-io/thanos?

Reduce the number of records behind the queried name (split the service, shrink the SRV set)

got truncated message on TCP (64kiB limit exceeded?)

Error message

got truncated message on TCP (64kiB limit exceeded?)

What it means

Raised when a DNS response arrives with the Truncated (TC) bit set even over TCP, where truncation should be impossible under normal conditions (64KiB message limit exceeded). Since TCP was already used as the fallback for UDP truncation, there is no further downgrade and the lookup fails outright.

Solutions

Reduce the number of records behind the queried name (split the service, shrink the SRV set)
Check whether EDNS0/advertising larger UDP sizes is misconfigured on the server
Inspect the DNS server logs for why it truncates a TCP answer
Use a different resolution mechanism (e.g. smaller record TTLs/trees, or query with a more specific name)

Defensive patterns

Strategy: fallback

Validate before calling

// estimate answer size beforehand
n, _ := countSRVRecords(name) // if huge, avoid one giant SRV query
if n > 1000 { /* split queries or use pagination via multiple names */ }

Try / catch

if err != nil && strings.Contains(err.Error(), "truncated message on TCP") {
    return fallbackToSmallerQueries(name) // e.g. query subsets of the record set
}

Prevention

Keep SRV/record sets per name small (hundreds, not thousands of targets)
Prefer many smaller service names over one mega-service with huge record sets
Ensure the DNS server supports EDNS0 and large answers correctly
Alert on growing record counts behind discovery targets

When it happens

Trigger: askServerForName retries over TCP after a truncated UDP reply, but the TCP response is still flagged Truncated — typically a response exceeding the 64KiB DNS message maximum.

Common situations: Extremely large SRV record sets (thousands of targets) behind one service name; DNS zone transfer-like responses; misbehaving DNS server setting TC incorrectly; EDNS0 size caps not helping.

Related errors

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/19d1321089dc11bf. Report an issue: GitHub.

Appendix: source

Thrown at pkg/discovery/dns/miekgdns/lookup.go:145

// name (and qtype). Retries with TCP in the event of response truncation,
// but otherwise just sends back whatever the server gave, whether that be a
// valid-looking response, or an error.
func askServerForName(name string, qType dns.Type, client *dns.Client, servAddr string, edns bool) (*dns.Msg, error) {
	msg := &dns.Msg{}

	msg.SetQuestion(dns.Fqdn(name), uint16(qType))
	if edns {
		msg.SetEdns0(dns.DefaultMsgSize, false)
	}

	response, _, err := client.Exchange(msg, servAddr)
	if err != nil {
		return nil, errors.Wrapf(err, "exchange")
	}

	if response.Truncated {
		if client.Net == "tcp" {
			return nil, errors.New("got truncated message on TCP (64kiB limit exceeded?)")
		}

		// TCP fallback.
		client.Net = "tcp"
		return askServerForName(name, qType, client, servAddr, false)
	}

	return response, nil
}

View on GitHub (pinned to 35b8b99117)