cilium/cilium · warning

Cannot forward proxied DNS response: %w

Error message

Cannot forward proxied DNS response: %w

What it means

Writing the (already received and processed) upstream DNS response back to the original client failed via w.WriteMsg. The client typically sees no answer and retries; note the server is still added/no success bookkeeping is skipped since the write failed.

Source

Thrown at pkg/fqdn/dnsproxy/proxy.go:1138

	scopedLog.Debug("Notifying with DNS response to original DNS query")
	if err := p.NotifyOnDNSMsg(time.Now(), ep, epIPPort, targetServerID, targetServer, responseDetails, protocol, true, &stat); err != nil {
		scopedLog.Error(
			"Failed to process DNS response",
			logfields.Error, err,
			logfields.Response, response,
		)
		p.sendErrorResponse(scopedLog, w, request, false)
		return
	}

	scopedLog.Debug("Responding to original DNS query")
	// Ensure the ID matches the initial request - the upstream query may have changed the ID to avoid duplicates.
	response.Id = requestID
	response.Compress = p.EnableDNSCompression && shouldCompressResponse(request, response)
	err = w.WriteMsg(response)
	if err != nil {
		scopedLog.Error("Cannot forward proxied DNS response", logfields.Error, err)
		stat.Err = fmt.Errorf("Cannot forward proxied DNS response: %w", err)
		p.NotifyOnDNSMsg(time.Now(), ep, epIPPort, targetServerID, targetServer, responseDetails, protocol, true, &stat)
	} else {
		p.Lock()
		// Add the server to the set of used DNS servers. This set is never GCd, but is limited by set
		// of DNS server IPs that are allowed by a policy and for which successful response was received.
		p.usedServers[targetServer.Addr()] = struct{}{}
		p.Unlock()
	}
}

func (p *DNSProxy) enforceConcurrencyLimit(ctx context.Context) error {
	if p.ConcurrencyGracePeriod == 0 {
		// No grace time configured. Failing to acquire semaphore means
		// immediately give up.
		if !p.ConcurrencyLimit.TryAcquire(1) {
			return ErrFailedAcquireSemaphore{
				parallel: option.Config.DNSProxyConcurrencyLimit,
			}

View on GitHub (pinned to ac7b90affa)

Solutions

1. Check for client-side resolver timeouts: if upstream latency is near client timeout, lower ProxyForwardTimeout or speed up the upstream.
2. Verify NAT/conntrack is not dropping return traffic (conntrack -L, check DROP counters).
3. Enable DNS compression (EnableDNSCompression) to shrink large responses under UDP size limits.
4. Look for EDNS0/truncation handling: ensure clients use EDNS or fall back to TCP for large answers.
5. Inspect the wrapped error: 'connection refused'/'no such file' implies client vanished; retry behavior is on the client side.

Example fix

// before: large responses dropped over UDP
// after: enable compression in proxy construction
dnsproxy.NewDNSProxy(..., dnsproxy.EnableDNSCompression) // Compress=true for >512B responses

// Verify the client is still reachable before a long upstream wait:
if tcpConn, ok := w.(interface{ Close() error }); ok {
    _ = tcpConn // for TCP, detect closed conns via read deadlines / keepalives
}

func isUDPWriter(w dns.ResponseWriter) bool {
    return strings.HasPrefix(w.RemoteAddr().Network(), "udp")
}

if err := w.WriteMsg(response); err != nil {
    // client likely gone; do not retry - rely on client resolver re-querying
    log.Warn("client write failed; client probably abandoned query", "err", err)
}

When it happens

Trigger: w.WriteMsg(response) errors after a successful upstream exchange - client went away (UDP client timed out and freed the port, TCP connection closed by peer), connection reset, or the underlying socket buffer/MTU rejects the packet (oversized UDP response with truncation handling issues).

Common situations: Slow upstream exceeding the client resolver's own timeout so the client abandons the query; NAT conntrack entry expired dropping the return packet; TCP RST because the client closed the connection; response too large for UDP path and truncation mishandled.

Related errors

• cannot send error response (rcode=%d): %w
• error opening dns proxy socket(s): %w
• failed to ack DNS proxy port: %w
• failed to parse DNS target server address: %w
• unknown protocol %q

AI-assisted analysis of cilium/cilium@ac7b90affa (2026-08-31). Data as JSON: /api/errors/8c12cbb834edb800. Report an issue: GitHub.