{"record":{"id":"bbe378a3ec3662b6","repo":"thanos-io/thanos","slug":"failed-to-dial-peer-s","errorCode":null,"errorMessage":"failed to dial peer %s","messagePattern":"failed to dial peer (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/receive/writecapnp/client.go","lineNumber":41,"sourceCode":")\n\ntype Dialer interface {\n\tDialContext(ctx context.Context) (net.Conn, error)\n}\n\ntype TCPDialer struct {\n\taddress string\n}\n\nfunc NewTCPDialer(address string) *TCPDialer {\n\treturn &TCPDialer{address: address}\n}\n\nfunc (t TCPDialer) DialContext(ctx context.Context) (net.Conn, error) {\n\tvar d net.Dialer\n\tconn, err := d.DialContext(ctx, \"tcp\", t.address)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"failed to dial peer %s\", t.address)\n\t}\n\treturn conn, nil\n}\n\ntype RemoteWriteClient struct {\n\tmu sync.Mutex\n\n\tdialer Dialer\n\tconn   *rpc.Conn\n\n\twriter Writer\n\tlogger log.Logger\n}\n\nfunc NewRemoteWriteClient(dialer Dialer, logger log.Logger) *RemoteWriteClient {\n\treturn &RemoteWriteClient{\n\t\tdialer: dialer,\n\t\tlogger: logger,","sourceCodeStart":23,"sourceCodeEnd":59,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/receive/writecapnp/client.go#L23-L59","documentation":"TCPDialer.DialContext wraps the net.Dialer error when establishing the TCP connection to the capnp remote-write peer fails, adding the peer address for diagnosis. The connection never opened, so no capnp RPC was attempted.","triggerScenarios":"Any call chain reaching RemoteWriteClient that must (re)connect: the target address is unreachable, refuses connections, DNS fails, or the context is canceled/deadline exceeded during dial.","commonSituations":"Wrong store-gateway/receive address in config; peer down or restarted; firewall or NetworkPolicy blocking the port; DNS misconfiguration in Kubernetes; dial timeout too small.","solutions":["Verify the configured address (host:port) is correct and the peer is listening (nc/teleport test)","Check DNS resolution and Kubernetes service/endpoints","Inspect network policies/firewalls between sender and receiver","Increase dial timeout or check the context deadline; examine the wrapped underlying cause (connection refused vs timeout vs no such host)"],"exampleFix":"// before\nclient := writecapnp.NewRemoteWriteClient(...address: \"store-gw:19391\"...) // typo in port\n// after\n// use the correct capnp receive port from the peer's config\nclient := writecapnp.NewRemoteWriteClient(...address: \"store-gw:19390\"...)","handlingStrategy":"retry","validationCode":"// pre-flight connectivity check before dialing through the client:\nconn, err := net.DialTimeout(\"tcp\", host, 2*time.Second)\nif err != nil {\n    return fmt.Errorf(\"peer %s unreachable before write: %w\", host, err)\n}\nconn.Close()","typeGuard":"func isDialError(err error) bool {\n    return strings.Contains(err.Error(), \"failed to dial peer\")\n}","tryCatchPattern":"_, err := client.RemoteWrite(ctx, in)\nif err != nil && strings.Contains(err.Error(), \"failed to dial peer\") {\n    if nerr, ok := err.(net.Error); ok && nerr.Timeout() { /* backoff longer */ }\n    return retryWithBackoff(ctx, writeFn)\n}","preventionTips":["Validate peer address/port in config at startup","Use service DNS names, not pinned pod IPs","Add readiness checks before sending","Set generous dial/context timeouts"],"tags":["network","tcp","dial","capnp"],"backgroundTag":"connection-refused","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}