{"record":{"id":"b61b1125303f00ee","repo":"openimsdk/open-im-server","slug":"failed-to-initialize-connections-for-service-s","errorCode":null,"errorMessage":"failed to initialize connections for service %s: %v","messagePattern":"failed to initialize connections for service (.+?): (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/common/discovery/kubernetes/kubernetes.go","lineNumber":110,"sourceCode":"\tk.mu.RLock()\n\n\tconns, exists := k.connMap[serviceName]\n\tk.mu.RUnlock()\n\tif exists {\n\t\treturn conns, nil\n\t}\n\n\tk.mu.Lock()\n\t// Check if another goroutine has already initialized the connections when we released the read lock\n\tconns, exists = k.connMap[serviceName]\n\tif exists {\n\t\treturn conns, nil\n\t}\n\tk.mu.Unlock()\n\n\tif err := k.initializeConns(serviceName); err != nil {\n\t\tfmt.Println(\"Failed to initialize connections:\", err)\n\t\treturn nil, fmt.Errorf(\"failed to initialize connections for service %s: %v\", serviceName, err)\n\t}\n\n\treturn k.connMap[serviceName], nil\n}\n\n// GetConn returns a single gRPC client connection for a given Kubernetes service name.\nfunc (k *KubernetesConnManager) GetConn(ctx context.Context, serviceName string, opts ...grpc.DialOption) (*grpc.ClientConn, error) {\n\tvar target string\n\n\tif k.rpcTargets[serviceName] == \"\" {\n\t\tvar err error\n\n\t\tsvcPort, err := k.getServicePort(serviceName)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\ttarget = fmt.Sprintf(\"%s.%s.svc.cluster.local:%d\", serviceName, k.namespace, svcPort)","sourceCodeStart":92,"sourceCodeEnd":128,"githubUrl":"https://github.com/openimsdk/open-im-server/blob/175a7bb0673eca18e9d1b10bff4f728da6b1b513/pkg/common/discovery/kubernetes/kubernetes.go#L92-L128","documentation":"GetConns lazily initializes the connection pool for a service: if no connections exist yet it calls initializeConns and, on any failure, wraps it in this error naming the service. It is the top-level error callers of discovery see when Kubernetes-based connection setup fails for any reason.","triggerScenarios":"First GetConns(serviceName) call when connMap has no entry and initializeConns returns an error (endpoint Get failure, dial failure, service port lookup failure).","commonSituations":"Calling GetConns before the service/endpoints exist in the cluster; RBAC or namespace misconfiguration; cascades from errors 12/13/15/16.","solutions":["Inspect the wrapped %v cause — it is one of: endpoint get failure, dial failure, or port lookup failure; fix that root cause.","Verify the service is deployed and has ready endpoints before starting clients: kubectl get endpoints <serviceName>.","Add retry with backoff around GetConns since endpoints may appear shortly after service creation.","Check the earlier 'Failed to initialize connections:' log line for the inner error."],"exampleFix":"// before\nconns, err := mgr.GetConns(serviceName)\n// after\nvar conns []*grpc.ClientConn\nerr := retry.OnError(wait.Backoff{Steps: 5, Duration: time.Second}, func(err error) bool { return true }, func() error {\n    conns, err = mgr.GetConns(serviceName)\n    return err\n})","handlingStrategy":"retry","validationCode":"eps, err := clientset.CoreV1().Endpoints(ns).Get(ctx, svcName, metav1.GetOptions{})\nready := err == nil && len(eps.Subsets) > 0","typeGuard":null,"tryCatchPattern":"conns, err := mgr.GetConns(serviceName)\nif err != nil {\n    // retry with backoff; error wraps port/dial/endpoint causes\n    return fmt.Errorf(\"conns for %s: %w\", serviceName, err)\n}","preventionTips":["Gate client startup on the service having ready endpoints","Use exponential backoff around GetConns during rollouts","Log errors.Unwrap(err) to distinguish port vs endpoint vs dial failures"],"tags":["kubernetes","grpc","service-discovery"],"backgroundTag":"service-discovery-failed","analyzedSha":"175a7bb0673eca18e9d1b10bff4f728da6b1b513","analyzedAt":"2026-09-04T16:52:56.821Z","contentChangedAt":"2026-09-04T16:52:56.821Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}