{"record":{"id":"75b9407c6f2742af","repo":"grpc/grpc-go","slug":"connection-active-but-health-check-failed-status","errorCode":null,"errorMessage":"connection active but health check failed. status=%s","messagePattern":"connection active but health check failed\\. status=(.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"health/client.go","lineNumber":113,"sourceCode":"\n\t\t\t// Reports healthy for the LBing purposes if health check is not implemented in the server.\n\t\t\tif status.Code(err) == codes.Unimplemented {\n\t\t\t\tsetConnectivityState(connectivity.Ready, nil)\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\t// Reports unhealthy if server's Watch method gives an error other than UNIMPLEMENTED.\n\t\t\tif err != nil {\n\t\t\t\tsetConnectivityState(connectivity.TransientFailure, fmt.Errorf(\"connection active but received health check RPC error: %v\", err))\n\t\t\t\tcontinue retryConnection\n\t\t\t}\n\n\t\t\t// As a message has been received, removes the need for backoff for the next retry by resetting the try count.\n\t\t\ttryCnt = 0\n\t\t\tif resp.Status == healthpb.HealthCheckResponse_SERVING {\n\t\t\t\tsetConnectivityState(connectivity.Ready, nil)\n\t\t\t} else {\n\t\t\t\tsetConnectivityState(connectivity.TransientFailure, fmt.Errorf(\"connection active but health check failed. status=%s\", resp.Status))\n\t\t\t}\n\t\t}\n\t}\n}\n","sourceCodeStart":95,"sourceCodeEnd":118,"githubUrl":"https://github.com/grpc/grpc-go/blob/0c51461d27177d997e14c642fe18c11668fc09a3/health/client.go#L95-L118","documentation":"The health-check Watch RPC succeeded and returned a HealthCheckResponse whose Status is not SERVING (i.e. NOT_SERVING, SERVICE_UNKNOWN, or UNKNOWN). The connection is up but the server is explicitly reporting unhealthy, so the client puts the subchannel into TransientFailure. Emitted at health/client.go:113.","triggerScenarios":"Server called healthServer.SetServingStatus(svc, NOT_SERVING) (e.g. during shutdown or dependency failure), or returned SERVICE_UNKNOWN because the requested service name is not tracked. The %s prints the enum value (NOT_SERVING / SERVICE_UNKNOWN / UNKNOWN).","commonSituations":"Server intentionally marks itself NOT_SERVING during graceful shutdown or readiness probes failing; service name mismatch (client asks for \"foo.Bar\" but server only sets status for \"\" or a different name) yielding SERVICE_UNKNOWN; a dependency (DB) goes down so the app flips its own health; rolling deploy flipping status.","solutions":["On the server, ensure healthServer.SetServingStatus(<service>, SERVING) is called once the service is ready; for whole-server health use the empty service name \"\".","Match the service name on both sides: client config service_name must equal what the server sets.","If NOT_SERVING is expected during shutdown, treat it as a drain signal in your load balancer; the client will retry when status flips back.","For SERVICE_UNKNOWN, register a Health server and explicitly set status for the requested service, or have the client use \"\" for overall server health.","Investigate downstream dependencies if the app is flipping to NOT_SERVING unexpectedly."],"exampleFix":"// before: server never sets serving status\nhs := healthpb.NewServer()\nhealthsvc.RegisterHealthServer(srv, hs)\n\n// after\nhs := healthpb.NewServer()\nhs.SetServingStatus(\"\", healthpb.HealthCheckResponse_SERVING)\nhs.SetServingStatus(\"foo.Bar\", healthpb.HealthCheckResponse_SERVING)\nhealthsvc.RegisterHealthServer(srv, hs)","handlingStrategy":"fallback","validationCode":"// Verify the server reports SERVING for the service the client will request.\nimport healthpb \"google.golang.org/grpc/health/grpc_health_v1\"\n\nresp, err := healthpb.NewHealthClient(conn).Check(ctx, &healthpb.HealthCheckRequest{Service: svc})\nif err != nil { return err }\nif resp.Status != healthpb.HealthCheckResponse_SERVING {\n    return fmt.Errorf(\"server reports %s for %q; do not send traffic\", resp.Status, svc)\n}","typeGuard":null,"tryCatchPattern":"// The client already maps non-SERVING to TransientFailure. In your app, fall back to another replica or shed load.\nif state := conn.GetState(); state == connectivity.TransientFailure {\n    // route around this backend; retry will resume when status flips to SERVING\n}","preventionTips":["Call healthServer.SetServingStatus(\"\", SERVING) at server startup and flip to NOT_SERVING only during graceful drain.","Use the same service name on client (service_name) and server (SetServingStatus).","Register a Health server even if you only track overall server health (empty service name)."],"tags":["health-check","client","load-balancing","serving-status"],"backgroundTag":null,"analyzedSha":"0c51461d27177d997e14c642fe18c11668fc09a3","analyzedAt":"2026-08-11T14:49:15.055Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}