{"record":{"id":"f8714ec88e62a348","repo":"hyperledger/fabric","slug":"connection-to-d-s-is-in-state-s-f8714e","errorCode":null,"errorMessage":"connection to %d(%s) is in state %s","messagePattern":"connection to (.+?)\\((.+?)\\) is in state (.+?)","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"orderer/common/cluster/commauth.go","lineNumber":173,"sourceCode":"\n\t// Activate the stub\n\tstub.Activate(ac.createRemoteContext(stub, channel))\n}\n\nfunc (ac *AuthCommMgr) createRemoteContext(stub *Stub, channel string) func() (*RemoteContext, error) {\n\treturn func() (*RemoteContext, error) {\n\t\tac.Logger.Debugf(\"Connecting to node: %v for channel: %v\", stub.RemoteNode.NodeAddress, channel)\n\n\t\tconn, err := ac.Connections.Connect(stub.Endpoint, stub.RemoteNode.ServerRootCA)\n\t\tif err != nil {\n\t\t\tac.Logger.Warningf(\"Unable to obtain connection to %d(%s) (channel %s): %v\", stub.ID, stub.Endpoint, channel, err)\n\t\t\treturn nil, err\n\t\t}\n\n\t\tprobeConnection := func(conn *grpc.ClientConn) error {\n\t\t\tconnState := conn.GetState()\n\t\t\tif connState == connectivity.Connecting {\n\t\t\t\treturn errors.Errorf(\"connection to %d(%s) is in state %s\", stub.ID, stub.Endpoint, connState)\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\n\t\tclusterClient := orderer.NewClusterNodeServiceClient(conn)\n\t\tgetStepClientStream := func(ctx context.Context) (StepClientStream, error) {\n\t\t\tstream, err := clusterClient.Step(ctx)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\tmembersMapping, exists := ac.Chan2Members[channel]\n\t\t\tif !exists {\n\t\t\t\treturn nil, errors.Errorf(\"channel members not initialized\")\n\t\t\t}\n\t\t\tnodeStub := membersMapping.LookupByIdentity(ac.NodeIdentity)\n\t\t\tif nodeStub == nil {\n\t\t\t\treturn nil, errors.Errorf(\"node identity is missing in channel\")","sourceCodeStart":155,"sourceCodeEnd":191,"githubUrl":"https://github.com/hyperledger/fabric/blob/2736b63f8fd5932511d56fe68b7039d15977f7f6/orderer/common/cluster/commauth.go#L155-L191","documentation":"During stub activation, probeConnection checks the gRPC ClientConn state and rejects it if it is still in connectivity.Connecting - meaning the TCP/TLS connection to the remote cluster node was initiated but the handshake has not completed yet. The RemoteContext's ProbeConn is invoked before the connection is considered usable, so half-open connections are not handed to consensus.","triggerScenarios":"createRemoteContext successfully obtained a ClientConn via Connections.Connect, but conn.GetState() returned connectivity.Connecting when probed - typically when the connection attempt is still in flight (slow network, remote peer not yet listening, TLS handshake pending) at the moment the stub is activated and probed.","commonSituations":"Remote orderer is down or still starting; network latency or DNS slowness delaying the gRPC connect; TLS certificate problems causing a long/hanging handshake; transient race right after channel configure when stubs are activated en masse.","solutions":["Check the remote orderer at that endpoint is running and its cluster (port 7050) listener is up: verify stub.Endpoint host:port.","Retry after a short delay - the state is transient and will become Ready or the underlying Connect will fail with a definitive error.","Verify TLS certs (ServerRootCA vs the remote server cert) and that the gRPC connection is not stuck in backoff due to failed handshakes.","Check network/DNS connectivity between orderers (firewall rules on the cluster port)."],"exampleFix":"// before\nconn, _ := connections.Connect(endpoint, ca) // may be Connecting\nrc, err := remoteContext() // probe fails: state Connecting\n// after\n// retry with backoff until state is Ready or a hard error surfaces\nfor attempts := 0; attempts < 5; attempts++ {\n    if conn.GetState() == connectivity.Ready { break }\n    time.Sleep(backoff)\n}\nrc, err := remoteContext()","handlingStrategy":"retry","validationCode":"func connectionReady(conn *grpc.ClientConn) bool {\n    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\n    defer cancel()\n    return conn.WaitForStateChange(ctx, conn.GetState()) && conn.GetState() == connectivity.Ready\n}","typeGuard":"func isConnecting(state connectivity.State) bool {\n    return state == connectivity.Connecting\n}","tryCatchPattern":"rc, err := comm.Remote(channel, id)\nif err != nil {\n    if strings.Contains(err.Error(), \"is in state Connecting\") {\n        // transient: probe caught the conn mid-handshake; retry with backoff\n        time.Sleep(500 * time.Millisecond)\n        rc, err = comm.Remote(channel, id)\n    }\n    return err\n}","preventionTips":["Pre-warm connections (Connect ahead of activation) so probes see Ready instead of Connecting.","Monitor gRPC connectivity state transitions and backoff on the cluster port.","Verify remote orderers are up and TLS handshakes complete quickly (check certs, clock skew).","Apply bounded exponential backoff retries around Remote()/stream creation during startup windows."],"tags":["grpc","connectivity","network","hyperledger-fabric"],"backgroundTag":"grpc-connection-connecting","analyzedSha":"2736b63f8fd5932511d56fe68b7039d15977f7f6","analyzedAt":"2026-09-04T08:52:36.465Z","contentChangedAt":"2026-09-04T08:52:36.465Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}