{"record":{"id":"fd8863f98e03948d","repo":"hyperledger/fabric","slug":"communication-has-been-shut-down-fd8863","errorCode":null,"errorMessage":"communication has been shut down","messagePattern":"communication has been shut down","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"orderer/common/cluster/commauth.go","lineNumber":48,"sourceCode":"\n\tLock           sync.RWMutex\n\tshutdown       bool\n\tshutdownSignal chan struct{}\n\n\tChan2Members MembersByChannel\n\tConnections  *ConnectionsMgr\n\n\tSendBufferSize int\n\tNodeIdentity   []byte\n\tSigner         identity.Signer\n}\n\nfunc (ac *AuthCommMgr) Remote(channel string, id uint64) (*RemoteContext, error) {\n\tac.Lock.RLock()\n\tdefer ac.Lock.RUnlock()\n\n\tif ac.shutdown {\n\t\treturn nil, errors.New(\"communication has been shut down\")\n\t}\n\n\tmapping, exists := ac.Chan2Members[channel]\n\tif !exists {\n\t\treturn nil, errors.Errorf(\"channel %s doesn't exist\", channel)\n\t}\n\tstub := mapping.ByID(id)\n\tif stub == nil {\n\t\treturn nil, errors.Errorf(\"node %d doesn't exist in channel %s's membership\", id, channel)\n\t}\n\n\tif stub.Active() {\n\t\treturn stub.RemoteContext, nil\n\t}\n\n\terr := stub.Activate(ac.createRemoteContext(stub, channel))\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)","sourceCodeStart":30,"sourceCodeEnd":66,"githubUrl":"https://github.com/hyperledger/fabric/blob/2736b63f8fd5932511d56fe68b7039d15977f7f6/orderer/common/cluster/commauth.go#L30-L66","documentation":"AuthCommMgr.Remote (orderer/common/cluster/commauth.go) checks an internal shutdown flag under a read lock before performing any work; if the manager has been shut down, it refuses all remote operations with this error. It signals the comm manager's lifecycle has ended - typically during orderer termination - and further calls are invalid.","triggerScenarios":"Calling AuthCommMgr.Remote(channel, id) after Shutdown()/Close() was invoked on the manager, or while the orderer is shutting down and another goroutine still issues remote requests.","commonSituations":"Race during graceful shutdown where in-flight consensus or forwarding calls hit Remote after shutdown begins; tests tearing down the comm manager while background tasks still run; double-close patterns leaving dependent components calling into a dead manager.","solutions":["Ensure all callers stop issuing Remote() requests before calling Shutdown() - coordinate via context cancellation or wait groups.","Restart the orderer/servicer; after shutdown the manager cannot be revived, so a new instance is required.","Audit goroutine lifecycle so background tasks are joined/stopped before the comm manager is shut down.","In tests, close the manager only after all dependent operations complete (defer close last or use sync.WaitGroup)."],"exampleFix":"// before\ngo func() { commMgr.Remote(ch, id) }() // may run after Shutdown\ncommMgr.Shutdown()\n// after\nvar wg sync.WaitGroup\nwg.Add(1)\ngo func() { defer wg.Done(); commMgr.Remote(ch, id) }()\nwg.Wait() // drain callers first\ncommMgr.Shutdown()","handlingStrategy":"try-catch","validationCode":"// Go: expose/track shutdown state and check it before calling Remote\nif mgr.IsShutdown() { // if exposed; otherwise rely on the error\n    return errors.New(\"comm manager already shut down\")\n}","typeGuard":"func (ac *AuthCommMgr) Available() bool {\n    ac.Lock.RLock()\n    defer ac.Lock.RUnlock()\n    return !ac.shutdown\n}","tryCatchPattern":"remote, err := authMgr.Remote(channel, id)\nif err != nil {\n    if err.Error() == \"communication has been shut down\" {\n        // manager closed; recreate or propagate termination\n        return errShutdown\n    }\n    return err\n}","preventionTips":["Cancel dependent goroutines (via context) before calling Shutdown() on the comm manager.","Use sync.WaitGroup or errgroup to drain in-flight Remote calls before shutdown.","In tests, defer Shutdown() last so all operations complete first.","Never reuse a shut-down manager; construct a fresh instance if the service resumes."],"tags":["fabric","orderer","cluster","shutdown","lifecycle"],"backgroundTag":"communication-shut-down","analyzedSha":"2736b63f8fd5932511d56fe68b7039d15977f7f6","analyzedAt":"2026-09-04T08:52:36.465Z","contentChangedAt":"2026-09-04T08:52:36.465Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}