{"record":{"id":"7cbc25e55ca9d1bd","repo":"hyperledger/fabric","slug":"too-many-requests-for-s-exceeding-concurrency-li","errorCode":null,"errorMessage":"too many requests for %s, exceeding concurrency limit (%d)","messagePattern":"too many requests for (.+?), exceeding concurrency limit \\((.+?)\\)","errorType":"http","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"internal/peer/node/grpc_limiters.go","lineNumber":53,"sourceCode":"\t}\n\tif gatewayConcurrency != 0 {\n\t\tlogger.Infof(\"concurrency limit for gateway service is %d\", gatewayConcurrency)\n\t\tsemaphores[\"/gateway.Gateway\"] = semaphore.New(gatewayConcurrency)\n\t}\n\n\treturn semaphores\n}\n\nfunc unaryGrpcLimiter(semaphores map[string]semaphore.Semaphore) grpc.UnaryServerInterceptor {\n\treturn func(ctx context.Context, req any, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (any, error) {\n\t\tserviceName := getServiceName(info.FullMethod)\n\t\tsema, ok := semaphores[serviceName]\n\t\tif !ok {\n\t\t\treturn handler(ctx, req)\n\t\t}\n\t\tif !sema.TryAcquire() {\n\t\t\tlogger.Errorf(\"Too many requests for %s, exceeding concurrency limit (%d)\", serviceName, cap(sema))\n\t\t\treturn nil, errors.Errorf(\"too many requests for %s, exceeding concurrency limit (%d)\", serviceName, cap(sema))\n\t\t}\n\t\tdefer sema.Release()\n\t\treturn handler(ctx, req)\n\t}\n}\n\nfunc streamGrpcLimiter(semaphores map[string]semaphore.Semaphore) grpc.StreamServerInterceptor {\n\treturn func(srv any, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {\n\t\tserviceName := getServiceName(info.FullMethod)\n\t\tsema, ok := semaphores[serviceName]\n\t\tif !ok {\n\t\t\treturn handler(srv, ss)\n\t\t}\n\t\tif !sema.TryAcquire() {\n\t\t\tlogger.Errorf(\"Too many requests for %s, exceeding concurrency limit (%d)\", serviceName, cap(sema))\n\t\t\treturn errors.Errorf(\"too many requests for %s, exceeding concurrency limit (%d)\", serviceName, cap(sema))\n\t\t}\n\t\tdefer sema.Release()","sourceCodeStart":35,"sourceCodeEnd":71,"githubUrl":"https://github.com/hyperledger/fabric/blob/2736b63f8fd5932511d56fe68b7039d15977f7f6/internal/peer/node/grpc_limiters.go#L35-L71","documentation":"A unary gRPC server interceptor on the peer enforces a per-service concurrency limit using a semaphore. When a request arrives and no semaphore slot is available (TryAcquire fails), the interceptor logs and rejects the call with this error instead of invoking the handler, protecting the peer from overload.","triggerScenarios":"Sending a unary RPC to a peer service (e.g. endorsement, deliver) whose configured limiter semaphore is already at capacity with concurrent in-flight requests.","commonSituations":"Load spikes or benchmarking against the peer without tuning vm/chaincode or deliver concurrency limits; many SDK clients hammering the peer simultaneously; leaking streams that keep concurrency slots occupied.","solutions":["Retry the request with backoff; the error is transient and clears when in-flight requests finish.","Increase the peer's gRPC concurrency limit configuration for that service if load is legitimately high.","Reduce client concurrency / add client-side rate limiting or load balancing across multiple peers."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// client-side: cap concurrent in-flight unary RPCs below the peer's configured limiter\nsem := make(chan struct{}, maxClientConcurrency)\nsem <- struct{}{}\ndefer func() { <-sem }()","typeGuard":null,"tryCatchPattern":"for attempt := 0; attempt < 5; attempt++ {\n    _, err := client.Invoke(ctx, req)\n    if err != nil && strings.Contains(err.Error(), \"exceeding concurrency limit\") {\n        time.Sleep(backoff(attempt))\n        continue\n    }\n    return err\n}","preventionTips":["Load-test against the peer's configured concurrency limits.","Add client-side rate limiting and exponential backoff.","Monitor peer in-flight RPC metrics and scale peers horizontally before limits are hit."],"tags":["fabric","grpc","rate-limiting","concurrency"],"backgroundTag":"grpc-concurrency-limit-exceeded","analyzedSha":"2736b63f8fd5932511d56fe68b7039d15977f7f6","analyzedAt":"2026-09-04T08:52:36.465Z","contentChangedAt":"2026-09-04T08:52:36.465Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}