{"record":{"id":"ec2e6468471dba04","repo":"kopia/kopia","slug":"unable-to-acquire-semaphore","errorCode":null,"errorMessage":"unable to acquire semaphore","messagePattern":"unable to acquire semaphore","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"internal/server/grpc_session.go","lineNumber":145,"sourceCode":"\n\t\tfor req, err := srv.Recv(); err == nil; req, err = srv.Recv() {\n\t\t\t// propagate any error from the goroutines\n\t\t\tselect {\n\t\t\tcase err := <-lastErr:\n\t\t\t\tuserLog(ctx).Errorf(\"error handling session request: %v\", err)\n\n\t\t\t\tcontentlog.Log1(ctx, log,\n\t\t\t\t\t\"error handling session request\",\n\t\t\t\t\tlogparam.Error(\"error\", err))\n\n\t\t\t\treturn err\n\n\t\t\tdefault:\n\t\t\t}\n\n\t\t\t// enforce limit on concurrent handling\n\t\t\tif err := s.sem.Acquire(ctx, 1); err != nil {\n\t\t\t\treturn errors.Wrap(err, \"unable to acquire semaphore\")\n\t\t\t}\n\n\t\t\tgo func() {\n\t\t\t\tdefer s.sem.Release(1)\n\n\t\t\t\ts.handleSessionRequest(ctx, dw, authz, usernameAtHostname, req, func(resp *grpcapi.SessionResponse) {\n\t\t\t\t\tif err := s.send(srv, req.GetRequestId(), resp); err != nil {\n\t\t\t\t\t\tselect {\n\t\t\t\t\t\tcase lastErr <- err:\n\t\t\t\t\t\tdefault:\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t})\n\t\t\t}()\n\t\t}\n\n\t\treturn nil\n\t})","sourceCodeStart":127,"sourceCodeEnd":163,"githubUrl":"https://github.com/kopia/kopia/blob/82495e54b584c1ef6073c9e1be048f57f8aef078/internal/server/grpc_session.go#L127-L163","documentation":"Thrown in the gRPC session server when s.sem.Acquire(ctx, 1) fails while enforcing the limit on concurrently handled requests. Acquire only fails when the provided context is cancelled or its deadline expires while waiting for a slot, meaning the request was abandoned before a worker became available.","triggerScenarios":"Client request arrives while the concurrency semaphore is full and the client's context is cancelled or times out before a slot frees up.","commonSituations":"Many concurrent kopia client operations saturating the server; client-side timeout shorter than the queue wait; server wedged on slow handlers so the queue never drains.","solutions":["Retry the request; this often means the client gave up while waiting.","Increase the server's concurrency limit if handlers are habitually saturated.","Investigate slow/stuck handlers blocking semaphore slots.","Align client timeout settings with realistic server queue wait times."],"exampleFix":"// before: default low concurrency\nkopia server start --max-concurrency=4\n// after\nkopia server start --max-concurrency=32","handlingStrategy":"retry","validationCode":"// client-side: use a deadline long enough to cover queue wait\nctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)\ndefer cancel()","typeGuard":null,"tryCatchPattern":"err := client.Connect(ctx)\nif err != nil && strings.Contains(err.Error(), \"unable to acquire semaphore\") {\n    // server saturated or client timed out waiting; back off and retry\n    return retryWithBackoff(ctx, Connect)\n}","preventionTips":["Raise server concurrency limits to match client load","Use generous client deadlines for bursty workloads","Monitor server saturation and slow handlers","Apply client-side backoff instead of hammering a saturated server"],"tags":["grpc","concurrency","semaphore","timeout"],"backgroundTag":"request-timeout","analyzedSha":"82495e54b584c1ef6073c9e1be048f57f8aef078","analyzedAt":"2026-09-07T20:35:21.689Z","contentChangedAt":"2026-09-07T20:35:21.689Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}